Senior Data Engineer

Company: Citigroup
Location: Irving
Posted on: February 22, 2021

Job Description:

Responsibilities:

• Integral team member of our AI and Analytics team responsible for design and development of Big data solutions Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop or Google Cloud Platform Responsible for delivering data as a service framework from Google Cloud Platform
• Responsible for moving all legacy workloads to cloud platform
• Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
• Ensure automation through CI/CD across platforms both in cloud and on-premises
• Ability to research and assess open source technologies and components to recommend and integrate into the design and implementation
• Be the technical expert and mentor other team members on Big Data and Cloud Tech stacks
• Define needs around maintainability, testability, performance, security, quality and usability for data platform
• Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
• Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems
• Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance
• Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.
• Supervise day-to-day staff management issues, including resource management, work allocation, mentoring/coaching and other duties and functions as assigned
• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparencyQualifications:
  • 8+ years of experience in hadoop/big data technologies.
  • 3+ years of experience in spark.
  • 1+ year of experience working on Google or AWS cloud developing data solutions. Certifications preferred.
  • Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;
  • Experience with containerization and related technologies (e.g. Docker, Kubernetes)
  • Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)
  • 1 year Hadoop administration experience preferred
  • 1+ year of SAS experience preferred.
  • Comprehensive knowledge of the principles of software engineering and data analytics
  • Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)
  • Knowledge of agile(scrum) development methodology is a plus
  • Strong development/automation skills
  • Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.
  • System level understanding - Data structures, algorithms, distributed storage & compute
  • Can-do attitude on solving complex business problems, good interpersonal and teamwork skills Education:
    • Bachelor's degree/University degree or equivalent experience
    • Master's degree preferredThis job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required. ------------------------------------------------- Job Family Group: Technology ------------------------------------------------- Job Family:Data Analytics ------------------------------------------------------ Time Type: ------------------------------------------------------ Citi is an equal opportunity and affirmative action employer.Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.Citigroup Inc. and its subsidiaries ("Citi ) invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi .View the " EEO is the Law " poster. View the EEO is the Law Supplement .View the EEO Policy Statement .View the Pay Transparency Posting - provided by Dice

Keywords: Citigroup, Irving , Senior Data Engineer, Engineering , Irving, Texas