Data Engineer - Observability

Company: Citibank
Location: Irving
Posted on: November 22, 2021

Job Description:

Role Description: The Observability organization at Citi is responsible to provide full stack observability solutions (metrics, events, logs, traces) to infrastructure, security, and applications SRE, operations and development teams covering over 10,000 application instances. The observability solutions utilize products like Splunk, Elasticsearch, Grafana / Prometheus, AppDynamics, NOI, and ITM6 hosted on Linux machines. These products are managed using a robust ansible based deployment model. We are looking for a data engineer to join the team with experience in data modelling, analyzing enterprise inventory technical reference data as well as granular telemetry and an appetite for architecting common data model that can be adopted by any product and solution.This is a senior level, hands-on position responsible for a variety of data engineering activities.Responsibilities:Create and maintain optimal data model and data pipeline for enterprise observability solutionProvide SRE teams with automation opportunities based on data analysisCollaborate in building event co-relations and grouping policies based on event data and reducing tickets for application and infrastructure teamsArchitect and design a common data model, standard metadata taxonomy, data pipeline and curation of data for complex enterprise observability solutions covering infrastructure, system, and security logs and metricsArchitect and implement measurement criteria in the data pipeline for completeness, timeliness, and accuracy of dataBuild processes and policies supporting data transformation, data structures, metadata, dependency, and data dictionary across the pipelineUtilize Citi's data analytics tools like Grafana and become the data analytics SME to harness available technical asset inventory data to gain insights, improve data quality and increase self-serviceQualifications:3+ years of experience as a Data EngineerTechnical expertise required in big data and cloud technologies - Kafka, spark, Hadoop, Hive, HDFS, Cloudera.ERD - entity relationship diagramsExperience with Lucene or DSL based on JSON, Splunk SPL, relational SQL and NoSQL databases, including Oracle, Postgres, Mongo or Cassandra.Experience building and optimizing large data volumes in data pipelines and architecture and data sets using Kafka, ELK and SplunkExperience with Data connectivity methodologies such as APIs (Rest/Soap), ODBC/JDBC, HTTP web hooks, JSON, etc.Working knowledge of message queuing, stream processing, and highly scalable 'big data' data storesExperience supporting and working with cross-functional teams in a dynamic environmentExcellent teamwork and proactive attitudeExperience with Python, Java, Golang, or ScalaGood to have Skills:Experience with Grafana, ELK and Splunk will be a plusExperience in Machine learning, Data Scientist to predict event, fatal issues in applications and infrastructure will be a plusEducation:Bachelor's degree/University degree or equivalent experience in Computer Science, Statistics, Information Systems, or another quantitative fieldMaster's degree is a plusJob Family Group: TechnologyJob Family:Data ArchitectureTime Type:Citi is an equal opportunity and affirmative action employer.Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.Citigroup Inc. and its subsidiaries ("Citi") invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.View the "EEO is the Law" poster. View the EEO is the Law Supplement.View the EEO Policy Statement.View the Pay Transparency PostingEffective November 1, 2021, Citi requires that all successful applicants must be fully vaccinated against COVID-19 as a condition of employment and provide proof of such vaccination prior to commencement of employment.

Keywords: Citibank, Irving , Data Engineer - Observability, Engineering , Irving, Texas