Data Engineer- Databricks
Headquartered in Dublin, Ohio, Cardinal Health, Inc. (NYSE: CAH) is a global, integrated healthcare services and products company connecting patients, providers, payers, pharmacists and manufacturers for integrated care coordination and better patient management. Backed by nearly 100 years of experience, with more than 50,000 employees in nearly 60 countries, Cardinal Health ranks among the top 20 on the Fortune 500.
Job Overview:
The Data Engineer - Databricks will be responsible for building and optimizing the data pipelines, architectures, and data sets. You will work closely with Business users, data analysts, data scientists and other engineers to support their data needs and maximize the value of our data processing capabilities.
Responsibilities:
- Design, develop, and maintain scalable and robust data pipelines on Databricks.
- Collaborate with analysts and business to understand data requirements and deliver solutions.
- Optimize and troubleshoot existing data pipelines for performance and reliability.
- Ensure data quality and integrity across various data sources.
- Implement data security and compliance best practices.
- Monitor data pipeline performance and conduct necessary maintenance and updates.
- Document data pipeline processes and technical specifications. Optimizing data pipelines for performance and cost for data pipelines.
Desired Qualifications:
- Bachelor's degree in engineering or equivalent work experience
- 5+ years of engineering experience in Big Data systems, Data Analytics and Data Integration related fields
- 3+ years of hands-on experience working with Databricks and Spark.
- Experience in working in databricks notebooks
- Wellversed in Databricks SQL including performance turning, Data aggregations and Statistical functions.
- Experience with Databricks Workflow – Scheduling, Error debug, Repairing failed runs.
- Experience with CI/CD pipelines on Databricks with usage of different tools.
- Experience of exploring and transforming large datasets using information schema and Databricks Catalog.
- Ability to read and understand complex Microsoft SQL server Stored procedures.
- Prior experience of writing complex SQL queries and Python scripts.
- Experience in working with Cloud Data Engineering Platforms (E.g. Google, AWS or Azure) and Cloud Analytics solutions are preferred.
- Experience in designing and optimizing data models on GCP cloud using GCP data stores such as Big Query is preferred.
- Knowledge of data analytics, data warehousing, data engineering and ETL jobs.
- Agile development skills and experience.
- Databricks and Google Cloud Platform certifications are a plus
Candidates who are back-to-work, people with disabilities, without a college degree, and Veterans are encouraged to apply.
Cardinal Health supports an inclusive workplace that values diversity of thought, experience and background. We celebrate the power of our differences to create better solutions for our customers by ensuring employees can be their authentic selves each day. Cardinal Health is an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, ancestry, age, physical or mental disability, sex, sexual orientation, gender identity/expression, pregnancy, veteran status, marital status, creed, status with regard to public assistance, genetic status or any other status protected by federal, state or local law.
To read and review this privacy notice click here
