The Data Engineer will design and maintain data pipelines, build data warehouses, and ensure data quality while collaborating with data scientists and engineers.
Job Summary:
We are looking for a talented and experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, building, and maintaining our data pipelines and infrastructure. You will work closely with our data scientists and analysts to ensure that our data is accessible, reliable, and secure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines.
- Build and manage data warehouses and data lakes.
- Implement data quality and data governance best practices.
- Work with data scientists and analysts to support their research and development projects.
- Collaborate with other engineers to build and maintain our data infrastructure.
Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field
- 3-5 years of experience in Data Warehousing and Data Integration projects, with exposure to ETL processes, data pipelines, and data quality management.
- practical experience working within large scale data environments and demonstrate strong analytical and problem-solving capabilities.
- Hands-on experience in SAS, including development and optimization of SAS programs.
- Strong programming skills in Python, Java, or Scala
- Experience with big data technologies such as Hadoop, Spark, and Kafka
- Experience with cloud computing platforms such as AWS, Azure, or Google Cloud Platform
- Experience with database systems such as SQL, PostgreSQL and Cassandra
- Experience with data warehousing and ETL (extract, transform, load) processes
- Excellent problem-solving and analytical skills
- Strong communication and teamwork skills
Bonus Points:
- Experience with SAS, Kubernetes, or Docker
- Experience with machine learning and artificial intelligence
- Experience with cloud-native development
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead and execute data engineering initiatives: define metrics, instrument logging, ingest and model data, build ETL/ELT pipelines, ensure data quality and governance, optimize processing performance, enable analytics/visualization, and lead projects that drive product and business strategy.
Top Skills:
Anomaly DetectionDashboardingData ModelingData QualityData VisualizationEltETLLoggingProduct AnalyticsPythonRelational DatabasesSQL
Automotive
Design, build, and maintain scalable GCP-based data ingestion and curation pipelines using Python, SQL, DBT/Dataform and GCP services (BigQuery, Dataflow, Pub/Sub). Implement data governance, security, IaC (Terraform), orchestration (Astronomer, Tekton), performance and cost optimizations, and collaborate with cross-functional teams to translate business requirements into reliable data assets and automation.
Top Skills:
AstronomerBigQueryCi/CdCloud FunctionsCloud RunDataflowDataformDataprocDbtGoogle Cloud Platform (Gcp)Infrastructure As Code (Iac)JavaMicroservicesMySQLNoSQLPostgresPub/SubPythonService-Oriented Architecture (Soa)SQLTektonTerraform
Information Technology • Database • Consulting
Lead technical design and development of software solutions, propose designs for complex business needs, prepare technical documentation, resolve complex problems, provide technical guidance, review code/testing/docs, and apply modern methodologies and tools.
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
