Design, build, and maintain scalable AWS-based data pipelines and ETL processes. Collaborate with data scientists, optimize storage and retrieval for performance and cost, enforce data governance and security, troubleshoot pipeline issues, and recommend new data engineering technologies.
AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have 6-8 years of prior experience in data engineering, with a strong background in AWS (Amazon Web Services) technologies. This role offers an exciting opportunity to work on diverse projects, collaborating with cross functional teams to design, build, and optimize data pipelines and infrastructure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes leveraging AWS services such as S3,
- Glue, EMR, Lambda, and Redshift.
- Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.
- Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness.
- Implement data governance and security best practices to ensure compliance and data integrity.
- Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
- Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 6-8 years of prior experience in data engineering, with a focus on designing and building data pipelines.
- Proficiency in AWS services, particularly S3, Glue, EMR, Lambda, and Redshift.
- Strong programming skills in languages such as Python, Java, or Scala.
- Experience with SQL and NoSQL databases, data warehousing concepts, and big data technologies.
- Familiarity with containerization technologies (e.g., Docker, Kubernetes) and orchestration tools (e.g., Apache Airflow) is a plus.
Similar Jobs
Artificial Intelligence • Fintech • Software • Financial Services
Lead migration from Matillion ETL to AWS-native data platforms. Design and optimize scalable pipelines using AWS Glue and Apache Airflow, build reusable ETL/ELT components, and work with Snowflake for data warehousing and performance tuning. Implement data quality, monitoring, security, governance, and compliance practices. Collaborate with architects and stakeholders, support CI/CD and DevOps processes, conduct code reviews, mentor engineers, and improve processing performance, cost, and scalability.
Top Skills:
Apache AirflowAWSAws GlueAws LambdaAws Step FunctionsCi/CdDevOpsGitMatillion EtlPysparkPythonShell ScriptingSnowflakeSnowpipeSnowsqlSparkSQL
Healthtech • Pharmaceutical • Manufacturing
Design, build, and maintain scalable AWS-based data platform services and high-performance data pipelines. Provide technical leadership, establish data engineering best practices, and partner with cross-functional teams to deliver data integration, transformation, governance, and physical data models for healthcare solutions.
Top Skills:
Aurora PostgresqlAws EmrAws LambdaData LakehouseData MeshDynamoDBGluePysparkPythonRdsRedshiftS3ScalaSQLTerraform
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Manages end-to-end resolution of regulatory, management, nodal officer, and online grievance escalations. Analyzes complex and repeat complaints, performs root-cause analysis, ensures policy compliance and timely resolution, monitors turnaround times and quality, maintains escalation records, and guides BPO and cross-functional teams. The role also drives process improvements, consumer satisfaction, team cross-training, and effective handling of exceptional cases.
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
