Design, build, and maintain scalable AWS-based data pipelines and ETL processes. Collaborate with data scientists, optimize storage and retrieval for performance and cost, enforce data governance and security, troubleshoot pipeline issues, and recommend new data engineering technologies.
AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have 6-8 years of prior experience in data engineering, with a strong background in AWS (Amazon Web Services) technologies. This role offers an exciting opportunity to work on diverse projects, collaborating with cross functional teams to design, build, and optimize data pipelines and infrastructure.
Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes leveraging AWS services such as S3,
- Glue, EMR, Lambda, and Redshift.
- Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.
- Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness.
- Implement data governance and security best practices to ensure compliance and data integrity.
- Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
- Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.
Requirements
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 6-8 years of prior experience in data engineering, with a focus on designing and building data pipelines.
- Proficiency in AWS services, particularly S3, Glue, EMR, Lambda, and Redshift.
- Strong programming skills in languages such as Python, Java, or Scala.
- Experience with SQL and NoSQL databases, data warehousing concepts, and big data technologies.
- Familiarity with containerization technologies (e.g., Docker, Kubernetes) and orchestration tools (e.g., Apache Airflow) is a plus.
Similar Jobs
Software
Designs, develops, deploys, and monitors Databricks data pipelines on AWS using PySpark and SQL. Builds scalable data solutions, quality dashboards, lineage and monitoring tools, and supports legacy data migration. Ensures data security, retention, privacy, governance, and auditability while optimizing processes through automation and code improvements.
Top Skills:
Amazon S3SparkAtlassian StackAWSCi/CdCsvDatabricksDelta LakeDevOpsGitJSONParquetPysparkPythonSpark Structured StreamingSQLWeb ApisXML
Artificial Intelligence • Analytics • Consulting • Pharmaceutical
Build and maintain scalable AWS data pipelines, warehouses, schemas, and ingestion systems for large structured and unstructured datasets. Develop ETL/ELT workflows using AWS services including Glue, Lambda, S3, Redshift, and CodePipeline. Create dimensional models, analytics tools, and data marts while ensuring data quality, privacy, compliance, and performance. Collaborate with cross-functional stakeholders to resolve data issues, improve infrastructure, and deliver actionable business insights.
Top Skills:
Amazon Ec2Amazon EmrAmazon S3AWSAws CodepipelineAws GlueAws LambdaAws RedshiftEltETLGitNoSQLOlapParquetSQL
Artificial Intelligence • Big Data • Cloud • Machine Learning • Software
Designs and operates enterprise data governance infrastructure across Snowflake and AWS. Responsibilities include RBAC, authentication, masking, row-level security, compliance automation, Terraform provisioning, access workflows, monitoring, SOX evidence, data sharing, Iceberg governance, orchestration, cost optimization, and Snowflake AI/ML governance. The role also supports platform reliability, self-service data access, and scalable identity integration using Okta and SCIM.
Top Skills:
AlationAmazon AthenaAmazon EventbridgeApache AirflowApache IcebergSparkAWSAws LambdaCollibraGithub ActionsIamMwaaOktaPythonS3ScimSnowflakeSnowflake CortexSnowpark MlSQLTerraform
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
