Lead the design and management of scalable, AI-ready data platforms and products. Build batch and real-time pipelines, semantic layers, and data models while ensuring governance, quality, reliability, and SLA adherence. Enable ML and agentic AI capabilities, implement DataOps and CI/CD, optimize cloud costs, and maintain platform stability. Collaborate with stakeholders to translate business needs into sustainable technical solutions.
HIRING | LEAD DATA ENGINEER
Location: Bangalore | Hybrid
Experience: 8–12 Years
CTC: Up to ₹25 LPA
ROLE OVERVIEW
We are looking for a Lead Data Engineer to design and manage scalable, AI-ready data platforms and products. The role focuses on data engineering, governance, reliability, cloud optimization and enabling ML/Agentic AI capabilities.
MUST-HAVE SKILLS
• Big Data & Modern Data Engineering
• AWS / Azure / GCP
• Python / PySpark / SQL
• Snowflake / Databricks / Spark / BigQuery / Synapse
• Kafka / Kinesis / Event Hub / Dataflow
• Apache Airflow / Astronomer
• Data Modeling, Quality & Reliability
• Data Governance & Metadata Management
• CI/CD & DataOps
• Data Platform Engineering & Administration
• Agentic AI / LLM Integrations & Orchestration
• Scalable, resilient & cost-efficient data solutions
KEY RESPONSIBILITIES
• Design and build scalable AI-ready data solutions
• Develop semantic layers and data models
• Ensure data quality, governance and reliability
• Enable ML and Agentic AI capabilities
• Drive cloud cost and platform optimization
• Build real-time and batch data pipelines
• Implement DataOps, CI/CD and orchestration
• Collaborate with business and technology stakeholders
• Translate business needs into scalable technical solutions
• Ensure platform stability, SLA adherence and long-term sustainability
GOOD TO HAVE
• Multi-cloud experience
• Data observability and monitoring
• API / Microservices integrations
• Power BI
• Retail domain experience
• React / Modern Frontend Technologies
Similar Jobs
Healthtech • Software
Designs and maintains scalable data platforms, architectures, ETL pipelines, and complex SQL solutions across multiple databases. Develops Python applications and LLM-powered solutions using LangChain, RAG, and AWS Bedrock. Implements Airflow orchestration, data quality and governance practices, AWS deployments, CI/CD, and production support. Collaborates with data scientists, analysts, product managers, and engineering teams to translate requirements into reliable data-driven solutions and reporting.
Top Skills:
Ag-GridAmazon BedrockAmazon Ec2Amazon S3Apache AirflowAWSAws LambdaAws Secrets ManagerCi/CdConfluenceCSS3DjangoExcelFlaskGitGithub CopilotHTML5JavaScriptJIRALangchainLlmsMicrosoft Sql ServerMicrosoft TeamsNumpyOraclePandasPlotlyPostgresPower BIPythonRetrieval-Augmented Generation (Rag)SnowflakeSQL
Information Technology • Productivity • Software • Manufacturing
Own the architecture, scalability, governance, and technical direction of Amtech’s cloud data platform, including warehouses, tenant data stores, feature stores, and event pipelines. Lead multi-tenant data architecture, hybrid AI data routing, data security, observability, and reliability standards. Build data foundations for analytics, machine learning, and AI agents, including vector databases and RAG pipelines. Provide technical leadership through design reviews, mentoring, roadmap planning, and cross-functional architecture collaboration.
Top Skills:
Amazon KinesisAmazon RedshiftApache AirflowApache FlinkApache KafkaSparkAWSAws GlueAzureBigQueryDbtErpGCPGrafanaGreat ExpectationsIot/OtJavaLangchainMesNoSQLPrometheusPythonRag PipelinesScadaScalaSnowflakeSQLVector Databases
Cloud • Database
Build and deploy production-ready AI applications and data pipelines on Azure Databricks. Responsibilities include PySpark and SQL ETL/ELT, CI/CD, Azure integrations, ML and generative AI deployment, RAG and vector search, LLM fine-tuning, monitoring, optimization, governance, and responsible AI. The role also involves Databricks Apps, Unity Catalog, prompt engineering, FastAPI and React applications, testing, guardrails, and collaboration with data science and platform teams.
Top Skills:
Adls Gen2AirflowArmAzureAzure Cognitive SearchAzure Data FactoryAzure DatabricksAzure DevopsAzure FunctionsAzure Key VaultAzure Machine LearningAzure OpenaiBicepClaude ApisDatabricks Asset BundlesFaissFastapiHugging FaceLangchainLoraMlflowPysparkPythonQloraReactSQLTerraformUnity CatalogVisual Studio Code
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
