Develop and maintain scalable data pipelines using Python and PySpark. Build Azure-based ETL and ELT solutions with Data Factory, Data Lake, Blob Storage, Databricks, Synapse, and Azure SQL. Write complex SQL queries, stored procedures, and transformations; troubleshoot pipeline issues; collaborate with technical and business teams; and follow engineering best practices using Git, Azure DevOps, and Jira.
Data Engineer – Azure & PySparkJob Summary
We are looking for a Data Engineer with 3+ years of experience in Python/PySpark, SQL, Data Warehousing, and Azure ETL to build scalable and reliable data pipelines.
Key Responsibilities- Develop and maintain data pipelines using Python/PySpark.
- Build ETL/ELT solutions using Azure Data Factory, Data Lake, Blob Storage, Databricks, Synapse, and Azure SQL.
- Write complex SQL queries, stored procedures, and transformations.
- Collaborate with business and technical teams to deliver data solutions.
- Troubleshoot data and pipeline issues and follow engineering best practices.
- Use Git, Azure DevOps, and Jira for development and project tracking.
- 3+ years of experience in Data Engineering/Data Warehousing.
- Strong Python/PySpark and SQL skills.
- Hands-on experience with Azure data services and ETL.
- Good understanding of data integration and data warehousing.
- Strong analytical, problem-solving, and communication skills.
- Azure Databricks/Spark experience.
- Azure Data Engineer certification is a plus.
Role: Data Engineer | Employment: Full-Time, Permanent
Education: B.Sc./BCA/B.Tech/B.E. or any Postgraduate degree
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Consulting
Develop and sustain complex data center packet-forwarding software for routers and switches. Design and implement networking features including IPv4/IPv6, MPLS, SRv6, EVPN-VXLAN, multicast, firewalling, QoS, DHCP, and services. Debug Linux and microkernel issues, improve automated unit testing, collaborate across teams, and resolve customer-reported issues. The role also requires working with datapath ASICs, multicore architectures, and software development lifecycle practices.
Top Skills:
BarefootBroadcomCC++Datapath AsicsDhcpEvpn-VxlanInnoviumIpv4Ipv6LinuxMellanoxMicrokernelMplsMulticastNpuPythonSrv6
Financial Services
Designs, validates, and productionizes multi-cloud lakehouse and data platform architectures. Builds scalable batch and streaming pipelines using Python, Scala, SQL, and Spark; develops Iceberg data models; improves performance, reliability, governance, and observability; and creates AI-ready data products and services. Leads POCs through production, supports operational readiness, and mentors L2/L3 teams maintaining legacy platforms.
Top Skills:
AlationApache IcebergSparkAWSAzureCi/CdClaude Code AssistantCollibraDatabricksDevOpsGCPGithub CopilotLarge Language Models (Llms)PythonRest ApisRetrieval-Augmented Generation (Rag)ScalaSnowflakeSQLUnity CatalogVector Indexing
eCommerce • Fintech • Logistics • Software • Transportation • Big Data Analytics
Develop and maintain cloud-native ELT data pipelines using Snowflake, S3 Iceberg, dbt, Airflow, and Python. Build workflow orchestration, automated testing, CI/CD pipelines, monitoring, data-quality checks, and documentation. Support incremental loads, backfills, failure recovery, deployment across environments, and troubleshooting of data workflows. Collaborate with senior engineers while using AI-assisted development tools to improve productivity and engineering practices.
Top Skills:
Apache AirflowDbtGitGithub ActionsLlm-Based Coding AssistantsPythonS3 IcebergSnowflake
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
