Develop and maintain scalable data pipelines using Python and PySpark. Build Azure-based ETL and ELT solutions with Data Factory, Data Lake, Blob Storage, Databricks, Synapse, and Azure SQL. Write complex SQL queries, stored procedures, and transformations; troubleshoot pipeline issues; collaborate with technical and business teams; and follow engineering best practices using Git, Azure DevOps, and Jira.
Data Engineer – Azure & PySparkJob Summary
We are looking for a Data Engineer with 3+ years of experience in Python/PySpark, SQL, Data Warehousing, and Azure ETL to build scalable and reliable data pipelines.
Key Responsibilities- Develop and maintain data pipelines using Python/PySpark.
- Build ETL/ELT solutions using Azure Data Factory, Data Lake, Blob Storage, Databricks, Synapse, and Azure SQL.
- Write complex SQL queries, stored procedures, and transformations.
- Collaborate with business and technical teams to deliver data solutions.
- Troubleshoot data and pipeline issues and follow engineering best practices.
- Use Git, Azure DevOps, and Jira for development and project tracking.
- 3+ years of experience in Data Engineering/Data Warehousing.
- Strong Python/PySpark and SQL skills.
- Hands-on experience with Azure data services and ETL.
- Good understanding of data integration and data warehousing.
- Strong analytical, problem-solving, and communication skills.
- Azure Databricks/Spark experience.
- Azure Data Engineer certification is a plus.
Role: Data Engineer | Employment: Full-Time, Permanent
Education: B.Sc./BCA/B.Tech/B.E. or any Postgraduate degree
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Consulting
Develop and sustain complex data center packet-forwarding software for routers and switches. Design and implement networking features including IPv4/IPv6, MPLS, SRv6, EVPN-VXLAN, multicast, firewalling, QoS, DHCP, and services. Debug Linux and microkernel issues, improve automated unit testing, collaborate across teams, and resolve customer-reported issues. The role also requires working with datapath ASICs, multicore architectures, and software development lifecycle practices.
Top Skills:
BarefootBroadcomCC++Datapath AsicsDhcpEvpn-VxlanInnoviumIpv4Ipv6LinuxMellanoxMicrokernelMplsMulticastNpuPythonSrv6
Cloud • Information Technology • Insurance • Software • Analytics
Build and maintain business intelligence solutions, analytical data models, reporting datasets, dashboards, and ETL/ELT pipelines. Translate stakeholder needs into technical specifications, optimize SQL, automate data-quality validation with Python, and support AWS-based analytics platforms including Amazon Redshift. Establish testing standards, document data assets, and use AI-assisted development tools while validating generated output.
Top Skills:
Amazon RedshiftAWSClaude CodeEltETLGeminiGitGitPythonSQLThoughtspot
Biotech
Build, test, optimize, and maintain batch and real-time data pipelines across AWS or Azure platforms. Perform data mapping, requirements analysis, data quality management, and architecture planning while collaborating with engineering, analytics, project management, and global stakeholders. Apply Agile, DevOps, DataOps, and DevSecOps practices, develop efficient PySpark solutions, and explore LLM and generative AI applications for data engineering productivity.
Top Skills:
AWSAzureCassandraDatabricksDataopsDevOpsDevsecopsGptLarge Language ModelsMongoDBMySQLPostgresPower BIPysparkPythonRetrieval-Augmented GenerationSnowflakeSQLTableau
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
