Designs and maintains scalable Azure data infrastructure, including ADLS and Delta Lake architectures, ETL pipelines with Azure Data Factory and PySpark, and data warehouses using Azure Synapse Analytics. Develops Python and SQL scripts for automation, data quality, and analytics; integrates diverse data sources and collaborates with cross-functional teams to deliver reliable data solutions.
Data Engineer (Azure)
Summary
We are seeking a skilled Data Engineer to design, build, and maintain robust data infrastructure within the IT Services sector. This pivotal role focuses on architecting scalable data solutions using Microsoft Azure technologies to ensure seamless data flow, integrity, and accessibility. The successful candidate will play a critical part in transforming raw data into actionable insights, supporting strategic business decisions through advanced data warehousing and analytics capabilities.
Responsibilities
- Design and implement scalable data storage and processing architectures utilizing Azure Data Lake Storage (ADLS) and Delta Lake.
- Develop and optimize complex ETL pipelines using Azure Data Factory (ADF) and PySpark to ensure efficient data movement and transformation.
- Build and manage data models and warehouses, leveraging Azure Synapse Analytics for high-performance querying and reporting.
- Construct and maintain data pipelines that integrate diverse data sources, ensuring accuracy and consistency across the ecosystem.
- Write and refine Python and SQL scripts to automate data processes, perform data quality checks, and support analytical workflows.
- Collaborate with cross-functional teams to define data requirements and deliver solutions that align with organizational goals.
Requirements
Requirements:
- Possess 4 to 9 years of professional experience in data engineering or a related technical field.
- Demonstrate deep proficiency with Azure cloud services, specifically Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
- Show strong expertise in data modeling, data warehousing concepts, and the implementation of Delta Lake architecture.
- Be highly skilled in developing ETL pipelines using Azure Data Factory (ADF) and executing big data processing with PySpark.
- Have a solid command of Python and SQL for data manipulation, scripting, and query optimization.
- Experience working within Databricks environments for collaborative data analysis and engineering is essential.
- Target compensation range is approximately ₹24 LPA, commensurate with experience and skill level.
Similar Jobs
Information Technology • Software • Business Intelligence
Provides technical leadership for Azure data engineering projects, including solution architecture, data modeling, pipelines, data integration, code quality, QA, performance optimization, governance, security, and documentation. Collaborates with stakeholders and project managers to define scope, manage risks, and deliver scalable, efficient solutions. Requires strong Python, SQL, PySpark, Databricks, Azure Data Factory, Delta Lake, Synapse, Spark, data warehousing, governance, and Azure DevOps expertise.
Top Skills:
SparkAzureAzure Data FactoryAzure Data LakeAzure DatabricksAzure DevopsAzure SynapseCi/CdData GovernanceData PipelinesData WarehousingDelta LakeETLPysparkPythonSQL
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Principal Software Engineer focused on encryption, working remotely within Atlassian’s distributed-first engineering organization. The provided description contains limited role-specific responsibilities.
Top Skills:
Encryption
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Principal Software Engineer role focused on building and developing Atlassian’s Trusted Data Platform. The position is full-time and based in Bengaluru, India, with remote flexibility. The provided description emphasizes Atlassian’s distributed-first work model, virtual interviews and onboarding, and location flexibility for eligible employees.
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
