The Data Engineer will design and implement scalable data pipelines in Azure, collaborate with DevOps for CI/CD, and ensure data quality.
Job Title - Azure Data Engineer
Experience - 7- 10 years
We are seeking a skilled and experienced Data Engineer to join our growing team. The ideal candidate will have a solid background in designing, building, and maintaining scalable data solutions in Azure, along with strong programming and DevOps skills. You will work closely with cross-functional teams to deliver high-performance data pipelines and services that drive business insights and support key strategic initiatives.
Key Responsibilities:
- Design and implement scalable and secure data pipelines using Azure Data Factory and Azure Databricks (Spark).
- Utilize Azure cloud services such as Blob Storage, Azure Functions, and API Gateway to build and maintain robust data solutions.
- Develop and maintain code using Python or similar programming languages to support ETL workflows and data transformation processes.
- Collaborate with DevOps teams to set up and manage CI/CD pipelines using Azure DevOps or equivalent toolsets.
- Ensure high code and data quality standards through performance tuning, testing, and documentation.
- Work across the full development lifecycle, from design through deployment and maintenance of distributed systems.
- Leverage knowledge of SQL and NoSQL databases to design effective data models and ensure data reliability.
- Apply Object-Oriented Programming and ETL best practices to develop modular and reusable components.
- Troubleshoot and resolve cloud networking and connectivity issues in collaboration with infrastructure teams.
- Translate complex technical problems into simple and scalable solutions, aligning with architectural best practices.
- Advocate for and implement data governance, monitoring, and quality frameworks to ensure the integrity of business-critical data.
Required Qualifications:
- Proven experience working with Azure Data Engineering services.
- Hands-on experience with Databricks (Spark) and Azure Data Factory.
- Strong programming skills in Python, with a solid grasp of object-oriented programming.
- Experience with Azure DevOps or other CI/CD tools.
- Demonstrated ability to develop and scale high-performance, distributed services.
- Deep understanding of relational and non-relational (NoSQL) databases.
- Familiarity with cloud networking, ETL concepts, and performance tuning.
- Excellent problem-solving abilities and strong understanding of data architecture principles.
Internal Skills And Expertise
Required Qualifications:
- Proven experience working with Azure Data Engineering services.
- Hands-on experience with Databricks (Spark) and Azure Data Factory.
- Strong programming skills in Python, with a solid grasp of object-oriented programming.
- Experience with Azure DevOps or other CI/CD tools.
- Demonstrated ability to develop and scale high-performance, distributed services.
- Deep understanding of relational and non-relational (NoSQL) databases.
- Familiarity with cloud networking, ETL concepts, and performance tuning.
- Excellent problem-solving abilities and strong understanding of data architecture principles.
Top Skills
Azure Cloud Services
Azure Data Factory
Azure Databricks
Azure Devops
NoSQL
Python
SQL
Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Design, build, and optimize data pipelines on Azure. Implement event-driven architectures and microservices, ensuring reliability and performance. Collaborate with teams to maintain SLAs and document solutions.
Top Skills:
Azure Data FactoryAzure DatabricksAzure FunctionsAzure Key VaultAzure PipelinesCi/CdDelta LakePysparkService BusSparkSQL
Information Technology
The role involves leading data engineering efforts, focusing on Azure data architecture and implementing data solutions using various technologies.
Top Skills:
Adls G2Azure MonitoringBlobCosmosdbData ExplorerData FactoryData WarehousingDatabricksETLEvent HubLogic AppPlsqlPurviewPythonSQLSQL ServerStored ProceduresStream AnalyticsSynapseT-Sql
Fintech • Professional Services • Software • Financial Services
The Azure Data Engineer - Manager will design and manage data solutions on Azure, develop ETL pipelines, and enhance data architectures while ensuring compliance and collaboration across teams.
Top Skills:
SparkAzure Cosmos DbAzure Data FactoryAzure Data Lake StorageAzure DatabricksAzure Sql DatabaseAzure
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
