Yadimen Consulting Limited Logo

Yadimen Consulting Limited

Data Engineer — PySpark + AWS Glue

Posted One Month Ago
Be an Early Applicant
In-Office
Bangalore, Bengaluru Urban, Karnataka, IND
Senior level
In-Office
Bangalore, Bengaluru Urban, Karnataka, IND
Senior level
Design, develop, optimize, and maintain scalable ETL pipelines using PySpark and AWS Glue. Build data ingestion and transformation workflows, orchestrate processes with AWS Step Functions, and develop Python-based AWS Lambda functions. Support AWS data lake architectures, analytical use cases, monitoring, reliability, and performance optimization. The role also involves SQL and may include Java microservices, REST APIs, and backend integrations.
The summary above was generated by AI

Data Engineer — PySpark + AWS Glue

ETL & Data Pipelines | 2 Openings

 

 

Location: Remote (UK)

Employment Type: Contract / Permanent

Experience Level: 4–8 Years

Openings: 2

 

About the Role

We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.



Requirements

Key Responsibilities

•       Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue

•       Build and optimise data ingestion, transformation, and loading workflows

•       Orchestrate data workflows using AWS Step Functions

•       Develop serverless functions using AWS Lambda (Python)

•       Work with data lake architectures on AWS to support analytical use cases

•       Ensure pipeline reliability, monitoring, and performance optimisation

 

Required Skills & Experience

•       Strong hands-on experience with PySpark and AWS Glue

•       Proven track record in ETL pipeline development and optimisation

•       Experience orchestrating workflows with AWS Step Functions

•       Proficiency in serverless development using AWS Lambda (Python)

•       Good SQL skills and a solid understanding of data processing principles

 

Nice to Have

•       Exposure to Java-based microservices

•       Understanding of REST APIs and backend service integrations

•       Experience working with AWS-based data lakes



Similar Jobs

17 Minutes Ago
Hybrid
Senior level
Senior level
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Manage APAC payroll for large employee groups, ensuring accurate and timely processing across multiple countries. Analyze payroll data for discrepancies and trends, improve payroll processes, coordinate with HR, Finance, and payroll vendors, and maintain compliance with payroll cycles and service-level agreements.
2 Hours Ago
Hybrid
Expert/Leader
Expert/Leader
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Leads the SaaS Security Posture Management program across enterprise applications, owning strategy, roadmap, governance, vulnerability prioritization, remediation tracking, KPIs, dashboards, and executive reporting. Coordinates Cybersecurity, application, architecture, compliance, and business stakeholders to reduce risk, meet audit and SOX requirements, remove remediation barriers, and improve enterprise SaaS security posture.
Top Skills: CspmIamIso 27001Microsoft 365NistObsidianPalo Alto PrismaSaas SecuritySalesforceServicenowSIEMSoxSspmWizWorkday
2 Hours Ago
Hybrid
Senior level
Senior level
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Lead design and implementation of PHP-based web applications, guide team members, troubleshoot and maintain existing systems, translate client requirements into solutions, provide technical assistance and develop proofs-of-concept throughout the development lifecycle.
Top Skills: CSS3GitHTML5JavaScriptLinuxMySQLObject Oriented ProgrammingPHPSubversion

What you need to know about the Bengaluru Tech Scene

Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account