Design, build, and maintain scalable AWS-based data pipelines using Python, Spark, and SQL. Manage data in S3, catalog with Glue Crawlers, query with Athena, and process with EMR. Implement serverless workflows with Lambda, monitor with CloudWatch, integrate alerts via SNS, and support dashboards (QuickSight/Power BI/Tableau) for analytics and reporting.
Data Engineer (AWS)
Company: Tibil Solutions
Location: Bangalore
Experience: 4 – 6 Years
Employment Type: Full Time
We are looking for a skilled Data Engineer with strong experience in Python, SQL, and AWS data services to build and manage scalable data pipelines. The ideal candidate will work with large datasets, design efficient data processing workflows, and support analytics and reporting solutions.
Key Responsibilities- Design, develop, and maintain scalable data pipelines for processing large datasets.
- Develop data processing scripts using Python.
- Write optimized SQL queries for data extraction and transformation.
- Process large volumes of data using Apache Spark (Scala or PySpark).
- Manage and store data using AWS S3.
- Implement serverless data workflows using AWS Lambda.
- Use AWS Glue Crawlers for data cataloging and metadata management.
- Query data efficiently using Amazon Athena.
- Develop dashboards or reports using Amazon QuickSight or other visualization tools.
- Monitor and optimize data pipelines using AWS CloudWatch.
- Work with Amazon EMR for big data processing.
- Integrate alerting or messaging workflows using AWS SNS when required.
- Strong proficiency in Python programming.
- Hands-on experience with SQL.
- Experience with Apache Spark (Scala or PySpark).
- Experience working with AWS S3.
- Experience with AWS Lambda.
- Hands-on experience with AWS Glue Crawlers.
- Experience with Amazon Athena.
- Experience with any data visualization tool (QuickSight, Power BI, Tableau, etc.)
- Experience with Amazon EMR.
- Knowledge of AWS SNS.
- Experience with AWS CloudWatch monitoring.
- 4 – 6 years of experience in Data Engineering or Big Data technologies.
- Experience working with AWS cloud services and big data tools.
- Strong problem-solving and analytical skills.
Similar Jobs
.png)
Healthtech • Social Impact • Telehealth
As a Care Coordinator, you will guide older adults and their caregivers through the intake process, providing support and emotional guidance during their journey into care.
Top Skills:
CrmsEhr PlatformsGoogle Workspace
Gaming • Information Technology • Mobile • Software • Esports
The Technical Operations Engineer ensures infrastructure reliability for game services, leads automation efforts, and provides advanced troubleshooting for technical issues in a collaborative, global environment.
Top Skills:
AnsibleAWSAzureGCPHyper-VKvmLinuxProxmoxPulumiPuppetTerraformVMwareWindowsWsl
Gaming • Information Technology • Mobile • Software • Esports
The Technical Operations Center Analyst ensures high availability and performance of production systems, monitors incidents, resolves technical requests, and maintains operational documentation.
Top Skills:
AnsibleAWSAzureConfluenceGCPHyper-VJIRAKvmLinuxNetworking ProtocolsProxmoxPuppetSlackTerraformVMwareWindowsZoom
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
