Design, build, and maintain scalable AWS-based data pipelines using Python, Spark, and SQL. Manage data in S3, catalog with Glue Crawlers, query with Athena, and process with EMR. Implement serverless workflows with Lambda, monitor with CloudWatch, integrate alerts via SNS, and support dashboards (QuickSight/Power BI/Tableau) for analytics and reporting.
Data Engineer (AWS)
Company: Tibil Solutions
Location: Bangalore
Experience: 4 – 6 Years
Employment Type: Full Time
We are looking for a skilled Data Engineer with strong experience in Python, SQL, and AWS data services to build and manage scalable data pipelines. The ideal candidate will work with large datasets, design efficient data processing workflows, and support analytics and reporting solutions.
Key Responsibilities- Design, develop, and maintain scalable data pipelines for processing large datasets.
- Develop data processing scripts using Python.
- Write optimized SQL queries for data extraction and transformation.
- Process large volumes of data using Apache Spark (Scala or PySpark).
- Manage and store data using AWS S3.
- Implement serverless data workflows using AWS Lambda.
- Use AWS Glue Crawlers for data cataloging and metadata management.
- Query data efficiently using Amazon Athena.
- Develop dashboards or reports using Amazon QuickSight or other visualization tools.
- Monitor and optimize data pipelines using AWS CloudWatch.
- Work with Amazon EMR for big data processing.
- Integrate alerting or messaging workflows using AWS SNS when required.
- Strong proficiency in Python programming.
- Hands-on experience with SQL.
- Experience with Apache Spark (Scala or PySpark).
- Experience working with AWS S3.
- Experience with AWS Lambda.
- Hands-on experience with AWS Glue Crawlers.
- Experience with Amazon Athena.
- Experience with any data visualization tool (QuickSight, Power BI, Tableau, etc.)
- Experience with Amazon EMR.
- Knowledge of AWS SNS.
- Experience with AWS CloudWatch monitoring.
- 4 – 6 years of experience in Data Engineering or Big Data technologies.
- Experience working with AWS cloud services and big data tools.
- Strong problem-solving and analytical skills.
Similar Jobs
.png){kind=logo}
Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
Collects, processes, and analyzes complex datasets; builds automated reports and dashboards; applies analytics and AI/ML techniques; automates data workflows and low-code apps; ensures data quality; partners with stakeholders and engineering teams to deliver actionable insights across the Asia Pacific region.
Top Skills:
AWSExcelPower AppsPower AutomatePower BIPower QueryPythonSnowflakeSQLVBA
Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
Design and implement scalable enterprise Salesforce solutions, lead architecture and design reviews, build reusable components, integrate Salesforce with enterprise and AI systems, mentor engineers, and ensure security, scalability, and governance.
Top Skills:
AgentforceAi AgentsAnthropicApexAutomated TestingAws BedrockAzure OpenaiCi/CdData CloudEinstein AiEvent-Driven ArchitectureGeminiGitLightning Web ComponentsMcpMicroservicesModel BuilderMulesoftOpenaiPlatform EventsPrompt BuilderRagRest ApisSalesforceSalesforce ApisSalesforce FlowsSOQLSoslStreaming ApisVector Databases
Fintech • Machine Learning • Payments • Software • Financial Services
Lead AML-focused business analysis and product delivery: develop strategy, translate analytics into products, manage multiple projects, partner with Tech/Model Risk, mentor analysts, and communicate results to senior stakeholders.
Top Skills:
AWSMachine LearningPythonRRelational DatabasesSnowflakeSparkSQL
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
