Design, build, and maintain scalable AWS-based data pipelines using Python, Spark, and SQL. Manage data in S3, catalog with Glue Crawlers, query with Athena, and process with EMR. Implement serverless workflows with Lambda, monitor with CloudWatch, integrate alerts via SNS, and support dashboards (QuickSight/Power BI/Tableau) for analytics and reporting.
Data Engineer (AWS)
Company: Tibil Solutions
Location: Bangalore
Experience: 4 – 6 Years
Employment Type: Full Time
We are looking for a skilled Data Engineer with strong experience in Python, SQL, and AWS data services to build and manage scalable data pipelines. The ideal candidate will work with large datasets, design efficient data processing workflows, and support analytics and reporting solutions.
Key Responsibilities- Design, develop, and maintain scalable data pipelines for processing large datasets.
- Develop data processing scripts using Python.
- Write optimized SQL queries for data extraction and transformation.
- Process large volumes of data using Apache Spark (Scala or PySpark).
- Manage and store data using AWS S3.
- Implement serverless data workflows using AWS Lambda.
- Use AWS Glue Crawlers for data cataloging and metadata management.
- Query data efficiently using Amazon Athena.
- Develop dashboards or reports using Amazon QuickSight or other visualization tools.
- Monitor and optimize data pipelines using AWS CloudWatch.
- Work with Amazon EMR for big data processing.
- Integrate alerting or messaging workflows using AWS SNS when required.
- Strong proficiency in Python programming.
- Hands-on experience with SQL.
- Experience with Apache Spark (Scala or PySpark).
- Experience working with AWS S3.
- Experience with AWS Lambda.
- Hands-on experience with AWS Glue Crawlers.
- Experience with Amazon Athena.
- Experience with any data visualization tool (QuickSight, Power BI, Tableau, etc.)
- Experience with Amazon EMR.
- Knowledge of AWS SNS.
- Experience with AWS CloudWatch monitoring.
- 4 – 6 years of experience in Data Engineering or Big Data technologies.
- Experience working with AWS cloud services and big data tools.
- Strong problem-solving and analytical skills.
Similar Jobs
Cloud • Information Technology • Security • Software
Own and scale a $1M territory through partner recruitment, enablement, and GTM execution. Build executive partner relationships, manage co-selling and pipeline, run performance analysis, and coordinate internal teams to drive channel revenue in India.
Top Skills:
Active DirectoryGeminiGongGoogle Workspace (Gws)ImpartnerJumpcloudSalesforce (Sfdc)
Artificial Intelligence • Natural Language Processing • Professional Services • Analytics • Consulting • Conversational AI • Generative AI
Lead module functional consultant for end-to-end Oracle HCM Cloud (Fusion HCM) implementations. Must have functional/configuration experience in two or more modules such as Core HR, Absence, US/Canada Payroll, Benefits, Compensation, Talent Management, ORC, Learning, OTL, or HR Helpdesk.
Top Skills:
AbsenceBenefitsCanada PayrollCompensationCore HrFusion HcmHr HelpdeskLearningOracle Fusion HcmOracle Hcm CloudOrcOtlTalent ManagementUs Payroll
Artificial Intelligence • Natural Language Processing • Professional Services • Analytics • Consulting • Conversational AI • Generative AI
Lead design and architecture of modern data ecosystems (data lake/lakehouse), data modeling, ETL, and API-based architectures. Build RESTful/GraphQL and event-driven APIs, API gateway/auth patterns, and agentic AI agents/workflows. Provide solution leadership, stakeholder engagement, governance, and cloud data platform (preferably AWS) expertise.
Top Skills:
Agentic AiApi GatewayApi VersioningAuthentication/AuthorizationAWSData LakeData ModelingETLEvent-Driven ArchitectureGraphQLLakehouseRate LimitingRestful Api
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
