Design, build, and maintain scalable AWS-based data pipelines using Python, Spark, and SQL. Manage data in S3, catalog with Glue Crawlers, query with Athena, and process with EMR. Implement serverless workflows with Lambda, monitor with CloudWatch, integrate alerts via SNS, and support dashboards (QuickSight/Power BI/Tableau) for analytics and reporting.
Data Engineer (AWS)
Company: Tibil Solutions
Location: Bangalore
Experience: 4 – 6 Years
Employment Type: Full Time
We are looking for a skilled Data Engineer with strong experience in Python, SQL, and AWS data services to build and manage scalable data pipelines. The ideal candidate will work with large datasets, design efficient data processing workflows, and support analytics and reporting solutions.
Key Responsibilities- Design, develop, and maintain scalable data pipelines for processing large datasets.
- Develop data processing scripts using Python.
- Write optimized SQL queries for data extraction and transformation.
- Process large volumes of data using Apache Spark (Scala or PySpark).
- Manage and store data using AWS S3.
- Implement serverless data workflows using AWS Lambda.
- Use AWS Glue Crawlers for data cataloging and metadata management.
- Query data efficiently using Amazon Athena.
- Develop dashboards or reports using Amazon QuickSight or other visualization tools.
- Monitor and optimize data pipelines using AWS CloudWatch.
- Work with Amazon EMR for big data processing.
- Integrate alerting or messaging workflows using AWS SNS when required.
- Strong proficiency in Python programming.
- Hands-on experience with SQL.
- Experience with Apache Spark (Scala or PySpark).
- Experience working with AWS S3.
- Experience with AWS Lambda.
- Hands-on experience with AWS Glue Crawlers.
- Experience with Amazon Athena.
- Experience with any data visualization tool (QuickSight, Power BI, Tableau, etc.)
- Experience with Amazon EMR.
- Knowledge of AWS SNS.
- Experience with AWS CloudWatch monitoring.
- 4 – 6 years of experience in Data Engineering or Big Data technologies.
- Experience working with AWS cloud services and big data tools.
- Strong problem-solving and analytical skills.
Similar Jobs
.png){kind=logo}
AdTech • eCommerce • Information Technology • Software • Travel • Generative AI
Design, build, and operate scalable batch and streaming data pipelines, analytics layers, and impression data models to support ad inventory forecasting. Ensure data quality, observability, and performance; collaborate with engineers, data scientists, and product teams to translate requirements into reliable, production-grade data solutions and integrate AI/ML capabilities where appropriate.
Top Skills:
Ai/MlAlertingApi DesignCi/CdData LakeData ModelingData WarehouseDeployment AutomationDistributed ProcessingFeature PipelinesJavaMonitoringObservabilityPythonScalaSQLStream Processing
Fintech • Legal Tech • Software • Financial Services • Cybersecurity • Data Privacy
Perform and review fund accounting for Luxembourg real estate funds, calculate NAVs, fees and investor allocations, manage SPV consolidations, support audits, investor services and regulatory compliance (AIFMD/CSSF). Liaise with auditors, valuers, property managers and cross-functional teams, improve processes, and mentor junior staff.
Cloud • Security • Software • Cybersecurity • Automation
Design, build, and deploy enterprise UiPath RPA automations end-to-end. Gather requirements, create reusable solutions, document designs, support production bots, perform root-cause analysis, and continuously optimize automations while collaborating with stakeholders and Enterprise Apps teams.
Top Skills:
AdpCi/CdCoupaDevOpsEtradeJSONNavanNetSuiteOauthPythonRestSalesforceSAMLSoapSoxSQLUipath Agentic AiUipath Automation HubUipath OrchestratorUipath RobotsUipath StudioWorkdayXMLZipZuora BillingZuora Revenue
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
