Design, develop, and maintain data pipelines and ETL processes using Python, Hive, Spark, and other technologies while handling Big Data.
Senior Data Specialist
Primary Skills
- MapReduce, HDFS, Spark - Pyspark, ETL Fundamentals, SQL, SQL (Basic + Advanced), Spark - Scala, Python, Data Warehousing, Hive, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures, Oozie
Job requirements
- • Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
- • Design, develop, and maintain data pipelines and ETL processes using Python, Hive, Spark, and other relevant technologies.
- • Good Hands on Exp with java
- • Minimum of 6+ years of experience as a Data Engineer, with proven expertise in Python, Hive, and Spark. • Advance SQL experience and working experience with Big data Hadoop .
- • Strong understanding of data engineering concepts, including data modelling, data integration, and data warehousing. • Hands-on experience with tools such as Apache Airflow.
- • Experience with Docker and containerization technologies is a plus.
- • Strong SQL skills and experience with relational databases, such as PostgreSQL, MySQL, or similar.
- • Familiarity with Apache Spark.
- • Knowledge of data processing frameworks such as Apache Beam, Apache Flink, or similar.
- • Strong problem-solving and troubleshooting skills, with attention to detail.
- • Excellent communication and interpersonal skills, with the ability to work collaboratively in a team environment.
Top Skills
Apache Airflow
Data Warehousing
Docker
Etl Fundamentals
Hdfs
Hive
Mapreduce
Oozie
Plsql
Python
Spark - Pyspark
Spark - Scala
SQL
T-Sql
Similar Jobs
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Cybersecurity • Data Privacy
The Senior Renewal Sales Specialist manages renewals for existing accounts, ensuring timely completion, negotiating uplifts, and coordinating with teams.
Top Skills:
CRM
Fintech • Financial Services
The Operations Senior Manager oversees the servicing and governance of a syndicated loan portfolio, leading teams, ensuring compliance, managing resources, and driving process improvements.
Top Skills:
Loan Iq
Fintech • Financial Services
Wells Fargo is seeking an Institutional Investment Operations Associate to support lifecycle tasks for institutional trades, client onboarding, and operational issues.
Top Skills:
General Ledger ReconciliationSecurities Settlement ServicesSwift Messages
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
