The Big Data Developer will design, develop, and maintain big data solutions, working with large volumes of structured and unstructured data. Key responsibilities include optimizing ETL processes, implementing data models, collaborating with stakeholders, and ensuring data quality and security.
Job Description:
Unison Consulting Pte Ltd is seeking an experienced Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining big data solutions to enable data-driven decision making and analytics capabilities for our clients. You will work with large volumes of structured and unstructured data using cutting-edge technologies and platforms.
Responsibilities:
- Design, develop, and maintain big data solutions using technologies such as Hadoop, HDFS, Spark, and Hive.
- Ingest, process, and analyze large volumes of structured and unstructured data from diverse sources.
- Develop and optimize ETL processes for efficient data extraction, transformation, and loading.
- Implement data models, schemas, and data governance frameworks to ensure data quality and integrity.
- Perform data exploration, profiling, and validation to identify patterns, trends, and anomalies.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
- Implement and optimize data storage and retrieval strategies, including data partitioning and indexing.
- Build and maintain scalable data pipelines and workflows for data processing.
- Perform performance tuning and optimization of big data applications and queries.
- Implement and maintain data security measures to protect sensitive information.
- Maintain technical documentation of big data solutions, processes, and procedures.
Skills and Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
- Proven experience as a Big Data Developer with proficiency in technologies such as Hadoop, Spark, HDFS and Hive.
- Strong programming skills in languages such as Java, Scala, or Python.
- Experience with ETL tools and processes for data extraction, transformation, and loading.
- Good understanding of big data concepts, architectures, and technologies.
- Experience with SQL and NoSQL databases.
- Knowledge of data modeling and schema design.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills to work effectively with cross-functional teams.
- Understanding of data security and privacy principles.
- Ability to adapt to new technologies and learn quickly in a fast-paced environment.
Top Skills
Java
Python
Scala
Similar Jobs
AdTech • Artificial Intelligence • Marketing Tech • Software • Analytics
As a Senior Software Engineer at Zeta, you will develop and maintain the Zeta Identity Graph platform, manage extensive data ingestion, ensure data integrity, and collaborate with various teams to optimize and architect data infrastructures. You'll work extensively with data systems, supporting real-time and batch processing while participating in architecture discussions and project ownership.
Top Skills:
JavaPythonScala
AdTech • Artificial Intelligence • Marketing Tech • Software • Analytics
The Lead Software Engineer will design and maintain the data infrastructure for the Zeta Identity Graph platform, ensuring data accuracy and optimizing workflows. Responsibilities include building scalable solutions, collaborating with data engineers and analysts on ETL processes, and participating in architecture discussions.
Top Skills:
JavaPythonScala
Internet of Things
The Azure Big Data Engineer at Bosch will implement end-to-end Hadoop projects, primarily using MS-Azure technologies such as HDInsights and Databricks. The role requires extensive experience with SparkCore, PySpark, and other data management tools, along with excellent communication skills.
Top Skills:
HadoopMs-AzurePysparkScalaSparkcore
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
