Designs, builds, deploys, monitors, and supports production data pipelines and architectures. The role requires advanced Python and SQL, data modeling, transaction and fact table development, workflow orchestration, data quality controls, lineage, governance, and regulated-domain experience. It also involves scalable system design, CI/CD, Git-based development, YAML configuration, and collaboration with enterprise data platforms and tooling.
Job Description:
- 6+ years of experience in Data Engineering.
- Strong data architecture and system design skills with the ability to make sound trade-offs around data modeling, partitioning, and scalability.
- Demonstrated end-to-end ownership of production data pipelines, including design, deployment, monitoring, and operational support.
- Expertise in Python and SQL as primary tools; experience with Hive/Hadoop is a plus.
- Strong focus on data quality, validation, reconciliation, anomaly detection, and audit ability.
- Experience working with finance, risk, compliance, or other highly regulated data domains requiring strong data lineage and governance.
- Proven experience designing and building transaction and fact tables.
- Advanced SQL skills, including window functions, CTEs, CASE statements, and working across MySQL/PostgreSQL environments.
- Strong Python expertise, including pandas, SQLAlchemy, type hints, data classes, and object-oriented pipeline development.
- Experience with workflow orchestration platforms such as Airflow or similar tools.
- Strong understanding of data modeling concepts including fact/dimension modeling, normalization, and medallion architecture.
- Experience with YAML-based configurations, CI/CD practices, and Git-based development workflows.
Requirements
Strongly Preferred:
- Experience with enterprise data platforms, internal data tooling ecosystems, metadata/catalog solutions, secrets management tools, developer productivity platforms, and ETL orchestration frameworks.
Nice-to-Have:
- Experience with Hive, Hadoop, Spark, or related big data technologies.
Benefits
· Competitive salary and performance-based bonuses.
· Comprehensive insurance plans.
· Collaborative and supportive work environment.
· Chance to learn and grow with a talented team.
· A positive and fun work environment.
Prescience Decision Solutions Bengaluru, Karnataka, IND Office
Bengaluru, India
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Consulting
Build and optimize batch and streaming data pipelines, develop data models and transformations, integrate APIs and event streams, and implement data quality, testing, documentation, and monitoring. The role uses Snowflake, SQL, Python, dbt, orchestration tools, and cloud platforms while supporting DataOps, scalability, cost efficiency, and collaboration with engineers, analysts, data scientists, and business stakeholders.
Top Skills:
AirflowAWSAzureCi/CdCloudFormationDagsterDataopsDbtGCPKafkaKinesisPrefectPub/SubPythonSnowflakeSQLTerraform
.png)
Artificial Intelligence • Cloud • Payments • Software • Business Intelligence • Generative AI • Automation
Build and maintain scalable BigQuery data solutions, including tables, views, stored procedures, and data models. Manage SQL deployment manifests and multi-environment rollouts through GitLab CI/CD. Improve data quality, governance, performance, security, and accessibility across large-scale datasets. Collaborate with architects, data scientists, engineers, and global stakeholders while contributing to AI-enabled initiatives, code reviews, technical leadership, and team mentoring.
Top Skills:
BigQueryClaude AiGitGithub CopilotGitlab Ci/CdGoogle Cloud PlatformLookerLookmlPythonSQLYaml
Artificial Intelligence • Big Data • Cloud • Information Technology • Software • Big Data Analytics • Automation
Designs and optimizes scalable data ingestion pipelines for product telemetry and enterprise business data. Builds automated pipelines using Fivetran, Python, APIs, S3 lakehouse architectures, Kafka, Spark, and Snowflake. Applies advanced SQL, incremental loading, CDC, micro-batching, and rate-limit handling to support reliable, analytics-ready datasets. Implements CI/CD and DevOps practices, monitors pipeline performance, and partners with analysts and platform teams to rapidly deliver data solutions.
Top Skills:
Amazon S3SparkBulk ApisCdcCi/CdDevOpsFivetranKafkaPythonSnowflakeSQLVersion Control
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
