The Data Engineer will develop and maintain data-crawling infrastructure, work with teams to gather requirements, write efficient code, and mentor juniors.
About us:
Rubick.ai is one of the fastest-growing eCommerce enablement platforms. We specialise in Product Discovery, Search and Market Intelligence for marketplaces, brands, and sellers. We offer an end-to-end full-stack Product Information, Cataloging, and Marketing platform as a solution for eCommerce.
Rubick has catalogued over 5M SKUs for 200+ leading eCommerce brands like Amazon, Hudson Bay US, Zilingo Singapore, The Luxury Closet-UAE, and Myntra in India, the US, Singapore, and other international markets.
Visit us: https://www.rubick.ai/
Job Overview:
We are seeking a talented and experienced Software Development Engineer (SDE) I with expertise in Python, Machine learning, and web crawling to join our dynamic engineering team. The ideal candidate will contribute to the design, development, and maintenance of our software products, demonstrating strong technical skills across both frontend and backend technologies.
Responsibilities:
● Design, develop, and maintain scalable and high-performance data-crawling infrastructure.
● Collaborate with product management and other cross-functional teams to understand requirements and translate them into technical solutions.
● Write clean, efficient, and maintainable code adhering to best practices and coding standards.
● Conduct code reviews, provide constructive feedback, and mentor junior team members to foster a culture of continuous improvement.
● Perform testing, debugging, and troubleshooting of applications to ensure optimal performance and reliability.
● Stay updated on emerging technologies, tools, and industry trends to suggest and implement innovative solutions.
Requirements:
● Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
● 0-2 years of professional experience as a Data crawling engineer or similar role.
● Proficiency in frontend technologies such as Scrapy, Selenium.
● Strong backend development skills using languages like Python, Node.js, or similar, along with experience with backend frameworks (e.g., Django, Fast API).
Familiarity with breaking different web security and crawling at scale.
● Experience working with databases (SQL, NoSQL) and understanding of database design principles.
● Knowledge of RESTful APIs and experience in building and consuming them.
● Familiarity with version control systems (e.g., Git), CI/CD pipelines, and cloud services (AWS, Azure, or GCP).
● Excellent problem-solving skills and the ability to work in a collaborative team environment.
Rubick.ai Bengaluru, Karnataka, IND Office
Bengaluru, India
Similar Jobs
.png)
Artificial Intelligence • Cloud • Payments • Software • Business Intelligence • Generative AI • Automation
Designs, builds, operationalizes, and maintains scalable GCP data pipelines and warehouses. Responsibilities include ETL optimization, data modeling, governance, lineage, quality checks, monitoring, alerting, documentation, and troubleshooting. The role uses Python, SQL, BigQuery, dbt, Airflow, Dataflow, and related tools to integrate data from multiple sources and support analytics and reporting. It also requires collaboration with stakeholders, reusable software components, and ownership of production data pipelines.
Top Skills:
Apache AirflowBigQueryCeligoCloud ComposerDataflowDataprocDbtFivetranGitGoogle Cloud Platform (Gcp)JSONPythonSQLTalendXML
Cloud • Healthtech • Professional Services • Software • Pharmaceutical
Leads data engineering and technical consulting for clinical systems, including platform configuration, ETL, data warehouse development, custom analytics, reporting, integrations, validation, quality control, and SOP development. Provides technical escalation, solution design, project oversight, mentoring, training, process improvement, and client support while ensuring compliance with eClinical and industry standards.
Top Skills:
.NetAWSAzureBusiness ObjectsC#CognosDb2HTMLIbm DatastageInformInformaticaJavaJreviewExcelMS OfficeMicrosoft ProjectMicrostrategyOracleOracle ClinicalPl/SqlPythonQliksenseQlikviewRRaveRest ApisSASSparkSpotfireSQLSQL ServerSsrsT-SqlTableauTeradataTrial Master
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead data engineering architecture and execution for large-scale, low-latency analytics. Own technical direction, ensure operational data quality, design streaming and batch pipelines, mentor engineers, coordinate cross-functional teams, and deliver scalable solutions using Spark, Airflow, and AWS or equivalent data platforms.
Top Skills:
AirflowAthenaColumn StoresDatabricksDatabricks ApisEmrFlinkHiveKafkaKappa ArchitectureLambda ArchitectureMaster Data Management (Mdm)Microservices ArchitectureRedshiftSparkSQLStreaming Pipelines
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
