The Data Engineer will develop and maintain data-crawling infrastructure, work with teams to gather requirements, write efficient code, and mentor juniors.
About us:
Rubick.ai is one of the fastest-growing eCommerce enablement platforms. We specialise in Product Discovery, Search and Market Intelligence for marketplaces, brands, and sellers. We offer an end-to-end full-stack Product Information, Cataloging, and Marketing platform as a solution for eCommerce.
Rubick has catalogued over 5M SKUs for 200+ leading eCommerce brands like Amazon, Hudson Bay US, Zilingo Singapore, The Luxury Closet-UAE, and Myntra in India, the US, Singapore, and other international markets.
Visit us: https://www.rubick.ai/
Job Overview:
We are seeking a talented and experienced Software Development Engineer (SDE) I with expertise in Python, Machine learning, and web crawling to join our dynamic engineering team. The ideal candidate will contribute to the design, development, and maintenance of our software products, demonstrating strong technical skills across both frontend and backend technologies.
Responsibilities:
● Design, develop, and maintain scalable and high-performance data-crawling infrastructure.
● Collaborate with product management and other cross-functional teams to understand requirements and translate them into technical solutions.
● Write clean, efficient, and maintainable code adhering to best practices and coding standards.
● Conduct code reviews, provide constructive feedback, and mentor junior team members to foster a culture of continuous improvement.
● Perform testing, debugging, and troubleshooting of applications to ensure optimal performance and reliability.
● Stay updated on emerging technologies, tools, and industry trends to suggest and implement innovative solutions.
Requirements:
● Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
● 0-2 years of professional experience as a Data crawling engineer or similar role.
● Proficiency in frontend technologies such as Scrapy, Selenium.
● Strong backend development skills using languages like Python, Node.js, or similar, along with experience with backend frameworks (e.g., Django, Fast API).
Familiarity with breaking different web security and crawling at scale.
● Experience working with databases (SQL, NoSQL) and understanding of database design principles.
● Knowledge of RESTful APIs and experience in building and consuming them.
● Familiarity with version control systems (e.g., Git), CI/CD pipelines, and cloud services (AWS, Azure, or GCP).
● Excellent problem-solving skills and the ability to work in a collaborative team environment.
Rubick.ai Bengaluru, Karnataka, IND Office
Bengaluru, India
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Consulting
Leads complex supply chain planning and operations, including inventory, demand and supply matching, import/export, sales and operations planning, and trade compliance. Drives process improvements, performance metrics, cross-functional planning, demand signals, and strategic projects across global business units and supply bases. Manages regulatory trade compliance programs, mentors junior staff, and applies analytics, financial modeling, MRP, ATP, and master scheduling expertise.
Top Skills:
Available-To-Promise (Atp)Material Requirements Planning (Mrp)ExcelMS OfficeMicrosoft Powerpoint
Information Technology • Machine Learning • Software • Conversational AI • Generative AI • Manufacturing
Designs and owns production ETL/ELT pipelines, medallion architectures, data transformations, validation, monitoring, lineage, and orchestration. Builds GenAI and RAG services, REST APIs, and scalable backend capabilities using Spark and Databricks. Improves retrieval quality, system reliability, observability, data modeling, and performance while contributing to architecture decisions and frontend integrations.
Top Skills:
AngularSparkApi GatewaysCi/CdDatabricksDelta LakeDockerFastapiGitLlm ApisLoggingMicroservicesMonitoringPysparkPythonReactRest ApisSQLVector Search
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Design and build data services and pipelines supporting machine learning products. Develop automation tools for model deployment, maintain production data systems, improve infrastructure, participate in code reviews, and collaborate across engineering, data science, and product teams. The role requires expertise in distributed systems, large-scale data processing, CI/CD, container orchestration, and AI-enabled workflow improvements.
Top Skills:
AirflowAWSAws BatchCi/CdDockerEmrGlueGoKafkaKubernetesKv StoresLinuxPythonRelational DatabasesSagemakerSparkSpinnaker
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
