The Data Engineer will develop and maintain data-crawling infrastructure, work with teams to gather requirements, write efficient code, and mentor juniors.
About us:
Rubick.ai is one of the fastest-growing eCommerce enablement platforms. We specialise in Product Discovery, Search and Market Intelligence for marketplaces, brands, and sellers. We offer an end-to-end full-stack Product Information, Cataloging, and Marketing platform as a solution for eCommerce.
Rubick has catalogued over 5M SKUs for 200+ leading eCommerce brands like Amazon, Hudson Bay US, Zilingo Singapore, The Luxury Closet-UAE, and Myntra in India, the US, Singapore, and other international markets.
Visit us: https://www.rubick.ai/
Job Overview:
We are seeking a talented and experienced Software Development Engineer (SDE) I with expertise in Python, Machine learning, and web crawling to join our dynamic engineering team. The ideal candidate will contribute to the design, development, and maintenance of our software products, demonstrating strong technical skills across both frontend and backend technologies.
Responsibilities:
● Design, develop, and maintain scalable and high-performance data-crawling infrastructure.
● Collaborate with product management and other cross-functional teams to understand requirements and translate them into technical solutions.
● Write clean, efficient, and maintainable code adhering to best practices and coding standards.
● Conduct code reviews, provide constructive feedback, and mentor junior team members to foster a culture of continuous improvement.
● Perform testing, debugging, and troubleshooting of applications to ensure optimal performance and reliability.
● Stay updated on emerging technologies, tools, and industry trends to suggest and implement innovative solutions.
Requirements:
● Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
● 0-2 years of professional experience as a Data crawling engineer or similar role.
● Proficiency in frontend technologies such as Scrapy, Selenium.
● Strong backend development skills using languages like Python, Node.js, or similar, along with experience with backend frameworks (e.g., Django, Fast API).
Familiarity with breaking different web security and crawling at scale.
● Experience working with databases (SQL, NoSQL) and understanding of database design principles.
● Knowledge of RESTful APIs and experience in building and consuming them.
● Familiarity with version control systems (e.g., Git), CI/CD pipelines, and cloud services (AWS, Azure, or GCP).
● Excellent problem-solving skills and the ability to work in a collaborative team environment.
Rubick.ai Bengaluru, Karnataka, IND Office
Bengaluru, India
Similar Jobs
Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
Designs, develops, optimizes, and maintains scalable data architectures and ETL pipelines. Integrates large structured and unstructured datasets, supports data scientists, evaluates emerging engineering tools, and partners on enterprise architectures. The role also requires cloud deployment, troubleshooting, technical education, mentoring, and expertise in distributed data technologies, databases, DevOps, and R&D digital platforms.
Top Skills:
Big DataC++Cloud PlatformsComputerized System ValidationContinuous IntegrationData ArchitectureData PipelinesDevOpsETLHadoopJavaKubernetesMongoDBPl/SqlPythonScalaSdlcSparkSQL
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Leads enterprise-scale data engineering and architecture initiatives, designing secure cloud data platforms, pipelines, governance frameworks, and data products. Drives modernization using Azure, Databricks, Snowflake, Spark, SQL, and Python; establishes DataOps, CI/CD, monitoring, and automation practices. Partners with stakeholders and technical teams to deliver analytics, reporting, and AI/ML solutions. Provides technical leadership, mentors engineers, reviews designs, evaluates emerging technologies, and develops strategic data roadmaps.
Top Skills:
AWSAzureCi/CdDatabricksDataopsElt/EtlGCPPythonSnowflakeSparkSQL
Financial Services
Designs, develops, and troubleshoots secure, scalable software and data solutions. Builds cloud-based data pipelines using AWS, Airflow, Databricks, and related technologies; develops optimized Python and PySpark code for large datasets; creates architecture and design artifacts; performs data analysis, visualization, profiling, modeling, and ETL; and contributes to engineering practices and emerging technology initiatives.
Top Skills:
Apache AirflowApache NifiSparkAWSCi/CdData WarehousingDatabricksDatabricks WorkflowsDelta Live TablesETLHadoopInformaticaOraclePl/SqlPysparkPythonUnity Catalog
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
