The Data Engineer will develop and maintain data-crawling infrastructure, work with teams to gather requirements, write efficient code, and mentor juniors.
About us:
Rubick.ai is one of the fastest-growing eCommerce enablement platforms. We specialise in Product Discovery, Search and Market Intelligence for marketplaces, brands, and sellers. We offer an end-to-end full-stack Product Information, Cataloging, and Marketing platform as a solution for eCommerce.
Rubick has catalogued over 5M SKUs for 200+ leading eCommerce brands like Amazon, Hudson Bay US, Zilingo Singapore, The Luxury Closet-UAE, and Myntra in India, the US, Singapore, and other international markets.
Visit us: https://www.rubick.ai/
Job Overview:
We are seeking a talented and experienced Software Development Engineer (SDE) I with expertise in Python, Machine learning, and web crawling to join our dynamic engineering team. The ideal candidate will contribute to the design, development, and maintenance of our software products, demonstrating strong technical skills across both frontend and backend technologies.
Responsibilities:
● Design, develop, and maintain scalable and high-performance data-crawling infrastructure.
● Collaborate with product management and other cross-functional teams to understand requirements and translate them into technical solutions.
● Write clean, efficient, and maintainable code adhering to best practices and coding standards.
● Conduct code reviews, provide constructive feedback, and mentor junior team members to foster a culture of continuous improvement.
● Perform testing, debugging, and troubleshooting of applications to ensure optimal performance and reliability.
● Stay updated on emerging technologies, tools, and industry trends to suggest and implement innovative solutions.
Requirements:
● Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
● 0-2 years of professional experience as a Data crawling engineer or similar role.
● Proficiency in frontend technologies such as Scrapy, Selenium.
● Strong backend development skills using languages like Python, Node.js, or similar, along with experience with backend frameworks (e.g., Django, Fast API).
Familiarity with breaking different web security and crawling at scale.
● Experience working with databases (SQL, NoSQL) and understanding of database design principles.
● Knowledge of RESTful APIs and experience in building and consuming them.
● Familiarity with version control systems (e.g., Git), CI/CD pipelines, and cloud services (AWS, Azure, or GCP).
● Excellent problem-solving skills and the ability to work in a collaborative team environment.
Rubick.ai Bengaluru, Karnataka, IND Office
Bengaluru, India
Similar Jobs
.png){kind=logo}
Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
Designs, develops, and maintains scalable batch and real-time data pipelines, ETL/ELT workflows, data models, and cloud data platforms. Ensures data quality, governance, security, observability, and performance while supporting cloud modernization. Collaborates with product owners, architects, business stakeholders, and data scientists, and maintains technical documentation, lineage, monitoring, and operational runbooks.
Top Skills:
AirflowAmazon EmrAmazon S3Apache KafkaAWSAws GlueAws LambdaAzureBigQueryCi/CdDatabricksDataopsDbtGCPGitHiveJavaJenkinsKinesisMlopsOraclePostgresPythonRedshiftScalaSnowflakeSparkSQLSQL ServerSynapse
Food • Greentech • Logistics • Sharing Economy • Transportation • Agriculture • Industrial
Develops and maintains software applications using Java, Spring, Spring Boot, microservices, databases, and APIs. Responsibilities include automating deployments, collaborating with cross-functional teams, writing unit and integration tests, debugging complex issues, improving development processes, documenting systems, and providing technical support. Requires a computer science degree, 1–3 years of relevant experience, and knowledge of software design, data structures, algorithms, and system design.
Top Skills:
AlgorithmsAPIsCachingData StructuresDatabase ScalingJavaMicroservicesSpringSpring BootSQLSystem Design
Artificial Intelligence • Cloud • Information Technology • Consulting
Designs and maintains scalable data pipelines and cloud-native solutions for analytics, data science, and AI applications. Responsibilities include automated data ingestion and processing, data quality and performance optimization, cross-functional problem solving, and evaluating new technologies. The role requires cloud, big data, programming, distributed systems, and containerization expertise, and is primarily onsite at an HPE office.
Top Skills:
Agentic AiArangodbAWSAzureCi/CdClickhouseDistributed SystemsDockerElasticsearchGCPGenerative AiJavaKafkaKubernetesLinuxMessage QueuesNosql DatabasesPythonScalaShell ScriptingSpark
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
