The Data Engineer will develop and maintain data-crawling infrastructure, work with teams to gather requirements, write efficient code, and mentor juniors.
About us:
Rubick.ai is one of the fastest-growing eCommerce enablement platforms. We specialise in Product Discovery, Search and Market Intelligence for marketplaces, brands, and sellers. We offer an end-to-end full-stack Product Information, Cataloging, and Marketing platform as a solution for eCommerce.
Rubick has catalogued over 5M SKUs for 200+ leading eCommerce brands like Amazon, Hudson Bay US, Zilingo Singapore, The Luxury Closet-UAE, and Myntra in India, the US, Singapore, and other international markets.
Visit us: https://www.rubick.ai/
Job Overview:
We are seeking a talented and experienced Software Development Engineer (SDE) I with expertise in Python, Machine learning, and web crawling to join our dynamic engineering team. The ideal candidate will contribute to the design, development, and maintenance of our software products, demonstrating strong technical skills across both frontend and backend technologies.
Responsibilities:
● Design, develop, and maintain scalable and high-performance data-crawling infrastructure.
● Collaborate with product management and other cross-functional teams to understand requirements and translate them into technical solutions.
● Write clean, efficient, and maintainable code adhering to best practices and coding standards.
● Conduct code reviews, provide constructive feedback, and mentor junior team members to foster a culture of continuous improvement.
● Perform testing, debugging, and troubleshooting of applications to ensure optimal performance and reliability.
● Stay updated on emerging technologies, tools, and industry trends to suggest and implement innovative solutions.
Requirements:
● Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
● 0-2 years of professional experience as a Data crawling engineer or similar role.
● Proficiency in frontend technologies such as Scrapy, Selenium.
● Strong backend development skills using languages like Python, Node.js, or similar, along with experience with backend frameworks (e.g., Django, Fast API).
Familiarity with breaking different web security and crawling at scale.
● Experience working with databases (SQL, NoSQL) and understanding of database design principles.
● Knowledge of RESTful APIs and experience in building and consuming them.
● Familiarity with version control systems (e.g., Git), CI/CD pipelines, and cloud services (AWS, Azure, or GCP).
● Excellent problem-solving skills and the ability to work in a collaborative team environment.
Rubick.ai Bengaluru, Karnataka, IND Office
Bengaluru, India
Similar Jobs
Financial Services
Design, build, and maintain scalable, secure cloud-native data platforms and pipelines (primarily on AWS). Develop data processing solutions using Python and Spark/PySpark, implement orchestration and CI/CD, ensure data quality/integrations, and support analytics and microservice-based data workflows.
Top Skills:
AirflowAWSCi/CdDbtGlueInfrastructure As CodePysparkPythonSparkSQLStep FunctionsTrino
Financial Services
Design, build, and operate end-to-end cloud data pipeline solutions with a focus on reliability, scalability, security, and zero-downtime releases. Monitor and optimize performance, resolve incidents, support production lifecycle, and collaborate with product teams to deliver data products for finance, payments, and regulatory reporting.
Top Skills:
AWSEmrPublic CloudPythonRedshift
Financial Services
Lead Data Engineer responsible for making critical asset and wealth management data AI-ready: build and maintain scalable data pipelines, ensure lineage and metadata, enforce data quality and governance, drive automation and controls, collaborate with business and tech partners, and deliver executive visibility and dashboards to support analytics and AI initiatives.
Top Skills:
AWSAzureData CatalogingData Lineage ToolsEnterprise-Authorized AiGCPGraph DatabasesMetadata Management PlatformsNlqPower BIPythonRSparkSQLTableau
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
