The Senior Software Architect will lead data engineering initiatives, focusing on building and managing data lakes and warehouses, utilizing cloud services and open-source technologies for seamless data integration and analytics. They will guide development teams, ensuring optimal performance, scalability, and security of the data platform while staying updated with emerging technologies.
Job Title: Senior Software Architect – Cloud Data Platform
About the Role: We are seeking a seasoned Senior Architect with over 18 years of experience in technical, solutioning, and analytical roles to lead our data engineering initiatives. The ideal candidate will have extensive experience in building and managing data lakes, data warehouses, and BI/AI solutions, with a strong preference for utilizing open-source technologies alongside AWS, Azure or GCP platforms.
Key Responsibilities:
- Design and Development: Lead the design and implementation of a complex data platform for streaming real-time data from diverse environmental monitoring devices across various industries and tenants.
- Architecture Leadership: Architect scalable web applications leveraging the data platform to ensure optimal scalability, performance, and security.
- Strategic Integration: Utilize predominantly open-source technologies for data integration, processing, and analytics, integrating them with cloud-native services where necessary.
- Team Collaboration and Leadership: Collaborate with product managers, UX/UI designers, and other stakeholders to convert business requirements into robust technical solutions. Mentor and guide development teams, promoting a culture of excellence and continuous improvement.
- Innovation: Stay updated with emerging technologies and industry trends to assess their impact and integration into our projects, focusing on enhancing efficiency and performance.
Required Skills and Qualifications:
- Professional Experience: Minimum 18 years in relevant technical roles, including at least 5 years in data architecture and management.
- Education Qualification: Bachelor’s degree in computer science or related fields
Technical Expertise:
- Data Management and Integration: Expertise in data lakes, data warehouses, and complex data integrations using open-source tools such as Apache Spark, Pyspark/Scala, and Hadoop. Proficient in ETL processes and data pipelines with tools like Apache Airflow, Dagster and Kafka.
- Cloud and Database Technologies: Deep knowledge of various databases (RDS, Aurora, Oracle, Teradata, SQL Server) and integration with cloud services (AWS, Azure, GCP). Skilled in leveraging open-source databases like PostgreSQL and MySQL for data operations.
- Data Processing and Analytics: Strong experience in building data processing frameworks such as Spark SQL and Presto, and using data pipeline technologies like S3, Athena, Iceberg and Glue. Familiar with BI tools such as Tableau and Power BI.
- Machine Learning and AI: Experience with machine learning frameworks like TensorFlow and Pytorch, integrating AI into data solutions to enhance business insights.
- Cloud Architecture and Security: Understanding of cloud architecture (IaaS, PaaS, SaaS) and security protocols across cloud platforms. Ability to optimize data security and performance in cloud environments.
Other Attributes:
- Ability to provide high-level strategic thinking and fine-grained technical solutions.
- Excellent problem-solving skills and ability to work in dynamic, fast-paced environments.
- Strong communication skills and ability to act as a trusted advisor to clients and internal teams.
Top Skills
Spark
Pyspark
Scala
Picarro Bengaluru, Karnataka, IND Office
#1023, Nallurhali Road, 9th Floor, Brigade IRV, Whitefield, Bengaluru, Karnataka , India, 560066
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
As a Senior Principal Engineer, you will design and build cutting-edge infrastructure solutions for Atlassian products, leading architectural decisions and mentoring engineers. You will collaborate with core teams to enhance platform capabilities, drive operational excellence, and implement strategic initiatives.
Top Skills:
Cloud ComputingKubernetes
Automotive • Big Data • Information Technology • Robotics • Software • Transportation • Manufacturing
The Lead Engineer will develop and execute throughput simulation studies to optimize automotive plant processes, enhance efficiency, and reduce costs. Responsibilities include creating simulation models, performing analyses, and leading collaboration for process improvements.
Top Skills:
C-DrawPlant SimulationSimtalk2.0
Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
As a Data Science Associate Consultant, you'll design and implement analytics solutions for structured and unstructured data, develop deep learning models, collaborate with teams on machine learning deployment, and contribute to AI research.
Top Skills:
JavaPythonR
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
