Senior Databricks Data Engineer
About the Role
We are seeking an experiencedMicrosoft Power Platform Solution Architect & Developer
NYC
to join our growing data and analytics team. This role is ideal for a highly skilled data professional with deep expertise in Databricks, Spark, Kafka, Python, and cloud technologies who can design, build, and optimize enterprise-scale data platforms and real-time data processing solutions.
The successful candidate will bring strong hands-on experience in modern data engineering, distributed computing, cloud-native architectures, and large-scale data integrations.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Databricks and Python.
- Build, optimize, and support high-performance Apache Spark applications for batch and real-time data processing.
- Develop and manage Kafka-based streaming solutions for real-time data ingestion and processing.
- Design and implement data architectures leveraging Delta Lake and open table formats.
- Deploy, monitor, and manage data solutions across Azure and/or AWS cloud platforms.
- Develop and maintain integrations with SQL and NoSQL databases.
- Partner with data architects, developers, business stakeholders, and cross-functional teams to deliver scalable data solutions.
- Implement data quality, governance, security, and performance optimization best practices.
- Contribute to the design and development of microservices-based architectures supporting modern data platforms.
- Work within containerized environments using Kubernetes (K8S) and cloud-native technologies.
Required Qualifications
- 10+ years of experience in Data Engineering, Data Platform Development, or a related field.
- Strong hands-on experience with Databricks and Delta Lake.
- Deep expertise in:
- Apache Spark
- Apache Kafka
- Python
- SQL and NoSQL databases
- Open table formats and modern data lake architectures
- Experience with Azure and/or AWS cloud platforms.
- Strong knowledge of Java development and microservices architecture.
- Hands-on experience with Kubernetes (K8S) and containerized applications.
- Experience designing and supporting large-scale, enterprise-grade data solutions.
Preferred Qualifications
- Experience working in Agile/Scrum environments.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Experience building real-time streaming and event-driven architectures.
- Knowledge of data governance, data quality, and security best practices.
What We Offer
- Competitive compensation and comprehensive benefits package.
- Opportunity to work with cutting-edge cloud and data technologies.
- Challenging, high-impact projects in a collaborative environment.
- Professional growth and career development opportunities.
- Exposure to large-scale enterprise data modernization initiatives.
Location: Flexible / Hybrid / Remote
Experience Level: Senior (10+ Years)
Employment Type: Full-Time
🚀 Why Join Us?
You’ll work with a talented global team, contribute to meaningful technical initiatives, and help shape a high‑performing DevOps culture.
Matrix is a global, dynamic, and fast‑growing technology services and consulting company with over 17,000 employees worldwide. Founded in 2001, Matrix has grown through significant organic expansion and strategic acquisitions, executing some of the largest and most impactful technology projects in the market.
We specialize in the design, development, and implementation of advanced technologies, software solutions, and digital products. Our services include infrastructure and consulting, IT outsourcing and offshore delivery, training and assimilation programs, and acting as a trusted implementation and delivery partner for the world’s leading software vendors.
With deep expertise across both the private and public sectors—spanning Finance, Telecom, Healthcare, Hi‑Tech, Education, Defense, and Security—Matrix serves a strong customer base in Israel alongside a continuously expanding global client portfolio.
At Matrix, our people are at the heart of our success. We are a community of talented, creative, and dedicated professionals who are passionate about innovation and excellence. We actively attract, develop, and retain top talent, recognizing that every employee’s contribution is critical to our continued growth and future success.
Matrix’s success is built on a challenging and engaging work environment, competitive compensation and benefits, and meaningful career development opportunities. We foster a diverse, inclusive culture that encourages collaboration, continuous learning, and growth—together.
Join the winning team. Be challenged, grow your career, and enjoy the journey in a highly respected global organization.
To learn more, visit: www.matrix-ifs.com
.png){kind=logo}
