Design and build AWS-based ingestion pipelines for structured and unstructured data supporting analytics and AI/GenAI use cases. Prepare AI-ready datasets through document parsing, chunking, and embedding workflows, while developing reusable, metadata-driven ingestion frameworks.
Design ingestion pipelines feeding analytics and AI/GenAI use cases.
- Ingest structured and unstructured data from multiple sources into the AWS data lake
- Prepare AI-ready datasets, including document parsing, chunking and embedding pipelines
- Build reusable, metadata-driven ingestion frameworks
Requirements
- Must have: 4+ yrs, AWS (Glue, S3, Lambda, Step Functions, Kinesis), Python, PySpark.
- Nice to have: Bedrock, vector databases, OpenSearch
Similar Jobs
Fintech • Financial Services
Provides enterprise technical leadership for complex engineering challenges across networking, application delivery, information security, databases, operating systems, and web technologies. Defines architecture and engineering standards, leads hybrid-cloud modernization, and designs highly available, scalable, secure infrastructure using F5 BIG-IP and AVI load balancers. Drives automation, Infrastructure as Code, DevOps, resiliency, production support, and incident resolution while advising leadership, influencing strategy, and mentoring engineers.
Top Skills:
AnsibleAvi Load BalancerAWSAzureCi/CdCiscoDnsF5 Big-IpGCPGtmKubernetesLtmOpenshiftPythonSite Reliability EngineeringSsl/TlsTcp/IpTerraformVMwareWafZero Trust
Fintech • Financial Services
Owns product demand, roadmaps, backlogs, requirements, Agile delivery, testing, release readiness, governance, and executive reporting across market maker and options clearing. Coordinates Product, Technology, Operations, Risk, Compliance, Finance, and Control stakeholders while managing dependencies, risks, controls, regulatory initiatives, and process improvements. Supports automation, analytics, AI enablement, production governance, and operational readiness for clearing and post-trade capabilities.
Top Skills:
Agile Delivery MethodologiesAi-Enabled ToolsAnalyticsJIRALow-Code PlatformsExcelMS OfficeMicrosoft PowerpointMicrosoft WordWorkflow Automation
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Principal Software Engineer focused on encryption, working remotely within Atlassian’s distributed-first engineering organization. The provided description contains limited role-specific responsibilities.
Top Skills:
Encryption
What you need to know about the Bengaluru Tech Scene
Dubbed the "Silicon Valley of India," Bengaluru has emerged as the nation's leading hub for information technology and a go-to destination for startups. Home to tech giants like ISRO, Infosys, Wipro and HAL, the city attracts and cultivates a rich pool of tech talent, supported by numerous educational and research institutions including the Indian Institute of Science, Bangalore Institute of Technology, and the International Institute of Information Technology.
