Organization: At CommBank, we never lose sight of the role we play in other people’s financial wellbeing. Our focus is to help people and businesses move forward to progress. To make the right financial decisions and achieve their dreams, targets, and aspirations. Regardless of where you work within our organisation, your initiative, talent, ideas, and energy all contribute to the impact that we can make with our work. Together we can achieve great things.
Job Title: Sr Data Engineering
Location: Bangalore
Business & Team:
Technology Team is responsible for the world leading application of technology and operations across every aspect of CommBank, from innovative product platforms for our customers to essential tools within our business. We also use technology to drive efficient and timely processing, an essential component of great customer service.
CommBank is recognised as leading the industry in IT and operations with its world-class platforms and processes, agile IT infrastructure, and innovation in everything from payments to internet banking and mobile apps.
The Group Security (GS) team protects the Bank and our customers from cyber compromise, through proactive management of cyber security, privacy, and operational risk. Our team includes:
- Cyber Strategy & Performance
- Cyber Security Centre
- Cyber Protection & Design
- Cyber Delivery
- Cyber Data Engineering
- Cyber Data Security
- Identity & Access Technology
The Group Security Senior Data Engineering team provides specialised data services and platforms for the CommBank group & is accountable for developing Group’s data strategy, data policy & standards, governance and set requirements for data enablers/tools. The team is also accountable to facilitate a community of practitioners to share best practice and build data talent and capabilities.
Impact & contribution :-
To ensure the Group achieves a sustainable competitive advantage through data engineering, you will play a key role in supporting and executing the Group's data strategy.
We are looking for an experienced Data Engineer to join our Group Security Team, which is part of the wider Cyber Security Engineering practice. In this role, you will be responsible for setting up the Group Security Data Platform to ingest data from various organizations' security telemetry data, along with additional data assets and data products. This platform will provide security controls and services leveraged across the Group.
Roles & Responsibilities
You will be expected to perform the following tasks in a manner consistent with CBA’s Values and People Capabilities.
CORE RESPONSIBILITIES:
- Possesses hands-on technical experience working in AWS. The individual should have knowledge about AWS services like EC2, S3, Lambda, Athena, Kinesis, Redshift, Glue, EMR, DynamoDB, IAM, SecretManager, KMS, Step functions, SQS,SNS, Cloud Watch.
- The individual should possess a robust set of technical and soft skills and be an excellent AWS Data Engineer with a focus on complex Automation and Engineering Framework development.
- Being well-versed in Python is mandatory, and experience in developing complex frameworks using Python is required.
- Passionate about Cloud/DevSecOps/Automation and possess a keen interest in solving complex problems systematically.
- Drive the development and implementation of scalable data solutions and data pipelines using various AWS services.
- Possess the ability to work independently and collaborate closely with team members and technology leads.
- Exhibit a proactive approach, constantly seeking innovative solutions to complex technical challenges.
- Can take responsibility for nominated technical assets related to areas of expertise, including roadmaps and technical direction.
- Can own and develop technical strategy, overseeing medium to complex engineering initiatives.
Essential Skills:-
- About 10-13 years of experience as a Data Engineering professional in a data-intensive environment.
- The individual should have strong analytical and reasoning skills in the relevant area.
- Proficiency in AWS cloud services, specifically EC2, S3, Lambda, Athena, Kinesis, Redshift, Glue, EMR, DynamoDB, IAM, SecretManager, Step functions, SQS,SNS, Cloud Watch.
- Excellent skills in Python-based framework development are mandatory.
- Proficiency in SQL for efficient querying, managing databases, handling complex queries, and optimizing query performance.
- Excellent automation skills are expected in areas such as
- Automating the testing framework using tools such as PyPy, Pytest, and various test cases including unit, integration, functional tests, and mockups.
- Automating the data pipeline and expediting tasks such as data ingestion and transformation.
- API-based automated and integrated calls(REST, cURL, authentication & authorization, tokens, pagination, openApi, Swagger)
- Implementing advanced engineering techniques and handling ad hoc requests to automate processes on demand.
- Implementing automated and secured file transfer protocols like XCOM, FTP, SFTP, and HTTP/S
- Experience with Terraform, Jenkins, Teracity and Artifactory is essential as part of DevOps. Additionally, Docker and Kubernetes are also considered.
- Proficiency in building orchestration workflows using Apache Airflow.
- Strong understanding of streaming data processing concepts, including event-driven architectures.
- Familiarity with CI/CD pipeline development, such as Jenkins.
- Extensive experience and understanding in Data Modelling, SCD Types, Data Warehousing, and ETL processes.
- Excellent experience with GitHub or any preferred version control systems.
- Expertise in data pipeline development using various data formats/types.
- Mandatory knowledge and experience in big data processing using PySpark/Spark and performance optimizations of applications
- Proficiency in handling various file formats (CSV, JSON, XML, Parquet, Avro, and ORC) and automating processes in the big data environment.
- Ability to use Linux/Unix environments for development and testing.
- Should be aware of security best practices to protect data and infrastructure, including encryption, tokenization, masking, firewalls, and security zones.
- Well-structured documentation skills and the ability to create a well-defined knowledge base.
- Certifications such as AWS Certified Data Analytics/Engineer/Developer – Specialty or AWS Certified Solutions Architect.
- Should be able to perform extreme engineering and design a robust, efficient, and cost-effective data engineering pipelines which are highly available and dynamically scalable on demand.
- Enable the systems to effectively respond to high demands and heavy loads maintaining the high throughput and high I/O performance with no data loss
- Own and lead E2E Data engineering life cycle right from Requirement gathering, design, develop, test, deliver and support as part of DevSecOPS process.
- Must demonstrate skills and mindset to implement encryption methodologies like SSL/TLS and data encryption at rest and in transit and other data security best practices
- Hands on work experience with data design tools like Erwin and demonstrate the capabilities of building data models, data warehouse, data lakes, data assets and data products
- Must be able to constructively challenge the status quo and lead to establish data governance, metadata management, ask the right questions, design with right principles
Education Qualification :-
A Bachelor's or Master's degree in Engineering, specializing in Computer Science, Information Technology or relevant qualifications.
If you're already part of the Commonwealth Bank Group (including Bankwest, x15ventures), you'll need to apply through Sidekick to submit a valid application. We’re keen to support you with the next step in your career.
We're aware of some accessibility issues on this site, particularly for screen reader users. We want to make finding your dream job as easy as possible, so if you require additional support please contact HR Direct on 1800 989 696.
Advertising End Date: 30/01/2025
