Lead Data Engineer
Data Engineering
Security Bank Corporation
May 2019 – Present
Responsibilities:
· Lead a team of data engineers, setting priorities, assigning tasks, and providing mentorship and guidance to team members
· Collaborate with data scientists, analysts, and stakeholders to understand data requirements and provide them with high-quality data.
· Develop ETL processes using PySpark on AWS EMR clusters to transform and clean raw data.
· Design and manage data processing workflows and orchestration using Apache Airflow.
· Create and schedule DAGs (Directed Acyclic Graphs) to automate data pipelines and ETL tasks.
· Create repositories, code review and merging in Gitlab
· Maintain detailed documentation for data pipelines, ETL processes, Airflow workflows, and system architecture.
Senior Management Information Officer
Business Claims Analytics
QBE Insurance GSSC
May 2013 – May 2019
Responsibilities:
· Perform Ad hoc data gathering, consolidation, analysis and reporting while utilizing various data sources, Reporting software and delivery mechanism.
· Develop, update and maintain scheduled reports and dashboard
· Identify, develop and implement process improvement initiatives and automation projects
· Analyse, interpret and provide commentaries on Report Trends, Workflow and procedures.
· Administer, debug and maintain Databases to ensure smooth feeding of data to the Dashboard Reports.
· Report regularly to the management on performance, highlighting positive/negative variances so that the appropriate action is taken.