Lead Data Engineer

Company & Department description

Swift is a member-owned cooperative that provides the communications platform, products and services to connect more than 10,800 banking organizations, securities institutions and corporate customers in over 200 countries and territories. Our mission is to drive innovation and reliability in the financial industry, supporting our members with cutting-edge solutions and collaborative expertise

.
To reach this goal, Swift is becoming a more agile, data-driven organisation. We want to enable the production, access, mining and delivery of data in a highly secure environment as well as developing a standardized analytical environment to enable analytics at scal

e.
We are looking for talented individuals to help us build the next generation of our data products and services. If you want to have direct impact on how the world adapts to the changes in technology and have an affinity with financial services, then the Swift Data & Common Platform Applications tribe is a place for y

ou.
About the R

ole:
We are seeking a highly skilled Lead Data Engineer to architect, develop and maintain business use-cases. The ideal candidate will have extensive experience with Airflow, Spark, Starburst, Delta Lake and S2 and a strong background in testing and quality assurance using Jira XRay. This role requires leadership capabilities to guide a team of engineers and collaborate closely with business analysts, platform and business stakeholders. The squad is based in both Malaysia (Kuala Lumpur) and Belgium (La Hu

lpe).
Key Responsibil

• ities:
  Design, develop, and maintain reliable, scalable, and efficient data pip
• elines.Knowledge and experience with Medallion Architecture (Bronze, Silver
• , Gold)Orchestrate complex workflows using Apache A
• irflow.Develop distributed data processing jobs using Apache
• Spark.Implement object storage solutions using Cloudian S3 for data lake archite
• ctures.Build and manage Delta tables with schema enforcement and ev
• olutionOptimize data access and querying using Starburst
• /Trino.Ensure high data quality by adapting testing frameworks and writing automated tests using Jir
• a XRay.Ensure high availability, fault tolerance, and performance of data pla
• tforms.Collaborate with data analysts and business stakeholders to understand data requir
• ements.Implement data quality checks, monitoring, and alerting mech
• anisms.Enforce data governance, security, and compliance sta
• ndards.Participate in technical design discussions and contribute to architectural dec
• isions.Mentor junior engineers and promote engineering best pra
• ctices.Participate in on-call support and incident management when re

quired.
Required Qualifi

• cations
  Bachelor’s or master’s degree in Computer Science, Engineering, Information Technology, or relat
• ed field.10+ years of experience in data engineering, big data, or analytics p
• latforms.Strong hands-on experience with: Apache Airflow, Apache Spark, Delta Lake, Cloudian S3 or S3-compatible object storage, Starburst/Trino, Red hat O
• penShift.Proficiency in Java i
• s a must.Experience in testing and quality assurance using Jira Xray is a h
• uge plus.Experience with Python/Scala is an a
• dvantage.Experience with containerization and orchestration (Docker, Kubernetes, OpenShift) i
• s a plus.Familiarity with CI/CD pipelines and DevOps p
• ractices.Solid understanding of data modelling, data warehousing, and data lake arch
• itecture.Experience working with large-scale, high-volume, and high-velocity data
• systems.Strong analytical, communication and presentation skills, with excellent English, both written an
• d spoken.Team player with strong interpersonal skills, willing to work in a multicultural environment and able to lead, coach and groom other team
• members.Adaptability and responsiveness t