Join our exciting team as a Data Engineer with the following job scope:
- Architect and implement data solutions on our large-scale data platform Hadoop using PySpark.
- Write complex and efficient SQL queries to extract and transform data from various sources.
- Design, build, and maintain scalable and reliable data pipelines using tools like Apache Airflow and Kafka.
- Develop and optimize data processing jobs on the Microsoft Azure cloud platform, leveraging services such as Azure Data Lake, Databricks, and Azure Data Factory.
- Collaborate with data analysts and other stakeholders to understand data requirements and deliver high-quality data products.
- Ensure data quality and integrity across all systems.
- Monitor, troubleshoot, and optimize the performance of our data infrastructure.
- Stay up-to-date with the latest trends and technologies in the Big Data space.
