We are seeking an experienced Senior Data Engineer to lead the design, development, and optimization of large-scale data platforms across Azure, Google Cloud, and hybrid environments.
You will architect data pipelines that power AI-driven analytics, knowledge graphs, and agentic automation across enterprise and national-level projects.
The ideal candidate combines deep expertise in data architecture, orchestration, and cloud infrastructure with hands-on experience using AI-assisted development tools to accelerate delivery and ensure scalability, reliability, and intelligence in every layer of the data stack.
Key Responsibilities
Architect and lead the design of end-to-end data pipelines using Apache Airflow, Azure Data Factory, and Google Cloud Composer.
Design and manage data lake, data warehouse, and OLAP architectures using MSSQL, SSIS, and SSAS.
Build and optimize ETL/ELT frameworks integrating structured, semi-structured, and unstructured data across APIs, databases, and streaming platforms.
Implement and maintain graph and NoSQL databases (e.g., Neo4j, Cosmos DB, MongoDB, Elasticsearch) to support semantic analytics, search, and recommendation systems.
Establish standards for data quality, lineage, governance, and observability.
Collaborate with AI engineers, data scientists, and business analysts to enable retrieval-augmented generation (RAG), knowledge graph, and agentic orchestration use cases.
Drive multi-cloud infrastructure automation through Terraform, Pulumi, or Bicep and CI/CD pipelines.
Champion AI-assisted coding practices (e.g., GitHub Copilot, ChatGPT, Cursor) to enhance productivity, testing, and documentation.
Mentor junior engineers, review designs, and promote best practices in data architecture, DevOps, and AI-native engineering.
Contribute to strategic roadmap planning, including capacity scaling, cost optimization, and data-driven innovation initiatives.
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
- 5–10 years of progressive experience in data engineering, data architecture, or analytics infrastructure roles.
- Deep expertise in SQL/T-SQL, performance optimization, and data modeling.
- Proven hands-on experience with SSIS and SSAS, including cube design and deployment.
- Advanced proficiency with Airflow (custom DAGs, operators, sensors, and monitoring).
- Solid understanding of Azure (Data Factory, Synapse, Blob Storage, Functions) and Google Cloud (BigQuery, Cloud Storage, Pub/Sub) ecosystems.
- Strong Python development skills for ETL, automation, and API integration.
- Experience with Docker, Kubernetes, and CI/CD workflows for data services.
- Hands-on experience with graph databases (Neo4j, Neptune, TigerGraph) and NoSQL stores (MongoDB, Cosmos DB, Elasticsearch, etc.).
- Demonstrated use of AI-powered development tools (GitHub Copilot, ChatGPT, Replit Ghostwriter, Code Interpreter) in professional workflows.
- Knowledge of data security, governance, and compliance frameworks (GDPR, ISO 27001, NIST)
