SENIOR DATA ENGINEER - Python

Happiest Minds Technologies

3.6

92 reviews

Job Type   /   Job Level

Full-time   /   Others/Any

Job Location

Bengaluru, Karnataka, India

A free Jobstore account is required to proceed to the employer site.

Title

Senior Data Engineer

Skills (must Have)

5+ years of professional experience in data engineering or related fields.
Strong programming skills in Python, SQL and PySpark.
Advanced experience building & optimizing ETL/ELT pipelines using Azure and open-source tools:

Azure Data Factory (ADF)
Azure Databricks (Spark, Delta Lake)
Apache Airflow
Azure Functions or Azure Synapse Pipelines

Expert-level SQL development, including complex queries, stored procedures, analytical functions and performance tuning.
Strong experience with Azure Snowflake, including:

Warehouse tuning
Cost/performance optimization
Snowpipe, Streams, Tasks
Snowpark for advanced processing

Experience building scalable data models (Kimball, Data Vault, Lakehouse).
Strong experience with Azure Cloud ecosystem, including:

Azure Data Lake Storage (ADLS Gen2)
Azure Synapse Analytics (Serverless & Dedicated SQL Pools)
Azure Databricks
Azure Key Vault
Azure Event Hub / IoT Hub
Azure Monitor / Log Analytics

Experience integrating APIs, streaming data and external data sources.
Strong understanding of data governance & data quality frameworks:

Microsoft Purview (cataloging, lineage, classifications)
Collibra

Experience with testing frameworks: unit tests, integration tests, Great Expectations, dbt tests.
Hands-on with CI/CD pipelines using:

GitHub Actions
Azure DevOps
Jenkins

Experience working with Infrastructure as Code tools:

Terraform or Bicep

Skills (good To Have)

Experience with Azure Kubernetes Service (AKS) or Dockerized workloads.
Experience building high-performance APIs using FastAPI, Flask, or Django.
Experience with real-time/streaming systems:

Azure Event Hub
Azure Stream Analytics
Kafka

Familiarity with Microsoft Fabric including:

Lakehouse
Data Pipelines
Warehouses

Experience with ML Ops or Feature Store integration (Databricks Feature Store, Azure ML).
Experience with security frameworks (RBAC, ABAC, managed identities).

Responsibilities

Lead the design, development and deployment of high-performance ETL/ELT pipelines on Azure and Snowflake.
Partner with business stakeholders, architects and data consumers to understand requirements and build scalable data solutions.
Design and optimize Azure Lakehouse solutions using:

ADLS Gen2
Delta Lake
Azure Databricks
Synapse Analytics
OneLake (if using Microsoft Fabric)

Design and optimize data models to support BI, analytics and machine learning.
Build and maintain data ingestion frameworks for:

Batch pipelines
Real-time pipelines
API-based ingestion

Implement robust data quality, data validation and metadata management processes.
Drive performance tuning across Spark jobs, Snowflake warehouses, SQL pool queries and cost optimization.
Monitor and maintain production data systems using Azure-native monitoring tools.
Define, enforce and improve engineering standards, including automation, CI/CD and IaC best practices.
Collaborate closely with DevOps/Platform teams to automate data infrastructure deployments.
Troubleshoot complex issues across distributed systems, cloud networks and data platforms.
Mentor junior and mid-level engineers, conduct code reviews and guide best practices.
Maintain clear architectural and technical documentation.

Senior-Level Behavioral Expectations

Provide technical leadership and drive decision-making during architecture and design discussions.
Effectively communicate with both technical and non-technical audiences.
Strong ownership mindset and proactive approach to solving technical challenges.
Ability to break down complex requirements into actionable engineering tasks.
Promote continuous learning, experimentation and innovation within the data engineering team.

Python

A free Jobstore account is required to proceed to the employer site.

Jobs in India » Jobs in Karnataka » Jobs in Bengaluru » Senior Data Engineer - Python

More jobs

People also search

Search more Information Technology jobs Search more jobs in Bengaluru Search more Information Technology jobs in Bengaluru Information Technology jobs salary in Bengaluru

Jobseekers

Employer

SENIOR DATA ENGINEER - Python

Happiest Minds Technologies

More jobs

SENIOR DATA ENGINEER - Python

Senior Data Engineer - Python/SQL

Senior Data Engineer (Azure & Python)

Senior Software Engineer- Python, Data Engineer

Python Data Engineer

Python Data Engineer

People also search

Report Job