Role Summary
AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have 3-8 years of prior experience in data engineering, with a strong background in working on modern data platforms. This role offers an exciting opportunity to work on diverse projects, collaborating with cross-functional teams to design, build, and optimize data pipelines and infrastructure.
Responsibilities
- Design, develop, and maintain data pipelines using Databricks (PySpark / Spark SQL)
- Build and manage data pipelines across Bronze, Silver, and Gold layers using Delta Lake
- Implement ETL/ELT workflows for batch and near real-time processing
- Work with Databricks Workflows for orchestration and job scheduling
- Leverage Unity Catalog for data governance, access control, and metadata management
- Optimize Spark jobs, cluster configurations, and cost efficiency
- Collaborate with business and analytics teams to translate requirements into scalable data models
- Integrate data from multiple sources (APIs, databases, cloud storage)
- Ensure data quality, validation, and observability across pipelines
- Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring
Qualifications
- Bachelor’s degree in computer science, Engineering, or a related field.
- Overall 3+ years of prior experience in data engineering, with a focus on designing and building data pipelines
- Hands-on experience with Databricks platform and ecosystem
- Strong proficiency in Python (PySpark) and SQL
- Experience working with Delta Lake (ACID transactions, time travel, schema evolution)
- Good understanding of data warehousing concepts and dimensional modeling
- Familiarity with Unity Catalog (data governance, RBAC, lineage basics)
- Understanding of Spark performance tuning and optimization techniques
- Experience with cloud platforms (AWS / Azure / GCP)
- Working knowledge of Git and CI/CD practices
- Familiarity with implementing CI/CD processes or other orchestration tools is a plus.
Skills:- aws, PySpark, databricks and Python
IN
