Dice is the leading career destination for tech experts at every stage of their careers. Our client, Learn Beyond Consulting LLC, is seeking the following. Apply via Dice today!
Job Description
We are seeking an experienced
Data Engineer with strong hands-on expertise in
PySpark, Spark, SQL, and Python to join our team in
New Jersey. The ideal candidate will have experience designing and building scalable data pipelines, processing large datasets, and working with distributed data processing frameworks.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark and Spark.
- Process and transform large volumes of structured and unstructured data.
- Write optimized SQL queries for data extraction, transformation, and analysis.
- Develop robust data processing solutions using Python.
- Work closely with data analysts, data scientists, and business teams to support data-driven initiatives.
- Ensure data quality, performance optimization, and reliability of data pipelines.
- Troubleshoot and resolve data processing issues in production environments.
Required Skills
- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with PySpark and Apache Spark.
- Advanced proficiency in SQL and Python.
- Experience building large-scale ETL/ELT pipelines.
- Strong understanding of data processing and distributed computing concepts.
- Experience working with large datasets and data warehousing solutions.
Preferred Qualifications
- Experience with cloud platforms (AWS / Azure / Google Cloud Platform).
- Knowledge of data lake and big data architectures.
- Experience with performance tuning and optimization in Spark environments.
US
