Senior Azure Data Engineer Job at eTeam, Redmond, WA

  • eTeam
  • Redmond, WA

Job Description

Job Title: Senior Azure Data Engineer

Location: Redmond, WA 98052

Work Setup: Hybrid

Duration: 12 Months

Experience: 8-10 Years


Job Description


We are looking for an experienced Senior Azure Data Engineer with 8-10 years of experience in data engineering and strong hands-on expertise in Python, SQL, Apache Spark, PySpark, and Azure Databricks .


The candidate will be responsible for developing and supporting scalable data pipelines, transforming large datasets, and optimizing data processing solutions using Azure cloud technologies.


Must Have Skills

  • Strong experience with Python and SQL
  • Hands-on experience with Apache Spark, PySpark, and Spark SQL
  • Strong experience with Azure Databricks , including:
    • Notebooks
    • Jobs/Workflows
    • Clusters
  • Experience with Azure Synapse
  • Strong knowledge of Delta Lake
  • Experience working with Parquet
  • Hands-on experience with ADLS Gen2
  • Strong understanding of data engineering and ETL/ELT concepts
Nice to Have Skills
  • Experience with Scala
  • ETL/ELT development
  • Experience with MERGE/Upsert operations
  • Knowledge of data partitioning and partition pruning
  • Strong Spark performance tuning experience
  • Experience handling data skew, shuffles, caching, broadcast joins, and partition optimization
  • Experience with Git
  • Experience with Azure DevOps and CI/CD
Key Responsibilities
  • Develop and maintain scalable ETL/ELT data pipelines
  • Extract, transform, clean, and load data from various source systems
  • Build data solutions using Azure Databricks, Spark, Delta Lake, Synapse, and ADLS Gen2
  • Implement MERGE/Upsert logic using Delta Lake
  • Design effective partitioning strategies for large datasets
  • Optimize Spark jobs for performance and cost
  • Manage data skew, shuffles, caching, partitions, and cluster resources
  • Collaborate with development and data teams to deliver reliable data solutions
  • Follow best practices for Git, Azure DevOps, and CI/CD where applicable

Job Tags

Similar Jobs

Zerega Consulting

Legal Assistant Job at Zerega Consulting

 ...Our client, a boutique family law firm in Tampa, is seeking a Legal Assistant to join its growing team. This is an excellent opportunity...  ...become an integral part of a collaborative practice.Family law experience is strongly preferred. However, the firm may consider a... 

ANGARAI

Business Analyst Job at ANGARAI

 ...clients who are US Govt. agencies.Job DescriptionWe are seeking a detail-oriented and experienced Business Analyst to support enterprise ERP modernization initiatives within an SAP environment.This role will document current-state business processes, support gap analysis... 

Cactus Communications

Remote-First Sales Manager, Scholarly Publishing Job at Cactus Communications

 ...Communications is looking for a Sales Manager to drive revenue growth across the United States. The role focuses on partnering with academic publishers and scholarly societies to build strategic partnerships and close deals. Candidates should have over 5 years of B2B sales... 

Queen Miami Beach

Pastry Sous Chef Job at Queen Miami Beach

 ...and provide an engaging escape from the everyday. Join us at Queen Miami Beach and be part of something extraordinary.PositionPastry Sous ChefLocationMiami Beach, FLJob Id255# of Openings1Company BackgroundMr. Hospitality is a fully integrated hospitality... 

Rocketlab

Payroll Accountant Job at Rocketlab

 ...Under these Regulations, you may be ineligible for this role if you do not hold citizenship of Australia, Japan, New Zealand, Switzerland, the European Union, or a country that is part of NATO, or if you hold ineligible dual citizenship or nationality. For more information...