AI-enhanced for better readability
HYR Global Source Inc: Azure Data Engineer - Durham, NC (W2 candidates)
Headquarters: North Carolina
Job Title: Azure Data Engineer
Location: Durham, NC (3 Days Hybrid)
Job Type: Full-Time
About the Role
We are seeking a highly skilled Azure Data Engineer to join our data engineering team in Minneapolis, MN. This hybrid role requires collaborating with cross-functional teams to build scalable data pipelines and deploy robust Azure-based data solutions. The ideal candidate will have strong experience in Azure Data Factory, Databricks, and Azure Data Lake, with proven ability to build and optimize real-time and batch data pipelines.
Responsibilities
- Build large-scale batch and real-time data pipelines using Azure cloud platform technologies.
- Design and implement high-performance data ingestion pipelines from multiple structured and unstructured sources using Azure Databricks.
- Lead the design and development of ETL, data integration, and migration solutions.
- Develop scalable and reusable frameworks for ingestion, processing, and transformation of data sets.
- Partner with data architects, analysts, engineers, and stakeholders to deploy enterprise-grade data platforms.
- Integrate end-to-end data pipelines - from source systems to target data repositories - ensuring high data quality, reliability, and consistency.
- Work with event-driven and streaming technologies for real-time data ingestion and processing.
- Support related project components including API integrations and search services.
- Evaluate tools and platforms based on performance and business requirements.
Required Skills
- Azure Data Factory
- Azure Databricks
- Azure Data Lake Storage (ADLS)
- Azure SQL Database and Data Warehouse
- Proven hands-on expertise in:
- Implementing and optimizing Azure cloud data solutions
- Performance tuning in the Databricks environment
- SQL, T-SQL, and/or PL/SQL
- Working knowledge of:
- Azure Storage Accounts
- Azure Data Catalog, Azure Data Analytics
- Logic Apps and Function Apps
- Familiarity with Agile methodology and tools like JIRA
- Experience handling end-to-end data ingestion projects in an Azure environment is preferred