Cloud Data Engineer

Cloud Data Engineer

DIGITALL

Sofia, Bulgaria

About the job

We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. The ideal candidate should have extensive experience in Azure Databricks, Data Lakehouse concepts, Python, SQL, and a strong background in working with various data sources and platforms.

Azure Databricks:

• Design, develop, and maintain data pipelines using Azure Databricks.
• Optimize and tune Spark jobs for performance and scalability.
• Implement Spark DataFrames and Pandas DataFrames for efficient data processing.

Data Lakehouse Concept:

• Implement and manage data lakehouse concepts for effective data storage and processing.
• Work with Delta Tables and Delta Live Tables to maintain versioned and transactional data.

Azure Data Factory:

• Design and implement data workflows using Azure Data Factory.
• Handle data integration from various sources, both on-premises and in the cloud.

Data Sources:

• Integrate data from diverse sources such as Azure Data Lake, Azure SQL, Azure Synapse, APIs, and other relevant sources.
• Expertise in Copy Activity, Mapping Data Flow, and handling various data formats.

API Integration:

• Experience with making API calls and processing JSON responses.
• Ability to handle pagination and complex API interactions.

Azure Analysis Services (SSAS):

• Experience with Azure SSAS for creating and managing tabular models.

Azure Key Vault and Data Lake Concepts:

• Familiarity with Azure Key Vault for secure storage of sensitive information.
• Knowledge of Azure Data Lake concepts and best practices.

Additional Skills:

• The candidate should be proficient in SQL and understand the usage of functions and stored procedures for efficient data handling. He or she should possess a deep understanding of database design and normalization and have experience in database and query performance tuning and optimization.
• Familiarity with dbt (Data Build Tool) for analytics engineering (optional but advantageous).

Your Qualifications:

• Bachelor’s or higher degree in Computer Science, Data Science, or related field.
• Proven experience as a Data Engineer with a focus on Azure technologies.
• Strong proficiency in Python, SQL, and Spark DataFrames.
• Excellent understanding of data integration, data processing, and data modeling concepts.
• Experience with cloud and on-premises data sources and platforms.

Apply Now