• Data Engineering with Microsoft Fabrics:

1. Utilize Microsoft Fabrics to design, implement, and manage scalable, high-performance data pipelines that ingest and process large volumes of data.
2. Leverage Microsoft Fabrics features such as Lakehouse architecture, OneLake, and other data fabric components to integrate data from multiple sources into a unified platform.
3. Apply best practices for data engineering to ensure data quality, integrity, and security in Microsoft Fabrics environments.

• Azure Data Factory Integration:

1. Design and implement Azure Data Factory (ADF) pipelines to orchestrate data workflows, ensuring efficient and reliable data ingestion and transformation.
2. Automate data movement and transformation tasks across various data storage and processing services using ADF.
3. Monitor and troubleshoot ADF pipelines, ensuring optimal performance and error resolution.

• SQL Development:

1. Write and optimize complex SQL queries for data extraction, aggregation, and analysis, ensuring efficient processing of large datasets.
2. Develop stored procedures, views, and scripts to support data transformations and reporting.
3. Ensure the security and privacy of data by implementing appropriate SQL controls and policies.
4. Ability to work on different kinds of databases like My SQL, MariaDB, Microsoft SQL Server, and Dataverse.

• Data Architecture and Modeling:

1. Design and maintain data architecture, ensuring data storage solutions are aligned with business requirements and technical standards.
2. Create and maintain data models, ensuring accurate and efficient storage and retrieval of data.
3. Collaborate with stakeholders to understand data requirements and deliver solutions that meet business needs.

• Data Governance and Quality:

1. Implement data governance best practices, ensuring data is accurate, consistent, secure, and compliant with organizational standards.
2. Develop and enforce data quality controls and validation processes across all stages of data engineering workflows.
3. Monitor data pipelines and troubleshoot issues related to data quality, integrity, and performance.

• Collaboration and Stakeholder Communication:

1. Work closely with data scientists, analysts, product managers, and other engineers to deliver data solutions that support business objectives.
2. Collaborate with cross-functional teams to gather and refine data requirements, ensuring alignment with the organization's goals.
3. Provide technical guidance and expertise in data engineering with the best practices and tools to drive innovation and efficiency.

• Performance Tuning and Optimization:

1. Continuously monitor and optimize the performance of data pipelines, databases, and data models to ensure they are operating at maximum efficiency.
2. Conduct regular performance tuning of Azure, Databricks, PySpark, and SQL environments to handle large-scale data processing.

Education:Bachelor’s degree in computer science, Information Technology, Data Science, or a related field. Advanced degrees or relevant certifications in data engineering or cloud platforms (e.g., Azure) are a plus.Minimum of 3+ years of experience as a Data Engineer, with at least 1 year of hands-on experience in Microsoft Fabrics.Proven experience in working with Databricks, PySpark, Azure Data Factory, SQL, and other data engineering tools and platforms.Strong understanding of ETL processes, data integration, and data transformation techniques.Proven experience in building and maintaining data warehouses and applying data governance.Expertise in Microsoft Fabrics, including experience with Lakehouse architecture, OneLake, and data fabric integrations.Proficiency in Databricks and PySpark for building and optimizing scalable data pipelines.In-depth experience with Azure Data Factory for orchestrating and automating data workflows.Strong SQL skills, with the ability to write and optimize complex queries for large datasets and analytical functions.Knowledge of cloud platforms, particularly Microsoft Azure, including data storage, security, and cloud-native architectures.Familiarity with data modeling concepts and experience creating scalable data architectures.Understanding data governance, data security, and data privacy with the best practices.Knowledge of Microsoft Power Platform, especially Power BI and Power Apps.Excellent problem-solving skills with a focus on delivering scalable and efficient solutions.Strong communication skills, both verbal and written, with the ability to convey technical concepts to non-technical stakeholders.Team-oriented with the ability to collaborate effectively with backend developers, analysts, and other engineers.Strong attention to details, with a commitment to delivering high-quality, reliable data solutions.Certifications in Microsoft Azure, Databricks, or other relevant technologies.Experience with data visualization tools such as Power BI or Tableau.Familiarity with other programming languages such as Python, Java, or Scala for data processing and integration tasks.Microsoft Fabric Analytics certified.Knowledge of Microsoft Dynamics.