- Design, develop, and implement robust data pipelines and ETL processes to extract, transform, and load data from various sources into our data warehouse or data lake using modern tools like Python, Spark, Impala, Airflow, and related technologies.
- Collaborate with stakeholders to understand data solutions requirements and translate them into technical specifications.
- Develop and maintain robust and scalable data pipelines to ensure efficient data flow and integration between various systems and applications.
- Perform data profiling, analysis, and data quality assessments to identify and resolve data issues.
- Optimize data integration workflows and processes for performance, reliability, and scalability.
- Troubleshoot complex data integration problems, conduct root cause analysis, and implement effective solutions.
- Mentor and provide guidance to junior data integration engineers, assisting in their professional development.
- Collaborate with data architects, database administrators, and other team members to ensure data integrity, security, and compliance.
- Stay updated with the latest advancements in data integration technologies and tools and recommend their adoption when applicable.
- Document all processes, configurations, and best practices for knowledge sharing and future reference.
- Collaborate with cross-functional teams to define and implement data governance and data management practices.
Bachelor’s degree in computer science, Information Systems, or a related field.Minimum of 3 years of experience as a Data Integration Engineer or similar role.Strong expertise in modern data integration tools like Python, Spark, Impala, Kafka Airflow, and related technologies.Proficiency in traditional ETL tools and relational database systems.Solid understanding of Datawarehouse and Bigdata concepts, methodologies, and best practices.Experience with data modeling, data pipelines, and data quality management.Strong problem-solving capabilities with the ability to analyze complex data integration issues.