Data Pipeline Development & Infrastructure Design, build, and maintain scalable data pipelines Develop real-time and batch data processing frameworks for structured and unstructured data. Implement ETL/ELT workflows to ingest data from various sources, ensuring high availability and performance . Optimize data storage and retrieval of data in (near) real time and batch processes Ensure cost-efficient and high-performance data infrastructure that scales with business needs. Data Solutions & AI-Driven Applications develop data pipelines for ML recommenders, search functionality, and AI-enhanced features . Develop and maintain data models, APIs, and integrations to support analytics and customer applications. Support eCommerce-related data solutions , including product recommendations, customer segmentation, and personalization models. Collaboration & Continuous Improvement Work closely with Data Architects, Analysts, and Product Teams to understand data requirements and deliver best-in-class solutions. Monitor and troubleshoot performance issues , ensuring high availability and efficiency of data pipelines. Continuously optimize cost, performance, and scalability of data engineering solutions.

Desired Candidate Profile

Qualifications

• 5+ years of experience as a Data Engineer , working with Azure and Databricks
• Strong expertise in data pipeline development, ETL workflows, and real-time/batch data processing .
• Experience in big data technologies, and large-scale data storage .
• Familiarity with data governance, security, and compliance best practices and tools
• Proficiency in Python, and SQL, for data transformation, data quality and automation.
• Strong experience in optimizing data performance and cost-efficiency in cloud environments .

Nice to Have

• Experience with eCommerce data solutions , such as customer segmentation, recommendation engines, and personalization models.
• Knowledge of event-driven architectures, streaming data processing, and real-time analytics .
• Familiarity with LLM-based AI applications and OpenAI or similar APIs .