Job Description

Roles & Responsibilities

Job Description Data Pipeline Development & Infrastructure Design, build, and maintain scalable data pipelines Develop real-time and batch data processing frameworks for structured and unstructured data. Implement ETL/ELT workflows to ingest data from various sources, ensuring high availability and performance . Optimize data storage and retrieval of data in (near) real time and batch processes Ensure cost-efficient and high-performance data infrastructure that scales with business needs. Data Solutions & AI-Driven Applications develop data pipelines for ML recommenders, search functionality, and AI-enhanced features . Develop and maintain data models, APIs, and integrations to support analytics and customer applications. Support eCommerce-related data solutions , including product recommendations, customer segmentation, and personalization models. Collaboration & Continuous Improvement Work closely with Data Architects, Analysts, and Product Teams to understand data requirements and deliver best-in-class solutions. Monitor and troubleshoot performance issues , ensuring high availability and efficiency of data pipelines. Continuously optimize cost, performance, and scalability of data engineering solutions.

Desired Candidate Profile

We are looking for a Senior Data Engineer to design, build, and optimize our Azure and Databricks-based data infrastructure . You will play a critical role in developing scalable ETL pipelines, data ingestion frameworks, and contribute to AI/ML-powered solutions that drive internal decision-making and customer-facing insights. This is a hands-on role that requires expertise in Azure Data Services, Databricks, and Python/SQL . You will work closely with Data Architects, Engineers, and Product teams to develop robust data solutions that power analytics, personalization, and AI-driven customer experiences .

• 5+ years of experience as a Data Engineer , working with Apache Spark or bySpark.
• Strong expertise in data pipeline development, ETL workflows, and real-time/batch data processing .
• Experience in big data technologies, and large-scale data storage .
• Familiarity with data governance, security, and compliance best practices and tools
• Proficiency in Python, and SQL, for data transformation, data quality and automation.
• Strong experience in optimizing data performance and cost-efficiency in cloud environments .
• Nice to Have Experience with eCommerce data solutions , such as customer segmentation, recommendation engines, and personalization models.
• Knowledge of event-driven architectures, streaming data processing, and real-time analytics .
• Familiarity with LLM-based AI applications and OpenAI or similar APIs .