AI Engineer – Agentic AI & LLM Applications

Job Summary

We are seeking an AI Engineer to help transition enterprise AI capabilities from prototype to production. In this role, you will design, build, and deploy robust Agentic AI systems capable of planning, reasoning, and executing complex business workflows.

You will combine strong software engineering skills with hands-on expertise in LLMs, AI agents, RAG, orchestration frameworks, tool calling, evaluation, observability, and LLMOps. You will also experiment with emerging models, frameworks, and tools to identify technologies that can deliver measurable business value.

The ideal candidate is a hands-on and forward-thinking engineer who enjoys experimentation but understands what is required to operate AI systems reliably in production. You will work closely with product managers, business stakeholders, data engineers, and platform teams to deliver AI solutions that address real-world enterprise needs.

Project Overview

About the Client

Our client is a leading multi-brand technology solutions provider serving business, government, education, and healthcare customers across the United States, United Kingdom, and Canada.

The organization provides a broad range of technology offerings, spanning hardware and software through integrated IT solutions including security, cloud, data center, and networking.

About the Project

The project focuses on advancing enterprise AI capabilities from experimentation and prototyping into reliable, scalable, production-grade Agentic AI solutions.

The team is developing AI agents capable of interacting with enterprise systems, retrieving and reasoning over business data, invoking tools, maintaining state, and executing multi-step workflows with appropriate human oversight.

This role provides an opportunity to work at the intersection of AI engineering, software engineering, LLM applications, RAG, agent orchestration, and production AI operations.

Key Responsibilities

Agentic AI Engineering & Orchestration

• Architect, design, develop, and deploy complex Agentic AI workflows using modern AI technologies and the Microsoft AI ecosystem.
• Build multi-agent systems capable of handling complex workflows, loops, interruptions, and human-in-the-loop interventions.
• Design reliable agent orchestration patterns using frameworks such as LangChain, LangGraph, or equivalent technologies.
• Develop tool-use and function-calling capabilities that allow LLMs to safely interact with internal APIs, databases, and third-party SaaS platforms.
• Integrate agents with enterprise platforms such as Salesforce, Workday, ServiceNow, and other business systems.
• Design agent state-management solutions supporting memory, conversational history, context management, and persistence.
• Implement appropriate safeguards around agent actions, tool access, and autonomous execution.

LLM & Model Engineering

• Integrate and evaluate commercial and enterprise LLM APIs, including OpenAI, Azure OpenAI, Anthropic, Gemini, and similar providers.
• Experiment with emerging models, agent frameworks, tools, and techniques to identify opportunities for enterprise adoption.
• Design prompts and agent instructions using advanced techniques such as ReAct, few-shot prompting, structured outputs, and other appropriate approaches.
• Optimize applications to handle model limitations including rate limits, context-window constraints, latency, non-deterministic responses, and transient failures.
• Evaluate model performance across different use cases and select appropriate models based on quality, latency, cost, and business requirements.

RAG & Data Engineering

• Design and implement production-grade Retrieval-Augmented Generation (RAG) pipelines.
• Optimize document ingestion, chunking, embeddings, vector indexing, retrieval, and re-ranking strategies.
• Work with vector databases such as Pinecone, Weaviate, pgvector, or equivalent technologies.
• Collaborate with Data Engineering teams to develop and maintain high-quality golden datasets for agent and RAG evaluation.
• Implement strategies to improve retrieval accuracy, relevance, freshness, and context quality.
• Ensure data pipelines support reliable and secure consumption by AI agents.

LLMOps, Evaluation & Quality

• Develop automated evaluation pipelines for AI applications and agents.
• Implement LLM-as-a-Judge and other evaluation approaches to measure accuracy, relevance, hallucination, safety, and task completion.
• Integrate AI quality gates into CI/CD pipelines to prevent poorly performing agent versions from reaching production.
• Establish evaluation datasets, benchmarks, regression tests, and quality metrics for AI systems.
• Implement production observability and tracing to monitor agent workflows, model calls, tool usage, latency, errors, and failures.
• Use tracing and observability platforms to investigate agent behavior and troubleshoot production issues.
• Monitor token consumption, model usage, latency, and infrastructure costs.
• Optimize prompts, context size, caching, model selection, and application architecture to improve cost and performance.

Production Engineering

• Transform AI prototypes and proof-of-concepts into maintainable, scalable, production-ready applications.
• Apply software engineering best practices including testing, version control, code reviews, documentation, CI/CD, and automated quality controls.
• Design systems that account for the probabilistic nature of AI while maintaining predictable business outcomes.
• Implement appropriate error handling, retries, fallbacks, validation, and human-in-the-loop controls.
• Ensure AI applications meet enterprise requirements for reliability, security, scalability, and maintainability.

Innovation & Collaboration

• Evaluate emerging AI models, frameworks, agent protocols, and technologies.
• Identify opportunities to standardize emerging technologies across the enterprise.
• Collaborate with Product Managers and business stakeholders to translate business problems into practical AI solutions.
• Communicate technical concepts and AI limitations clearly to non-technical stakeholders.
• Document architecture, technical designs, implementation decisions, and operational practices.
• Share knowledge and establish engineering best practices across the AI development community.

Required Skills & Experience

Core Engineering

• Bachelor's degree with 5+ years of software engineering experience, including exposure to AI/ML applications; or
• 9+ years of software engineering experience with exposure to AI/ML applications.
• Strong hands-on Python development experience.
• Strong software engineering fundamentals, including APIs, distributed systems, testing, version control, and production application development.
• Experience building and deploying production software rather than only prototypes or proof-of-concepts.

Agentic AI

• Hands-on experience designing and building AI agents / Agentic AI solutions.
• Experience implementing multi-step agent workflows, tool calling, function calling, and agent orchestration.
• Experience with LangChain, LangGraph, or comparable agent orchestration frameworks.
• Understanding of agent state, memory, context management, and human-in-the-loop patterns.

LLM Engineering

• 2+ years of hands-on experience building applications with LLMs.
• Experience integrating LLM APIs such as OpenAI, Azure OpenAI, Anthropic, Gemini, or equivalent.
• Strong understanding of prompt engineering and LLM application design.
• Experience handling production challenges including rate limits, context-window limitations, latency, failures, and non-deterministic outputs.

RAG & Vector Databases

• Experience designing and implementing RAG solutions.
• Hands-on experience with vector databases such as Pinecone, Weaviate, pgvector, or equivalent.
• Understanding of embeddings, chunking, vector search, metadata filtering, retrieval optimization, and re-ranking.

LLMOps & Production AI

• Experience implementing AI evaluation and testing strategies.
• Experience with automated evaluation, regression testing, or LLM-as-a-Judge approaches.
• Experience with AI observability, tracing, and production monitoring.
• Understanding of AI cost optimization, including token usage, caching, prompt optimization, and model selection.

Preferred Skills

• Experience with Microsoft AI technologies and the Azure AI ecosystem.
• Experience with Azure OpenAI.
• Experience with enterprise SaaS integrations such as Salesforce, Workday, or ServiceNow.
• Experience with agent evaluation frameworks and AI observability platforms.
• Experience implementing multi-agent architectures.
• Experience with structured outputs and tool/function calling.
• Experience with AI security, guardrails, and responsible AI practices.
• Experience with CI/CD and MLOps/LLMOps platforms.
• Experience evaluating emerging agent protocols and connectivity standards.
• Experience with Kubernetes and cloud-native application deployment.

Technical Environment

• Language: Python
• LLMs: OpenAI, Azure OpenAI, Anthropic, Gemini
• Agent Frameworks: LangChain, LangGraph, and similar frameworks
• Vector Databases: Pinecone, Weaviate, pgvector
• AI Architecture: Agentic AI, Multi-Agent Systems, RAG
• Enterprise Integration: APIs, SaaS platforms, databases, enterprise applications
• AI Quality: LLM-as-a-Judge, evaluation datasets, regression testing
• Observability: AI tracing, distributed tracing, application monitoring
• Cloud: Microsoft Azure and enterprise cloud environments
• Engineering: Python, Git, CI/CD, automated testing

Key Competencies

• Agentic AI Architecture
• LLM Application Development
• Python Engineering
• RAG & Retrieval Engineering
• Agent Orchestration
• Tool & Function Calling
• LLM Evaluation
• AI Observability
• Production AI Engineering
• AI Cost & Performance Optimization

Education

• Master's degree in Computer Science, Artificial Intelligence, Machine Learning, Software Engineering, Data Science, or a related technical discipline is preferred.

Why This Position?

This is an opportunity to work on the transition from AI experimentation to enterprise-scale production AI.

You will have the opportunity to:

• Build sophisticated Agentic AI systems that solve real business problems.
• Work hands-on with rapidly evolving LLMs, models, and AI frameworks.
• Design production-grade architectures rather than isolated AI prototypes.
• Influence enterprise standards for agent orchestration, RAG, evaluation, and LLMOps.
• Collaborate with product, business, data, platform, and engineering teams.
• Help define how AI agents are safely and reliably deployed across the enterprise.