GOSST T&L - AI/ML Role for LLM Accuracy & Model DevelopmentJob Description GOSST Turnover & Launch is looking for a hands-on ML/AI engineer (contractor, L5 equivalent) to join our team and drive LLM accuracy improvements across our product portfolio. This role sits at the intersection of applied machine learning and operational technology - you will fine-tune models, build evaluation frameworks, and improve AI-driven automation for our Turnover & Launch tools (OrderPad, UTP, Field Installation, and supporting agentic systems).

Key Responsibilities:

• Fine-tune and optimize Large Language Models (LLMs) for domain-specific tasks including code estimation, sprint planning, and operational automation.

• Design and implement evaluation frameworks to measure model quality — including automated metrics, LLM-asjudge, and human evaluation workflows.

• Build and maintain RAG (Retrieval-Augmented Generation) pipelines that ground model outputs in codebase and operational data.

• Develop prompt engineering strategies and optimize prompt chains for multi-step agentic workflows.

• Collaborate with SDEs and SDMs to integrate model improvements into production services (Amazon Bedrock, SageMaker, or equivalent).

• Analyze model failure modes, identify accuracy gaps, and propose targeted improvements.

• Establish baseline metrics and track accuracy improvements over time with dashboards and reporting.

• Stay current on LLM advancements and recommend adoption of new techniques (LoRA, DPO, constitutional AI, etc.) where applicable.