About the Role

We're seeking an experienced AI Engineer to join our team. In this role, you will develop and design specialized DSPy optimizers for Large Language Models, creating custom optimization techniques to enhance our LLM agent performance.

What You'll Do

- Design and implement optimizers for LLM-based applications

- Create novel optimization approaches to improve AI agent capabilities

- Develop evaluation methodologies to measure agent effectiveness

- Implement trace-based optimization and programmatic prompt tuning strategies

- Collaborate on improving our AI agent infrastructure

Technical Must-Haves

- Experience designing LLM-based agents with retrieval-augmented generation (RAG)

- Proficiency in AI evaluation frameworks, NLP fundamentals (tokenization, embeddings, LLM architecture), and bias handling

- Systematic prompt engineering experience with experimental frameworks (tools like Weave or Langsmith)

- Strong Python back-end skills for data-intensive systems and data processing

- Demonstrable ability to balance token limits, cost, and speed while producing structured outputs

Requirements (Must Have)

- 3+ years of professional industry experience working with AI/ML technologies (internships not counted)

- Experience building industry-focused, commercial AI applications (not academic or research-only projects)

- Proven track record deploying and maintaining LLM-based applications in production with medium to large user bases (1000+ users)

- Experience handling real-world challenges in AI agent performance, reliability, and cost optimization in commercial settings

- Track record of systematically improving AI agent performance through iterative optimization

- Hands-on experience with debugging and fixing AI agent failures in production environments

- Experience analyzing user interactions with AI systems to drive improvements

Nice to Have

- Experience with DSPy or similar LLM optimization frameworks

- Familiarity with building custom optimization techniques

- Background in ML/AI research

- Experience with multiple LLM providers and models

Apply if you've successfully deployed and optimized AI applications serving real users at scale in industry settings and can share concrete examples of how your engineering improved performance in production environments.