AI Evals For Engineers & PMs

It is a deep dive into bringing evals for LLM powered products. It gave a framework, process and some tools to apply evals to your apps.

Key Learnings

Three Gulfs Model

1. Gulf of Comprehension
   • Understanding your data nad how the LLM behaves with it.
2. Gulf of Specification
   1. Giving the LLM clear and precise instruction
   2. Why its hard. LLMs cannot gues your true intention. They need explicit and detailed instructions for what to do and what not to do.
3. Gulf of Generalization
   1. Ensure the bot performs well on new inuts it hasn't seen before.
   2. A perfect prompt can’t always fix the LLMs core inability to apply its knowledge to every possible new situation. LLMs may not generalize in the same way that humans do.

Analyze-Improve-Measure

1. Analyze
   1. Collect representative examples and categorize failure modes
   2. Pitfalls
      1. Outsourcing annotations
      2. Not looking at enough examples
2. Measure
   1. Translate qualitative insights into quantitative metrics
   2. Pitfalls
      1. Unaligned LLM judges
      2. Overfitting by testing LLM judges on examples in their prompts
3. Improve
   1. Refine prompts, models and pipeline architecture
   2. Pitfalls
      1. Premature improvement
      2. Jumping to most complex solution first (eg fine-tuning)