# LayerLens

> LayerLens builds Stratix, continuous evaluation infrastructure for AI. Stratix evaluates 175+ models across 52+ benchmarks, scores agent traces with LLM judges, and provides pre-deployment quality gates for enterprise AI teams.

## Docs

- [What is Stratix](https://layerlens.ai/what-is-stratix): Product overview, six-stage workflow, key capabilities
- [Agentic Evaluations](https://layerlens.ai/products-agentic-evals): Trace-level evaluation for AI agents
- [Model Catalog](https://stratix.layerlens.ai): Browse and compare 175+ models across 52+ benchmarks
- [Blog](https://layerlens.ai/blog): Benchmark analysis, evaluation guides, AI failure case studies
- [Stratix SDK on GitHub](https://github.com/LayerLens/stratix-python): Python SDK, 70+ runnable samples, open source
- [Migration from LangSmith](https://layerlens.ai/blog/from-langsmith-to-stratix): Step-by-step migration guide
- [Migration from MLflow](https://layerlens.ai/blog/mlflow-eval-library-vs-production-platform): Evaluation library vs production infrastructure

## Key Concepts

- **Continuous evaluation:** Score production traces on a recurring schedule, not just at deployment time. Catch regressions before users do.
- **LLM judges:** Subjective evaluation with natural language criteria. Versioned, auditable, optimizable via GEPA (judge optimization). Define scoring criteria in plain English.
- **Agentic evaluations:** Grade multi-step agent traces with assertions, deterministic rules, and LLM judges combined. Evaluate whether the agent took the right path, not just whether it got the right answer.
- **Deliberation panels:** Multi-model jury panels achieving 96-98% human alignment. Three judges that disagree force the system to pick one and explain why. Single-model judges top out around 70%.
- **Six-stage workflow:** Select (pick the model), Build (instrument your app), Observe (inspect production traces), Evaluate (score outputs), Improve (tune prompts and judges), Govern (enforce quality gates in CI/CD).
- **Code graders:** Deterministic checks with no LLM calls. Exact match, regex, JSON-schema validity, semantic similarity, Flesch-Kincaid. 100% reproducible, audit-trail ready.
- **Trace evaluations:** Score agent and LLM traces with judges on recurring schedules. Filter by judge, trace, outcome, time range. Returns per-step reasoning chains.
- **RAG evaluation:** Three dimensions: retrieval quality (precision@k, recall@k), faithfulness (every claim supported by context), answer quality (end-to-end usefulness).

## Company

LayerLens is headquartered in Atlanta, GA. Founded 2023. ~11 employees. CEO: Archie Chaudhury. Chief Product Officer: Marc Mercuri.

Stratix serves AI/ML engineers evaluating models before deployment, platform teams building evaluation infrastructure, and compliance/governance leads who need audit-ready evidence that AI systems meet quality standards.

## Comparison

LayerLens Stratix is vendor-neutral continuous evaluation infrastructure. It differs from:
- **LangSmith** (LangChain ecosystem): Stratix evaluations work across all frameworks, not just LangChain. No vendor lock-in.
- **Arize AI** (observability-first): Stratix is verification-first. Arize answers "what happened?" Stratix answers "should we ship this?"
- **Braintrust** (trace-driven eval with static scorers): Stratix adds GEPA judge optimization, multi-model jury panels, and a public model catalog at scale.
- **MLflow** (experiment tracking library): Stratix is the production infrastructure MLflow's eval library assumes you will build yourself.