Public Preview: Evaluations with Intelligent Trace Sampling

Microsoft Foundry observability introduces intelligent trace filtering and sampling for evaluations in public preview. Instead of running evaluations against all production traces, Foundry now select a representative subset of traces using a multi-stage MinHash farthest-first diversity algorithm, including deduplication, hard filers, aggregation and diversity-based selection for evaluation. This approach improves cost efficiency while producing significantly higher lexical diversity and diverse coverage compared to random sampling. Intelligent sampling is particularly effective for: evaluation and benchmarks, rubric generation and finetuning dataset curation.

Learn more.