Evaluations

Simulation is now available in public preview.

Simulation tests agent behavior through generated, multi-turn conversations using scenario sets, user simulator models, judge models, and conversational metrics.

The Agent Evaluations MCP server tool has been renamed to Evaluations.

Custom metrics are now available for DigitalOcean Evaluations.

You can define your own metrics to evaluate model behavior against criteria specific to your use case.

DigitalOcean Evaluations is now generally available.

Use Evaluations to create test cases, run evaluation datasets, and measure model performance against selected metrics.

Presets are now available for DigitalOcean Evaluations.

You can save and reuse evaluation configurations, including the candidate model, system prompt, hyperparameters, judge model, and metrics.

Model Evaluations is now renamed to DigitalOcean Evaluations.