# What is Flovia?

Understand how agents discover, choose, and use your product. Turn the point of failure into your next improvement.

Source: https://flovia.dev/docs/

## An evaluation loop for agent experience

Flovia is an Agent Experience Optimizer for teams that build APIs and MCP tools. It evaluates whether an AI agent can choose your product, use it correctly, and complete the user’s task. Each evaluation connects a point of failure to a concrete fix and a retest.

An agent can mention your product and still call the wrong endpoint or supply the wrong arguments. Flovia follows the journey past visibility to find the API, tool-description, or documentation change that would unblock successful use.

AX Simulation is Flovia’s evaluation methodology. It turns agent behavior into a controlled experiment. It combines task-based scenarios, native function calling, contract validation, and stage-level scoring to locate where the agent gets stuck between choosing a product and completing a task. Paired experiments then compare a targeted change under matched conditions. Your private dashboard connects the evidence to a fix and its retest.

To use Flovia, request an evaluation and agree on its scope with the team. Results, diagnosis, recommended changes, and retests are delivered through a private dashboard.

- **Define:** Tasks + success criteria
- **Simulate:** Models + tool environment
- **Diagnose:** Stages + evidence
- **Improve:** Change + controlled retest

## Example: improving API guidance

For a blockchain data API, we compared AI answers before and after a documentation update. Before the update, answers either failed to identify how to retrieve transaction IDs only or suggested a capability intended for another purpose. After the update, they explained the correct method.

For real-time streaming, answers after the update also correctly explained that getting connection details does not start the stream: the developer needs to run the connection code.

- [Explore the documentation improvement](https://flovia.dev/docs/improvement-loop/): See the before-and-after comparison for each capability.

## Review your results in the dashboard

Review the evaluation summary, prioritized changes, supporting run evidence, and implementation prompts. Understand why a change is recommended before applying it.

- [Explore the evaluation dashboard](https://flovia.dev/docs/improvement-loop/#dashboard-example): Follow one example from overview to finding and evidence.

## Technical methods that inform product decisions

The engine combines experimental design, executable task contracts, and traceable measurement. Each evaluation uses the methods suited to its question and execution path.

| Method | What it makes observable | Product decision |
| --- | --- | --- |
| Paired comparisons + Latin-square candidate balancing | Behavior before and after a description change, with task and candidate position controlled in supported studies | Which description deserves a retest across more use cases? |
| Schema + semantic argument validation | Whether the chosen operation and its arguments satisfy the task contract | Should the tool name, parameter guidance, or operation boundary change? |
| Result checks + criterion-based answer assessment | Where returned data or the final answer falls short of the task | Which fields or interpretation guidance would unblock completion? |
| Cluster bootstrap + versioned evidence | Uncertainty across task groups where supported, and the inputs behind each finding | How much evidence supports the result, and what should be tested next? |

## From evaluation request to improvement

Your first evaluation is free. Start with a 30-minute call to choose a task an agent should complete using your API or MCP. Flovia delivers the initial results in your private dashboard.

1. Prepare your context. Share your product and documentation URLs, a representative task, and any points of confusion you already know about.
2. Scope the evaluation in a 30-minute call. Your team and Flovia agree on the task, models, environment, and success criteria.
3. Review the results. Flovia runs the evaluation and shows, in your dashboard, where agents struggled, the evidence behind each finding, and the recommended changes.
4. Apply a change. Your team reviews the recommendation and implements it. Enterprise customers can also use a GitHub-connected, semi-automated workflow while retaining review and deployment control.
5. Check the improvement. Flovia retests under comparable conditions. Review the before-and-after evidence together and decide what to improve next. Retesting follows the scope of your plan.

## What to prepare

| Input | Why it matters |
| --- | --- |
| Product documentation and interface descriptions | Define what the agent can learn and which operations are available. |
| Representative user requests | Anchor the evaluation in tasks that matter to your product. |
| Expected behavior and required facts | Make successful completion testable. |
| Relevant alternatives and constraints | Make selection meaningful, including cases where another tool is a better fit. |
| A proposed change, if you have one | Define a baseline and a variant for a controlled comparison. |

Request an evaluation through the website. Flovia runs the engine and delivers results, failure analysis, remediation guidance, and retest findings in a private dashboard. Your team reviews and applies the change in its own environment.

Before the run, agree on the model endpoints and the completion Judge configuration. Separately, decide who prepares the tool-response data, which tasks and repeats to run, and when to review the results. Recording these in the evaluation plan makes preparation responsibilities and the decision the result will support clear to everyone.

## Prepare your evaluation request

Evaluation brief · copy and fill in

```text
Product / documentation URL:
Task an agent should complete:
Expected result and required facts:
Relevant alternatives:
Known failure or proposed change:
Constraints and questions for Flovia:
```

- [Prepare a brief with your agent](https://flovia.dev/skills/prepare-flovia-evaluation/SKILL.md): A downloadable skill for reading the docs and drafting a request. It does not run a simulation.
- [Plans and pricing](https://flovia.dev/pricing): See the current plans and what they include.
- [Discuss an evaluation](https://flovia.dev/contact): Share your product and task with the team. Confirm scope and data-handling needs before sharing private material.

## Explore the methodology

- [Design a scenario](https://flovia.dev/docs/scenario-design/): Turn one product decision into a task with testable success criteria.
- [Inside the simulation engine](https://flovia.dev/docs/simulation-engine/): Follow the model calls, tool contracts, and scoring path.
- [The seven-stage agent funnel](https://flovia.dev/docs/agent-funnel/): See what each stage measures and how failures are located.
- [Controlled experiments](https://flovia.dev/docs/controlled-experiments/): Understand pairing, candidate rotation, and comparison design.
- [Metrics and evidence](https://flovia.dev/docs/metrics-and-evidence/): Read denominators, uncertainty, and the evidence behind a result.