Get Experiment Run Metrics

Get metrics for an experiment run.

Requires the run_id query parameter. Use view to choose the breakdown (summary, by_document, by_target, or votes), and narrow with document_id or target_path. By default each score-bearing row also carries a prior_score from the previous completed run; pass include_prior=false to omit it or prior_run_id to compare against a specific run.

GET /v1/workflows/experiments/metrics

Python

from retab import Retab

client = Retab()

summary = client.workflows.experiments.metrics.get(
    "exprun_2",
    view="summary",
)

target_view = client.workflows.experiments.metrics.get(
    "exprun_2",
    view="by_target",
    target_path="line_items.*.unit_price",
)

200

{
  "experiment_id": "exp_abc",
  "run_id": "exprun_2",
  "kind": "summary",
  "view": "summary",
  "definition_fingerprint": "deadbeef",
  "block_kind": "extract",
  "score": 0.83,
  "prior_score": 0.79,
  "prior_run_id": "exprun_1",
  "documents": [
    {
      "id": "expdoc_1",
      "filename": "a.pdf",
      "score": 0.91,
      "prior_score": 0.85
    }
  ],
  "aggregate": {
    "likelihoods": { "total": 0.92, "vendor.name": 0.78 }
  }
}

Get metrics for a specific experiment run. Metrics are consensus likelihoods on the [0.0, 1.0] scale where 0.0 is low agreement and 1.0 is total agreement.

The view query parameter selects one of four successful response shapes. In responses, branch on kind; it is the shared discriminator for success and data-state payloads such as stale_metrics and no_metrics.

View	Use it to
summary	Read the overall score plus block-specific aggregates. Start here.
by_document	Drill into one document and see all its targets, sorted ascending. Requires document_id.
by_target	Drill into one target and see its score across every document. Requires target_path.
votes	See the per-voter consensus rows for one document/target cell. Requires both document_id and target_path.

Pass include_prior=false to omit prior-run comparison fields, or prior_run_id=... to override which run is treated as the prior.

Python

from retab import Retab

client = Retab()

summary = client.workflows.experiments.metrics.get(
    "exprun_2",
    view="summary",
)

target_view = client.workflows.experiments.metrics.get(
    "exprun_2",
    view="by_target",
    target_path="line_items.*.unit_price",
)

200

{
  "experiment_id": "exp_abc",
  "run_id": "exprun_2",
  "kind": "summary",
  "view": "summary",
  "definition_fingerprint": "deadbeef",
  "block_kind": "extract",
  "score": 0.83,
  "prior_score": 0.79,
  "prior_run_id": "exprun_1",
  "documents": [
    {
      "id": "expdoc_1",
      "filename": "a.pdf",
      "score": 0.91,
      "prior_score": 0.85
    }
  ],
  "aggregate": {
    "likelihoods": { "total": 0.92, "vendor.name": 0.78 }
  }
}

Authorizations

Authorization · string · header · required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Query Parameters

run_id · string · required

view · enum<string> · default: summary

Available options: summary, by_document, by_target, votes

document_id · string | null

target_path · string | null

include_prior · boolean · default: true

prior_run_id · string | null

Response

200

application/json

Successful Response

ExperimentSummaryMetricsResponse

ExperimentByDocumentMetricsResponse

ExperimentByTargetMetricsResponse

ExperimentVotesMetricsResponse

ExperimentMetricsStaleError

ExperimentMetricsMissingError

Run-level summary plus block-specific diagnostics.

prior_run_id + prior_score populate when the request opts into prior-comparison and a completed prior run exists.

experiment_id · string · required

run_id · string · required

block_type · enum<string> · required

Available options: extract, classifier, split, for_each

kind · string · default: summary

Allowed value: "summary"

view · string · default: summary

Allowed value: "summary"

definition_fingerprint · string | null

score · number | null

prior_score · number | null

documents · ExperimentSummaryMetricDocument · object[]

Show child attributes

aggregate · ExperimentExtractSummaryAggregate · object

Extract-only diagnostics attached to the summary response.

ExperimentExtractSummaryAggregate

ExperimentConfusionSummaryAggregate

Show child attributes

prior_run_id · string | null