{{ benchmark_label }} · Question {{ question_id }}

Back to Analytics Results Table

{% if not question %}

Question not found in the current dataset.

{% else %}

Context

{% if question.user_input %}

Instruction

{{ question.user_input if question.user_input else '[empty]' }}

{% endif %} {% if question.code_to_edit %}

Code to Edit

{{ question.code_to_edit }}

{% endif %}

Prefix

{{ question.prefix if question.prefix else '[empty]' }}

Suffix

{{ question.suffix if question.suffix else '[empty]' }}

Answer Options

{% if diff_enabled %} {% endif %}

{% for answer in question.answers %}

{{ answer.option_label }} (original {{ answer.original_label }})

{{ answer.text if answer.text else '[empty answer]' }}

{% if choices %}

Selected {{ choices.counts.get(answer.original_label, 0) }} times ({{ (choices.percentages.get(answer.original_label, 0) * 100) | round(1) }}%)

{% endif %}

{% endfor %}

Reviewer Summary

{% if stats %}

{{ stats.total }} Total reviews

{{ stats.matches }} Agreed

{{ stats.mismatches }} Disagreed

{{ (stats.accuracy * 100) | round(1) }}% Agreement rate

{% if probability is not none %}

{{ (probability * 100) | round(1) }}% Model accuracy

{% endif %}

{% if difficulty %}

Model votes: correct {{ difficulty.correct_votes }} / total {{ difficulty.total_votes }} (incorrect {{ difficulty.incorrect_votes }})

{% endif %} {% else %}

No reviewer data available for this question.

{% endif %}

Individual Responses

{% if rows %} {% for row in rows %} {% endfor %}

Reviewer	Selected	Accepted	Status	Comment	Submitted
{{ row.reviewer_id }}	{{ row.selected_answer }}	{{ row.accepted_label }}	{% if row.matches_accepted == 'true' %} Match {% elif row.matches_accepted == 'false' %} Disagree {% else %} N/A {% endif %}	{{ row.comment or '—' }}	{{ row.submitted_at }}

{% else %}

No recorded responses.

{% endif %}

{% if disagreements and disagreements.entries %}

Disagreements

Disagreement details

Option A: {{ (disagreements.percentages['A'] * 100) | round(1) }}% | Option B: {{ (disagreements.percentages['B'] * 100) | round(1) }}%

{{ entry.reviewer_id }} chose {{ entry.selected }} {% if entry.comment %}- {{ entry.comment }}{% endif %}

{% endif %} {% endif %}