# Tayyar dataset — v1.0

A frozen release of the Tayyar MENA political-position dataset. Frozen
2026-07-25; corrected and re-cut 2026-10-27 before its first public
release (see "Corrections" below). Instrument and methodology: **"The Model as
One Rater Among Several"** (arXiv:2606.23042, https://arxiv.org/abs/2606.23042)
and https://tarekgara.com/tayyar/methodology.

**Cite this exact version** (v1.0 and its DOI), never the live site; the live
dataset keeps changing. Citation metadata is in `CITATION.cff`; file integrity
in `SHA256SUMS.txt` (`sha256sum -c SHA256SUMS.txt`).

**License:** dataset CC BY-NC-SA 4.0; quoted source texts remain with their
rights-holders (quotation for research and criticism). Full terms in
`TAYYAR_LICENSE.md`.

## What this is, and what it is not

Scores are the output of a measurement instrument: a panel of nine language
models from eight laboratories, each scoring actors blind against written axis
rubrics. They are not authoritative judgments about any actor. The panel's
**reliability** is measured (Krippendorff's alpha 0.86 interval, 0.87 ordinal,
over 998 cells carrying at least two models). Its **validity** is an open
program: no independent human coder has scored these cells yet, and the tables
that will carry that work (`tayyar_human_codings`, `tayyar_method_runs`) are
included and empty, which is the honest state.

## Key semantics

- **Rounds** (`tayyar_ratings.round`):
  - `r2` is the instrument round: nine models, with axis definitions, parties
    only (80 parties in 15 countries).
  - Rows with `round` empty and `rater_id = author-baseline` are the author's
    hand-coded prior. It enters the published median as one rater.
  - `pilot-grounded` is a 205-rating pilot in which Claude scored from cited
    documents; it is folded into Claude's value for a cell, so Claude still
    counts once.
  - `round-1`, `round-2`, `round-3` are an early blind re-coding probe by two
    language-model re-coders (`coder-b`, `coder-c`) over 63 cells, 21 per round,
    each scored once by each re-coder. They also count as raters in the
    published median for those cells.
  - `r1` (two models) and `r1b` (ten models) score without definitions. They
    are the baselines of the rubric experiment and never enter the median.
  - Round `r7-dvb` is not in this release. It belongs to a pre-registered study
    (declared vs behavioral) whose data ships with its own paper.
- **Published value of a cell** (`tayyar_aggregate_positions`, lens
  `composite`, `superseded_at` empty): each rater's ratings collapse to that
  rater's median, and the cell's score is the median across raters. Status is
  *settled* (two or more raters, MAD ≤ 1.5), *contested* (MAD ≥ 3) or
  *provisional* (otherwise, including single-rater cells).
- **Applicability:** an empty score means "does not apply" or "insufficient
  evidence", never zero. Zero is a substantive midpoint.
- **Lenses:** besides `composite`, 36 hand-coded `declared`/`behavioral` pairs
  from an early pilot are included for reference.
- **Scale:** −10 to +10 per axis. Pole definitions are in
  `tayyar_axes.scoring_conventions`.
- **Verification:** entities, quotes and events carry a `verification_status`.
  `tarek-verified` rows were checked by the author against their sources;
  `unverified` rows were not. Of 989 quotes, 101 are verified.
- **Import flags:** `tayyar_ratings.import_flag` is empty for every row you
  can use. 483 rows of the no-definition round r1b carry
  `r1b-key-mismatch`: all of GPT's r1b rows and the rows of Gemini's first r1b
  import (2026-06-16), whose entity keys did not match the rows. Many point at
  entity ids that exist in no table. Leave them out.
- **Model versions:** `tayyar_ratings.model_version` is recorded where the
  scoring session logged it. It is empty for Gemini, Grok, Kimi and DeepSeek r2
  rows and part of Claude's; the versions those sessions ran are listed in the
  paper's appendix.

## Corrections in this re-cut

- tayyar_ratings: 154 GPT r2 rows moved to the parties their rationales name (CSV-import entity mismatch, 2026-06-17); 13 parties affected
- tayyar_aggregate_positions: 168 composite cells of those 13 parties re-synthesized (137 changed)
- tayyar_documents: 7 documents rejected in PI review and 1 placeholder record removed; a curator note removed from one title and translation
- tayyar_documents: 12 curator comments ("[NOTE: ...]") and 4 internal pipeline remarks removed from the source text of 6 documents; no rating cites these documents
- tayyar_ratings: new column import_flag; 483 r1b rows from two imports whose keys did not match flagged 'r1b-key-mismatch' and excluded from every analysis
- manifest checksum corrected; SHA256SUMS now covers every file

The GPT rows were attached to the wrong entities by the import of the returned
CSV file: nine entity ids that exist in no table, and three real parties
carrying another party's ratings. Each block's written rationales name the
party it was meant for, which is how the rows were reassigned. Where two
parties' rows collided on one key during the import, one was lost: GPT has no
ratings for the Free Destourian Party (Tunisia) or the Socialist Forces Front
(Algeria), and two for the Islamic Salvation Front. Those cells rest on the
other eight models.

The same kind of key mismatch hit two imports in the no-definition round r1b.
Those rows could not be reassigned, because many carry no rationale to read the
party from, so they are kept and flagged in `import_flag` instead.

## Contents

- `tayyar_countries.csv` / `tayyar_countries.json` — 20 rows
- `tayyar_axes.csv` / `tayyar_axes.json` — 20 rows
- `tayyar_parties.csv` / `tayyar_parties.json` — 98 rows
- `tayyar_politicians.csv` / `tayyar_politicians.json` — 274 rows
- `tayyar_party_alliances.csv` / `tayyar_party_alliances.json` — 4 rows
- `tayyar_aggregate_positions.csv` / `tayyar_aggregate_positions.json` — 1195 rows
- `tayyar_ratings.csv` / `tayyar_ratings.json` — 12867 rows
- `tayyar_analyses.csv` / `tayyar_analyses.json` — 0 rows
- `tayyar_documents.csv` / `tayyar_documents.json` — 425 rows
- `tayyar_quotes.csv` / `tayyar_quotes.json` — 989 rows
- `tayyar_events.csv` / `tayyar_events.json` — 353 rows
- `tayyar_bills.csv` / `tayyar_bills.json` — 61 rows
- `tayyar_comparison_essays.csv` / `tayyar_comparison_essays.json` — 20 rows
- `tayyar_human_codings.csv` / `tayyar_human_codings.json` — 0 rows
- `tayyar_irr_sample.csv` / `tayyar_irr_sample.json` — 212 rows
- `tayyar_second_rater_codings.csv` / `tayyar_second_rater_codings.json` — 126 rows
- `tayyar_method_runs.csv` / `tayyar_method_runs.json` — 0 rows