{
  "description": "Daily numbers for each AI model we observe: how often it gives the same answer when asked twice, how its answers wobble, and how often it refuses.",
  "license": "CC BY 4.0",
  "license_url": "https://creativecommons.org/licenses/by/4.0/",
  "models": [
    {
      "days": [
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-09-26",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 77,
          "wobble": {
            "answer": 8,
            "format": 2,
            "refusal": 0,
            "wording": 13
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-09-27",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 83,
          "wobble": {
            "answer": 5,
            "format": 0,
            "refusal": 0,
            "wording": 12
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-09-28",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 79,
          "wobble": {
            "answer": 8,
            "format": 0,
            "refusal": 0,
            "wording": 13
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-09-29",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 72,
          "wobble": {
            "answer": 7,
            "format": 2,
            "refusal": 0,
            "wording": 19
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-09-30",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 75,
          "wobble": {
            "answer": 7,
            "format": 1,
            "refusal": 0,
            "wording": 17
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-01",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 78,
          "wobble": {
            "answer": 4,
            "format": 0,
            "refusal": 0,
            "wording": 18
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-02",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 75,
          "wobble": {
            "answer": 5,
            "format": 2,
            "refusal": 0,
            "wording": 18
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-03",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 73,
          "wobble": {
            "answer": 11,
            "format": 1,
            "refusal": 0,
            "wording": 15
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-04",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 78,
          "wobble": {
            "answer": 7,
            "format": 2,
            "refusal": 0,
            "wording": 13
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-05",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 77,
          "wobble": {
            "answer": 8,
            "format": 0,
            "refusal": 0,
            "wording": 15
          }
        },
        {
          "boundary_observed": 0,
          "boundary_refused": 0,
          "change": false,
          "date": "2026-10-06",
          "observed": 0,
          "planned": 100,
          "same_answer_twice": 0,
          "wobble": {
            "answer": 0,
            "format": 0,
            "refusal": 0,
            "wording": 0
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": true,
          "date": "2026-10-07",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 95,
          "wobble": {
            "answer": 2,
            "format": 1,
            "refusal": 0,
            "wording": 2
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-08",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 95,
          "wobble": {
            "answer": 3,
            "format": 1,
            "refusal": 0,
            "wording": 1
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": true,
          "date": "2026-10-09",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 100,
          "wobble": {
            "answer": 0,
            "format": 0,
            "refusal": 0,
            "wording": 0
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-10-10",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 100,
          "wobble": {
            "answer": 0,
            "format": 0,
            "refusal": 0,
            "wording": 0
          }
        }
      ],
      "endpoint": "anthropic/claude-haiku-4-5",
      "observation_ended": null,
      "page": "https://aimodelrecord.com/m/anthropic/claude-haiku-4-5.json"
    },
    {
      "days": [
        {
          "boundary_observed": 15,
          "boundary_refused": 7,
          "change": false,
          "date": "2026-10-08",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 47,
          "wobble": {
            "answer": 27,
            "format": 4,
            "refusal": 2,
            "wording": 20
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 7,
          "change": false,
          "date": "2026-10-09",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 47,
          "wobble": {
            "answer": 31,
            "format": 2,
            "refusal": 1,
            "wording": 19
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 7,
          "change": true,
          "date": "2026-10-10",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 49,
          "wobble": {
            "answer": 27,
            "format": 3,
            "refusal": 1,
            "wording": 20
          }
        }
      ],
      "endpoint": "anthropic/claude-haiku-5-5",
      "observation_ended": null,
      "page": "https://aimodelrecord.com/m/anthropic/claude-haiku-5-5.json"
    },
    {
      "days": [
        {
          "boundary_observed": 15,
          "boundary_refused": 4,
          "change": false,
          "date": "2026-10-03",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 42,
          "wobble": {
            "answer": 30,
            "format": 5,
            "refusal": 1,
            "wording": 22
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-04",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 42,
          "wobble": {
            "answer": 30,
            "format": 5,
            "refusal": 2,
            "wording": 21
          }
        },
        {
          "boundary_observed": 0,
          "boundary_refused": 0,
          "change": false,
          "date": "2026-10-05",
          "observed": 74,
          "planned": 100,
          "same_answer_twice": 34,
          "wobble": {
            "answer": 15,
            "format": 3,
            "refusal": 0,
            "wording": 22
          }
        },
        {
          "boundary_observed": 0,
          "boundary_refused": 0,
          "change": false,
          "date": "2026-10-06",
          "observed": 0,
          "planned": 100,
          "same_answer_twice": 0,
          "wobble": {
            "answer": 0,
            "format": 0,
            "refusal": 0,
            "wording": 0
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-07",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 39,
          "wobble": {
            "answer": 28,
            "format": 7,
            "refusal": 1,
            "wording": 25
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-08",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 43,
          "wobble": {
            "answer": 26,
            "format": 7,
            "refusal": 4,
            "wording": 20
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-09",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 37,
          "wobble": {
            "answer": 30,
            "format": 11,
            "refusal": 1,
            "wording": 21
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-10",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 43,
          "wobble": {
            "answer": 22,
            "format": 6,
            "refusal": 2,
            "wording": 27
          }
        }
      ],
      "endpoint": "anthropic/claude-opus-5-5",
      "observation_ended": null,
      "page": "https://aimodelrecord.com/m/anthropic/claude-opus-5-5.json"
    },
    {
      "days": [
        {
          "boundary_observed": 15,
          "boundary_refused": 6,
          "change": false,
          "date": "2026-09-26",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 43,
          "wobble": {
            "answer": 23,
            "format": 13,
            "refusal": 2,
            "wording": 19
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 7,
          "change": false,
          "date": "2026-09-27",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 37,
          "wobble": {
            "answer": 32,
            "format": 11,
            "refusal": 1,
            "wording": 19
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 8,
          "change": false,
          "date": "2026-09-28",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 39,
          "wobble": {
            "answer": 33,
            "format": 7,
            "refusal": 3,
            "wording": 18
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-09-29",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 39,
          "wobble": {
            "answer": 34,
            "format": 11,
            "refusal": 2,
            "wording": 14
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 6,
          "change": false,
          "date": "2026-09-30",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 38,
          "wobble": {
            "answer": 33,
            "format": 10,
            "refusal": 1,
            "wording": 18
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-01",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 44,
          "wobble": {
            "answer": 29,
            "format": 12,
            "refusal": 2,
            "wording": 13
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 7,
          "change": false,
          "date": "2026-10-02",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 40,
          "wobble": {
            "answer": 29,
            "format": 13,
            "refusal": 1,
            "wording": 17
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 6,
          "change": false,
          "date": "2026-10-03",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 41,
          "wobble": {
            "answer": 34,
            "format": 9,
            "refusal": 0,
            "wording": 16
          }
        }
      ],
      "endpoint": "anthropic/claude-sonnet-5",
      "observation_ended": "2026-10-03",
      "page": "https://aimodelrecord.com/m/anthropic/claude-sonnet-5.json"
    },
    {
      "days": [
        {
          "boundary_observed": 15,
          "boundary_refused": 4,
          "change": false,
          "date": "2026-10-03",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 36,
          "wobble": {
            "answer": 24,
            "format": 7,
            "refusal": 1,
            "wording": 32
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-04",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 32,
          "wobble": {
            "answer": 31,
            "format": 4,
            "refusal": 1,
            "wording": 32
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-05",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 37,
          "wobble": {
            "answer": 30,
            "format": 5,
            "refusal": 2,
            "wording": 26
          }
        },
        {
          "boundary_observed": 0,
          "boundary_refused": 0,
          "change": false,
          "date": "2026-10-06",
          "observed": 0,
          "planned": 100,
          "same_answer_twice": 0,
          "wobble": {
            "answer": 0,
            "format": 0,
            "refusal": 0,
            "wording": 0
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 4,
          "change": false,
          "date": "2026-10-07",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 41,
          "wobble": {
            "answer": 24,
            "format": 4,
            "refusal": 1,
            "wording": 30
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-08",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 36,
          "wobble": {
            "answer": 28,
            "format": 3,
            "refusal": 1,
            "wording": 32
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 5,
          "change": false,
          "date": "2026-10-09",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 35,
          "wobble": {
            "answer": 31,
            "format": 5,
            "refusal": 0,
            "wording": 29
          }
        },
        {
          "boundary_observed": 15,
          "boundary_refused": 4,
          "change": false,
          "date": "2026-10-10",
          "observed": 100,
          "planned": 100,
          "same_answer_twice": 37,
          "wobble": {
            "answer": 33,
            "format": 4,
            "refusal": 0,
            "wording": 26
          }
        }
      ],
      "endpoint": "anthropic/claude-sonnet-5-5",
      "observation_ended": null,
      "page": "https://aimodelrecord.com/m/anthropic/claude-sonnet-5-5.json"
    }
  ],
  "terms": "- **Observed**: questions answered on that day, out of those asked\n- **Same answer twice**: questions whose answers were identical when asked twice on the same day, out of the observed questions\n- **Refused**: boundary questions (where refusing is a reasonable choice) that the model refused, out of the observed boundary questions\n- **Wobble**: how the two answers to the same question differed, out of the observed questions. *Wording*: same answer, different words. *Answer*: a different answer. *Format*: a different shape (code fences, lists, length). *Refusal*: it refused once and answered once\n- **Change**: the day a change from the previous observed day was detected. See the model's page for what changed\n\nWobble alone is not a change. Newer models reject any temperature other than their default, so they wobble more every day."
}