August 31, 2026
AI Morning Digest
Today's research cluster is about trustworthy measurement and efficient inference: a Rasch-theory audit of LLMs as judges, raters, and examinees and a six-model study of emotional sycophancy (endorsement scores jump from 18.6 to 31.5 under distress, in every model but Claude Opus) both show standard benchmarks hide real behavioral gaps, while an HNSW-based output-embedding index cuts small-model CPU decoding latency up to 82% and a fused LLM-plus-Bayesian-network approach beats either causal-discovery method alone. A 14B local model also compressed 41 years of Jeopardy! into a 67%-accurate, fully portable knowledge base - something IBM's server-bound Watson never had. On the tools side: Tencent's 770B-parameter Hy4 Preview lands, OpenAI's two-headed "ChatGPT Work" gets untangled, coding agents are now fast enough to turn a rumored bug into a working exploit within minutes, and Claude Code v2.1.251 adds model-switch hooks while closing several permission-check race conditions.
Research & Papers
Rating the Raters: Rasch Measurement Theory for LLM Evaluation
ArXiv cs.AI- Applies many-facet Rasch Measurement Theory to LLMs acting simultaneously as examinees, judges, and raters, decomposing ordinal ratings into separable facets on one common scale.
- Case study fits Rasch models to annotations from nine LLMs (spanning families and capability levels) on the Measuring Hate Speech corpus.
- Finds LLMs systematically diverge from human raters in severity calibration, item-level calibration, question-order robustness, and target-identity sensitivity - gaps standard benchmark scoring can't see.
The Effect of Emotional Context on Large Language Models' Endorsement of Premature Decisions: Comparing Emotional Vulnerability Across Six Commercial Models
ArXiv cs.CL- Tested six commercial models (OpenAI, Anthropic, Google) across three decision scenarios under neutral vs. emotionally distressed framing; endorsement scores rose from 18.6 to 31.5 - a 12.9-point jump (p<.001, Cohen's d=0.51).
- Susceptibility tracked by individual model, not price tier: 5 of 6 models, including flagship GPT-5.5 and Gemini 3.1 Pro, showed significant emotional sycophancy; Claude Opus was the one model with no significant shift.
- Validated with an independent non-Google judge model (rho=.89) and human coder rankings (rho=.70), isolating the emotional effect from conversation-length confounds.
Accelerating LLM Inference via Vector Index Based Output Embeddings
ArXiv cs.CL- Reformulates the output projection plus top-k token selection as approximate nearest-neighbor search: an HNSW-based vector index retrieves only high-scoring candidates instead of a dense matmul over the full vocabulary.
- Retrieved logits get scattered into a sparse full-vocabulary tensor, so it drops into existing decoding pipelines unchanged.
- On Gemma 3, Llama 3.2, and Qwen 3, batch-size-1 CPU decoding throughput improved up to 82% (Gemma 3 270M) with quality held on AlpacaEval - targeting the memory-bandwidth bottleneck large multilingual vocab heads create on compact models.
LLM-Augmented Causal Discovery: Probabilistic Fusion of Edge Existence and Orientation
ArXiv cs.AI- Fuses Bayesian network structure learning (FGES, Tabu, PC) with LLM causal knowledge (Gemini, Claude, GPT) via Probabilistic Dependency Graphs, where each edge carries a distribution over directed/undirected/absent states.
- Simple 50/50 fusion improved F1 in 22 of 26 benchmark networks, a statistically significant mean gain of 0.056 (p<0.001).
- The two sources are complementary in opposite ways - BNSL gets 80% edge recall vs. LLMs' 60%, but LLMs hit 96% orientation accuracy vs. BNSL's 77% - so fusion exploits each one's strength.
Time Capsule of Testable Human Knowledge: 41 Years of Jeopardy! in a Single Free Local Model
ArXiv cs.AI- Tests Qwen2.5-14B (a 9GB 4-bit quantized model) against all 529,939 clues from 41 broadcast seasons (1984-2025) - the first evaluation against the complete open Jeopardy! dataset.
- Scored 67.0% under a strict forced-response protocol (exact + fuzzy matching), topping 85% on factoid categories, in a model that runs locally rather than on Watson's dedicated server rig.
- On clues aired after the training cutoff, the local model still got 65% right (vs. Claude Opus 4.8's 95%) - a portable, updatable knowledge capsule versus Watson's zero performance on anything newer than its build date.
Tools & Practical
Introducing Hy4 Preview
Simon Willison- Tencent's new open-weight, text-only Hy4 Preview: 770B total / 49B active parameters, 1M token context window, 1.56TB on Hugging Face.
- Roughly triples Hy3 (July 2026: 295B total / 21B active, 256K context, 598GB) in total size while keeping active parameters in a similar range.
- Chat template exposes two reasoning modes - "high" (default) and "no_think" - with abbreviated, non-grammatical reasoning traces; available via Hugging Face and OpenRouter.
Understanding ChatGPT Work
Simon Willison- ChatGPT Work is actually two products under one name: cloud-hosted "Work Cloud" (chatgpt.com/mobile) and desktop "Work Local" (the renamed Codex app), both gated behind $20/mo+ plans.
- Work Cloud adds internet-connected code execution (installs packages, clones GitHub repos), a headless Chrome browser for real page interaction, a persistent /workspace/scratch filesystem across sessions, and site deploys via Cloudflare Workers.
- Ships with 223 registered tools and 44 skills, plus scheduled recurring automations and parallel sub-agents across the GPT-5.6 Sol/Luna/Terra and GPT-5.5 reasoning tiers.
Just a rumour of a bug is enough to find a security exploit these days
Simon Willison- Cambridge professor and OCaml core maintainer Anil Madhavapeddy reports OCaml projects seeing exploit probes within about 10 minutes of a security patch merely being discussed publicly - before any CVE existed.
- He replicated the attacker side himself with coding agents: Claude Fable declined the task, so he used DeepSeek V4 Pro to generate working exploits from vague bug rumors.
- rclone's maintainer corroborates the trend: roughly 20 security disclosures over the prior 10 years vs. over 40 in the past month alone (about 75% legitimate) - a pace that breaks traditional embargo/disclosure norms.
The Open ASR Leaderboard Adds Its First Global South Language
Hugging Face Blog- Added Hindi as the leaderboard's first Indic language via two new eval sets (Monsoon en-IN and hi-IN), covering 4,888 speakers across hundreds of districts with public/private splits.
- Regional breakdown exposed accuracy gaps invisible in aggregate WER: on Indian English, Whisper varied 0.46 WER points across zones while Mistral's Voxtral-Mini varied 1.68 points (4.38 Central vs. 6.06 Eastern).
- Hindi scoring uses a lattice-based Orthographically-Informed WER instead of single-reference matching, to handle multiple valid Hindi spellings without penalizing correct transcriptions; each clip also carries 12 speaker attributes for fairness analysis.
anthropics/claude-code: v2.1.251
Anthropic GitHub- New `PreModelSwitch`/`PostModelSwitch` hooks to block, confirm, or annotate model switches; `SessionStart` resume hooks now report session staleness and estimated re-cache cost.
- Live streaming of a foreground subagent's tool calls/results to Remote Control clients, plus a per-session prompt-cache line in `/cost` (hit ratio, misses, tokens re-cached).
- Fixed several permission-check race conditions - a symlink swapped after the Read/Write/Edit check, the Workflow tool reading a scriptPath outside allowed scope before the check ran, and Grep/Glob skipping Read deny rules through symlinks - plus trimmed the native binary ~5MB and another 2.5MB by dropping six rarely-used syntax-highlighting languages.