Each card names its own baseline: the submission count is against the 1 October export (365); the medians set the fortnight of 2 to 14 October against the whole of September, with 1 October beside them; the verified experts are set against the self-declared ones; the expert share sets the window since the check (19 August) against the window before it, counting submissions that declared a level; the repeat count sets the rows since the 2 September fix against the fortnight before it; the kept share is over every quiz submission, identical repeats aside; the time card sets the fastest third of the rows that recorded a time against the slowest third.
September closed at 49%. After 1 October's 23%, the next thirteen days came back to 54%
The level chooser went live on 9 March 2026 and the expert check on 19 August, so the 98 submissions from before 9 March answered no questions at all. Six later rows carry no quiz either: five took a sliders shortcut that has since been removed, one came through a retired knowledge-check path, and all six sit at or near the page defaults. They are left out of the timeline and the group strips; the monthly figures and the factor panels count every row. Nothing without a level has arrived since 2 September, when the page started refusing such rows: that is 222 rows now.
Every submission since 1 August, by quiz level
287 submissions arrived in these ten weeks, and the 285 with a quiz level are drawn, so they get the full width; the charts below cover every month. The tall column of dots two weeks from the right-hand edge is 1 October. Hollow dots are readers who took the expert quiz without a pass: those who simply called themselves experts before any check existed, and three who have since fallen short of the check and continued anyway. Filled dark ones are verified: they passed a 30-term vocabulary check before the expert quiz would open. The line is a running median, the median of each submission and the fourteen before it, with identical repeat rows left out, and it is dashed across gaps of more than two days between submissions. Hover or tap a dot for its p10–p90 range; with the chart focused, the arrow keys step through the dots. The two rows in this window that arrived with no quiz behind them are left out, as all six such rows are from the strips below.
Monthly median with interquartile band
Line: median of that month's submissions, every level and every row together; band: 25th–75th percentile. September is complete. October's point is hollow and joined by a dashed line because the month runs only to 14 October, and 37 of its 95 rows are one day, 1 October. The next chart splits the same line by quiz level.
September's median was 49% on 130 submissions. October so far is 39% on 95, and without its first day 54%.The first three months of the site, before any quiz existed, produced a median of 15%. The quiz arrived in March, the monthly median reached a peak of 69% in August, and September, still the busiest month in the series, came back down to 49%. October's 39% is two different halves averaged into one point: 1 October, 37 rows at 23%, and 2 to 14 October, 58 rows at 54%. The same split holds among visitors who set their own number, repeats excluded: 50% in August, 29% in September, 18% on 1 October and 40% in the fortnight after it. Why the visitors changed is not something a table of submissions can say. The author's impression of how the site's reception turned around August is in a note; it is an impression, not a measurement.
The burst did not come back. The thirteen days after it ran at September's pace.58 rows arrived between 2 and 14 October, 4.5 a day, with nine on the busiest days, 6 and 8 October; September averaged 4.3 a day. 1 October, at 37 rows, is still nearly twice the next busiest day in the series. Its low tail did not vanish entirely: two later days with five or more rows also have medians under 25%, 2 October at 20% on seven rows, three of them from one browser, and 8 October at 22% on nine. Those are the only two such days besides 1 October since the chooser went live. Where the 1 October crowd came from is still not in the table.
Monthly median by quiz level
Every row, then each quiz level, on one axis from March 2026, when the level chooser went live; the chart above has the months before it. Every row counts all submissions in the month, including kept ones, so it is the line above. Moved a slider redraws all four lines on the quiz rows where at least one midpoint left the quiz's proposal, identical repeats excluded, which takes the page's own arithmetic out; there the first line is every quiz-taker who moved. A point from fewer than five rows is small and hollow, and the segment into it is dashed: before August each level had between none and four submissions a month, so those stretches are anecdotes, not trends. October is shaded because it runs only to 14 October. Hover or tap a month for every value and its n; the monthly summary further down gives the same medians as a table.
On every row, the expert line has fallen three months running, 59%, 32%, 23%. The beginner line fell and came back, 72%, 48%, 56%.Medium went 69%, 63%, 40% over the same three months. In August the three levels sat within thirteen points of each other; in October beginner and expert are 34 points apart. Most of that gap is the proposal and how often each level keeps it: across the whole export the beginner quiz is kept on 42% of its rows, medium on 39% and expert on 20%, and a kept row is the formula's number. Switch to the rows where the visitor moved a slider and the lines close up: in September they sat at 31%, 27% and 28%, the convergence the last report described.
In October the movers come apart again on one side: medium at 20%, expert at 22%, beginner at 54%.Before reading anything into the beginner point: it is 16 rows, three of them one browser on 2 October that took the same quiz three times in six and a half minutes, and "moved" includes small moves. Of the twelve beginner movers after 1 October, two ended within two points of the proposal and two dragged it up to 100%. Across the whole export the movers' medians are still 36%, 33% and 31% on 54, 75 and 79 rows, and before October the beginner movers stood at 32%. The medium and expert movers are the ones that have kept falling, from 61% and 47% in August, though 1 October supplies 25 of their 57 October rows. The beginner line is something to watch at the next export, not a finding.
Every submission's p(doom), by quiz level
The stats page's histogram, the same twenty bands of five points over all 424 submissions, with each bar split by the quiz the visitor took. From the baseline up: beginner, medium, expert, and in grey the rows with no quiz, the 98 from before the level chooser existed and the six later ones without a level. Hover or tap a bar for its breakdown and for how many of its rows kept the quiz's proposal; with the chart focused, the arrow keys step through the bars.
The two spikes in the upper half are the page's proposal. The tallest bar, 0 to 5%, is not.The 70–75% band holds 36 rows, 17 of them beginners, and 20 of the 36 kept the proposal: it is where the beginner quiz puts a visitor who ticks most of its boxes. The 95–100% band holds 28, 17 of them medium-quiz takers, and 15 of the 28 kept it. The 0–5% band is the tallest at 54 rows, and none of them kept a proposal: 22 are from before any quiz existed and the other 32 are quiz-takers who took the page's number down to near zero, 15 of them on the medium quiz. The 10–15% band is mostly the site's first months, 25 of its 36 rows. A single-colour histogram, as on the stats page, shows three peaks; split by level, each peak is a different population for a different reason.
Who says what: distribution by quiz level
One mark per submission; the tall tick is the group median. An arrow pointing left is a visitor who moved the page's proposed number down, an arrow pointing right one who moved it up, and a round dot either kept the proposal or, in the bottom row, never had one. The expert rows are split three ways: the 22 who declared themselves experts before the check existed, the 74 who have passed it since, and the three who fell short of it and continued to the expert quiz anyway. Three marks do not make a median, so that row has no tick.
Monthly summary
| Month | n | Median | P25 | P75 | Beg. | Med. | Exp. |
|---|
2026-10 runs to 14 October. The last three columns are the median of every row at that level, kept rows included, as on the chart's first view; a dash is a month with no rows at that level, and a grey figure is one from fewer than five.
The belief chain: where the groups diverge
Group medians for each link of the chain, and the resulting p(doom). The beginner and medium lines sit high on every link because a kept row has all three factors at the cube root of one proposal. The expert line bends at the last link: a fifth of expert rows are kept, and the readers who pass the check discount that link. The dashed line is quiz-takers who moved a slider, and it discounts both conditionals, as the pre-chooser cohort does. Final p(doom) multiplies each person's own chain, so it sits below the medians shown.
The widest-spread factor tracks the total most. That is arithmetic, not behaviour
Each panel: one factor's midpoint against the submission's own final p(doom). Read this as a sensitivity check, not a finding about people. Final p(doom) is the product of these three numbers, so each factor is being correlated with something it helped compute. What separates the panels is how much each factor varies across this set of submissions, not where it sits in the chain; reordering the chain leaves every correlation unchanged.
A third of quiz-takers registered the number the page proposed, and they registered it quickly
After a quiz the page does not hand the visitor neutral sliders. It proposes a p(doom) from the answers, an opinion question picks a band and the share of boxes ticked moves the number up within it, and sets all three factors to the cube root of the proposal. It is tempting to read the quiz as a survey taken alongside the sliders. It is not: it is an input to them, and a visitor who answers it and presses submit has registered the page's arithmetic rather than their own. Every quiz submission is therefore split from here on into kept, where no midpoint was moved from the proposal, and moved. This export adds two views of that split: where each kind of row lands on the scale, and how long the visitor looked at the proposal before submitting.
What the page proposed against what was registered
One dot per quiz submission. Horizontal: the p(doom) the quiz set the sliders to, recomputed from the stored answers with the page's own rules. Vertical: what was registered. A round dot on the diagonal never had a midpoint moved; off the diagonal the mark is an arrow pointing the way the number went, the same encoding the strips use further down. Identical repeat rows are not drawn.
Kept and moved, by level
| Level | n | Kept | Median, kept | Median, moved |
|---|
The kept share has fallen since the page began saying that the quiz proposes a starting point. One sentence cannot claim the fall.Since 21 September the page says, above the sliders, "A short quiz proposes a starting point from what you already know", and 119 quiz rows have arrived under it. 34 of them kept the proposal, 29%, against 37 of the 103 rows in the nineteen days before it, 36%. The window since breaks into three pieces that differ from one another more than any of them differs from the window before: 12 of 24 kept in the last ten days of September, 8 of 37 on 1 October, and 14 of 58, 24%, from 2 to 14 October. By month the kept share was 39% in August, 38% in September and 23% in October. Most of the October fall is the expert quiz, where 1 of 15 rows since 2 October kept the proposal, against 5 of 17 beginner rows and 8 of 26 medium. That fits a sentence that works, and it fits an audience that changed, and neither the timing nor the size tells them apart.
Where the registered numbers land
Every quiz submission, repeats excluded, counted in ten bands of ten points. Top row: the 106 that kept the proposal. Bottom row: the 208 that moved a midpoint. The two rows share the horizontal scale and each bar is labelled with its count.
The commonest place for a mover to land is under 10%.44 of the 208 movers registered less than 10%, against 2 of the 106 kept rows, and no other band of ten points holds as many movers. Their proposals had a median of 64%: these are visitors who were handed a number near two in three and took it to near zero. The kept rows sit where the formula puts them, 38 in the seventies and 24 at 90% or above. The share of movers under 10% has been steady outside one day: 8 of 54 before September, 15 of 81 in September, 9 of 44 from 2 to 14 October, against 12 of 29 on 1 October. By level it is 7 of 54 beginner movers, 19 of 75 medium and 18 of 79 expert. So the movers' median is a median of two kinds of visitor, those who adjust the proposal and those who throw it out, and 40 of the 208 moved it up rather than down.
Time on the sliders, and who keeps the proposal
Since 12 September the page records the moment it applies the quiz's proposal, and 164 quiz submissions carry it. The time here runs from that moment to the submit press: time spent on the sliders, the uncertainty bands and the rest of the page. Each bar is the share of rows in that time band that kept the proposal, labelled with the count. Identical repeat rows are left out.
Of the visitors who submitted within a minute, 63% kept the page's number. Of those who took more than three minutes, 14% did.25 of the 40 rows submitted within a minute kept the proposal; 3 of the 21 that took more than three minutes. The median time is 67 seconds for a kept row and 102 for a moved one. It is not one level's habit: split each level at its own median time and the faster half kept the proposal on 12 of 21 beginner rows, 15 of 32 medium and 11 of 28 expert, the slower half on 4 of 22, 9 of 32 and 4 of 29. No row came in under 20 seconds, and 52 of the 55 kept rows that stored the proposed band changed it, so a quick kept row is not a visitor pressing submit blind: it is one who looked at the page's number briefly and let it stand. Among the movers, time does not predict where they land: the rank correlation with the number registered is +0.02, and with how far the number moved +0.07, so the slow ones do not drag further. Time cannot say which way the arrow runs: a quick submitter may be someone already satisfied by the proposal, or someone in a hurry who would have kept any number.
Kept means the three midpoints are exactly where the quiz put them, and nothing more. The uncertainty band is a separate control: of the 55 kept rows that stored the proposed band, 52 registered a different one. Identical repeat rows from one browser are left out of every kept and moved count, which is why the totals here are 314 and not 320. Since 12 September the page stores the proposal with each submission, and 164 rows carry it. For the quiz rows before that, the proposal is recomputed from the stored answers with the page's own tables, and the build checks itself two ways: every kept row sits at exactly the per-factor value the page would have set, and on the 164 rows that have both, the stored and the recomputed proposal agree to within a tenth of a point. Seven rows have three equal midpoints at a value no proposal produces, a visitor's own doing, and are counted as moved. One moved row was submitted 53 minutes after the proposal; a tab left open is the likeliest reading, and it sits in the last band.
Eighty-six took the check. Seventy-four cleared it. Three of the twelve who fell short went on to the expert quiz anyway.
Clicking Expert quiz opens the 30-term check, a score of 26 or more clears it, and a lower score offers the recommended quiz alongside a "continue to expert anyway" button. The verdict, the score and the exact terms ticked are stored with the submission. 86 people have now finished it, 17 more than at the last report, and it admits most of them, 74 of 86. Of the twelve it has turned away, nine took the quiz it recommended and three pressed the button.
What became of everyone who clicked Expert quiz
A score is recorded exactly when the check ran, which is exactly when the expert card was clicked. What the table cannot hold is anyone who took the check, saw the verdict and closed the tab.
Of the 17 new finishers, 14 cleared the check, two took the medium quiz it recommended, and one continued to the expert quiz anyway.The two who took medium scored 22 and 25 and both kept the medium quiz's proposal, at 67% and 95%. The one who continued scored 25 on 8 October and registered 10%, joining the two from the last report, who scored 24 and 25 and registered 27%, the expert proposal kept, and 0.4%. All nine who took a recommended quiz scored 19 to 25; eight were offered medium and the one who scored 19 was offered beginner. Their submissions span 33% to 99%, so falling short of a vocabulary bar says nothing about the number someone then registers. Everyone else scored 26 to 30, at a median of 29. The expert share of submissions that declared a level is 30% since the check, 77 of 254, against 33% before it, 22 of 66; without 1 October it is 27%, and from 2 to 14 October 26%, 15 of 58. Medium is still the largest level, 42% of everything since 19 August. Whether the check deters people from clicking the expert card at all is not something completed submissions can show: a visitor who took the check and closed the tab leaves no row.
The level chooser, before and since the check
Completed submissions per path in each window. In the first window a visitor could still skip the quizzes and go straight to the sliders, or take a standalone knowledge check; both were retired on 19 August, when the check moved onto the expert card, and since then every path on the page runs through a quiz. Two rows from 21 August arrived with no level anyway. They are unsigned, which is what a tab loaded before the 19 August deploy would send. They are left out of the right-hand panel, and since 2 September the database refuses any row without a level.
Term by term: what the 86 finishers ticked
Decoy-to-term pairings are an editorial reading of the term list; the AI-related flags come from the page. Scores here are recomputed from the ticked terms against the list as it stood on the taker's day; Mary Shelley was replaced by Embedding on 2 September, so the four earlier finishers were shown her and the eighty-two later ones were shown Embedding; which list a row saw is resolved from its page version, and the recomputed score agrees with the score the page stored on every one of the 86. The one submission through the retired standalone path (July 2026) is shown separately in the tooltip.
Verified experts against self-declared ones
Left: the share of options ticked on each expert-quiz question, for all 99 expert submissions. Right: how many of the 12 named organisations, commentators and creators each recognised, against the p(doom) they submitted. Dark dots are verified experts, grey ones self-declared, and the two groups sit to either side of each column; the three hollow dots are the readers who fell short of the check and continued.
The 74 who passed the check submitted a median 32%. The 22 who called themselves experts before the check existed, 68%.Among visitors who set their own number the gap is smaller, 25% against 44%, on 60 and 17 rows. The pooled gap includes the kept rows on both sides, five self-declared experts at a median 77% and fourteen verified ones at 69%; leaving them out changes who is being compared as well as what, so the two gaps are two different comparisons rather than a decomposition into proposal and belief. The last report saw the verified readers' recognition of names in the field fall, to a median 3 of 12, and put it down to the readers verified since 20 September, who recognised 2. This export says whose it was: the 18 verified on 1 October recognised a median 2, and the 14 verified since recognise a median 4, the same as the 34 verified before 20 September. The verified median over all 74 stays at 3, against 4.5 for the self-declared. The other difference has held too: all 22 self-declared experts ticked every option on the self-improvement question, against 43 of the 74 who passed the check. Ticking everything is consistent with a list that is half-recognised, though a checklist nobody can fail cannot tell that from thoroughness. The two groups are also separated in time, since the check only exists from 19 August, and the site's overall median fell over the same period, so some part of this is the audience changing rather than the check selecting.
The expert quiz's funding question is an opinion, not a recognition test, so it is left out of both panels. It appears in full in the expert section below.
Every beginner answer, and the p(doom) that went with it
The beginner quiz asks four questions: which AI films the visitor has seen, which kinds of catastrophe they recognise, which AI capabilities they know about, and a gut reaction to an AI misbehaving. For each question, the left panel counts how many ticked each answer and the right panel splits the respondents into thirds by how many they ticked, with the p(doom) each third submitted. First, the same split across all three checklists at once.
Ticked most and fewest, across all three beginner checklists
Every beginner's ticks on the film, catastrophe and capability lists added together, out of 50 possible, and the cohort cut into thirds on that total. Left: one mark per submission, the tall tick is the group median; an arrow points the way the visitor moved the quiz's proposed number, a dot kept it. Right: what each third did with that proposed number.
Pooled, the thirds sit at 57%, 61% and 51%. Among the 54 who moved a slider, the third who ticked most registered 40%, the middle 40% and the third who ticked fewest 31%.The pooled figures are mostly the page's proposal, which rises with the number of boxes ticked; 39 of 93 beginners kept it. At the last report the movers' thirds were level, at 32%, 34% and 29%. All three have risen since, the upper two most, which is the October beginner movers seen from another side; on cohorts of 14 to 21 that is a direction, not a slope. The catastrophe list is still the one place a gradient runs the other way, and it has held through two exports. The third who ticked most catastrophes and set their own number registered a median 25% (n = 31), the middle third 47% (n = 12) and the third who ticked fewest 52% (n = 11): the visitors who recognise most of the ways the world could end set the lowest number. Ticks on a checklist are a count of recognised names, not a measure of knowledge.
Every medium answer: among the movers, the vulnerability checklist now tracks nothing
The medium quiz asks six questions: three about what the visitor knows or has lived through, three about what they believe. It is the largest of the three cohorts, and its kept rows carry the highest proposed numbers on the site. The correlation figure is given three ways: all 128 rows, the 122 without the six identical repeats, and the 75 of those where the visitor moved a slider. Everywhere else the repeats are drawn but left out of the kept and moved counts.
How strongly each answer tracks the p(doom) submitted
Rank correlation per question. The filled mark uses all 128 rows; the hollow mark drops the six identical repeats; the square uses only the 75 rows where the visitor moved a slider.
On all rows the vulnerability checklist tracks p(doom) at ρ = +0.18. On the 75 rows where the visitor moved a slider, it is −0.02.That is the page's own formula being measured, not a fact about people: the share of vulnerabilities recognised is one of the inputs that moves the proposed number up, and 47 of the 122 medium rows without repeats kept the proposal. The pooled figure was +0.66 in the August report, +0.29 in September and +0.25 at the last one, and on the movers it has gone from +0.12 to nothing. What survives on the moved rows is the three opinion questions, governance at +0.41, competition at +0.32 and speed at +0.30, which pick the band and are near restatements of the sliders. System prompts sits at −0.02, and "seen AI go off the rails" falls from +0.23 pooled to +0.07 among movers. On this quiz, what a visitor recognises has no measurable association with the number they set for themselves.
The three opinion questions are not findings: a belief about whether governance can keep AI safe correlating with an estimate of catastrophe is close to tautological.
Ticked most and fewest on the vulnerability checklist
The one multi-select question on this quiz, so the overall split is the same as the per-question one. Thirds by how many of the 16 entries were recognised. Left: one mark per submission, the tall tick is the group median; an arrow points the way the visitor moved the quiz's proposed number, a dot kept it. Right: what each third did with that proposed number.
Pooled: 67%, 62% and 63%. Among those who moved a slider, repeats excluded: the third who ticked most registered 35%, the middle 22%, the third who ticked fewest 49%.At the last report the movers' thirds read 36%, 31% and 27%, a shallow slope in the pooled direction. With nineteen more movers the slope has gone and the third who ticked fewest is now the highest, on cohorts of 22 to 28. The ordering has changed at every export so far, which is what a checklist with no signal looks like when it is cut into thirds. Most of the pooled figure is still the proposal: the top third without repeats has 41 rows, 19 that kept the page's number at a median 94% and 22 that moved it, to 34%. The entries fewest people recognise are side-channel attacks (53 of 128), adversarial attacks (73) and model stealing (75).
Every expert answer: one list of names still sorts the experts, and the total does not
The expert quiz asks eight questions: three about mechanisms, four about names in the field, and one opinion on funding. The charts in this section colour marks by how many boxes were ticked and do not distinguish verified from self-declared experts; the tooltip on each mark says which it was, and the check section above draws the groups apart. 15 of its 99 rows are new since the last report, and only one of them kept the proposal.
Ticked most and fewest, across all seven expert checklists
Every expert's ticks on the seven multi-select questions added together, out of 24 possible, and the cohort cut into thirds on that total. Left: one mark per submission, the tall tick is the group median; an arrow points the way the visitor moved the quiz's proposed number, a dot kept it. Right: what each third did with that proposed number.
The third who ticked most submitted a median 45%, the middle third 28%, and the third who ticked fewest 31%.That is the shape of the last report, 44%, 25% and 32%, and among those who set their own number the thirds sit at 42%, 20% and 31%, on 32, 30 and 17 rows. A total across seven lists does not sort this cohort in one direction. One of the seven does, and it held through the fifteen new rows. On the campaign-organisations question, the 45 who recognised two or three of PauseAI, ControlAI and Microcommit submitted a median 54%, the 33 who recognised one 25%, and the 21 who recognised none 23%; among those who set their own number it is 42%, 23% and 14%. Knowing the organisations that campaign about AI risk and expecting a catastrophe go together, which is close to saying one thing twice. The mechanism questions are still ticked in full by most takers. 20 of 99 expert rows kept the proposal.
Do the visitors who know least claim most? Partly in what they tick, not in how sure they are
The Dunning–Kruger effect is the finding that people with the least skill in an area overestimate it the most, because the skill they lack is also the skill needed to judge it. Testing it needs three things from the same people: a measure of their skill that cannot be gamed, their own estimate of that skill, and ideally a measure of how certain they are. The site has partial versions of each. Clicking the expert card is a claim of expertise, and the 30-term check, with its fifteen decoys, scores the claim. The medium quiz asks visitors to rate their own skill with system prompts, which is the closest thing on the site to its definition of an expert, hands-on experience, and separately asks which vulnerabilities they know. And every visitor sets an uncertainty band on each slider, which is how sure they say they are. What the site cannot do is the classic test. Nobody is asked to estimate their own score, and only people who claimed expertise are scored at all, so a visitor who underrates themselves, the other half of the effect, never takes the check. This section does not draw the familiar Dunning–Kruger chart, self-estimate against actual score by quartile, because that chart takes the same shape from random numbers (Nuhfer and colleagues, 2016; Gignac and Zajenkowski, 2020). It compares within people instead, on three questions.
Claiming to know terms that are not AI terms, by how many real ones were spotted
The check shows fifteen real AI terms and fifteen decoys such as Agent Orange and Reinforced concrete. Ticking a decoy is overclaiming: saying you recognise something you cannot. The 86 who finished the check are cut into three groups by how many of the real terms they spotted, and each bar is the share of the group who ticked at least one decoy. Spotting a real term and ticking a decoy are counted separately, so one does not mechanically fix the other.
Check score against how sure the visitor said they were
One dot per check-taker. Horizontal: the score on the check, out of 30; the bar to clear it is 26. Vertical: the uncertainty band the visitor registered, the mean of the three sliders' ± spread, so lower is more certain. Colour is the quiz that followed, as in the legend: the expert quiz for those who cleared the check or went on anyway, otherwise the quiz the check recommended. The short horizontal lines are the medians of three score groups. The check score is not an input to the band the page proposes, so a slope here would be the visitors' own.
Self-rated skill with system prompts against what was recognised
Medium-quiz takers, identical repeats excluded. The self-rating is the visitor's own answer to "How familiar are you with system prompts?". The next two columns are from the vulnerability checklist: how many of its five AI-specific entries were ticked (prompt injection, sleeper agents, data poisoning, adversarial attacks, model stealing; which entries count as AI-specific is an editorial reading of the list), and how many ticked prompt injection, the attack that goes through a system prompt. The last three columns are the band, as a median ± in points: what the page proposed and the visitor's change to it, both on the 64 rows that stored the proposal, and what was registered, on every row. A negative change means the visitor narrowed the band.
| Self-rating | n | AI-specific, of 5 | Prompt injection | Band proposed | Band registered | Change |
|---|
The visitors who call themselves experts at system prompts recognise no more AI-specific vulnerabilities than regular users, and 3 of 18 did not tick prompt injection.Self-rated experts tick a mean 3.3 of the five AI-specific entries, against 3.7 for those who use system prompts regularly; 15 of the 18 ticked prompt injection, against 37 of 38 regular users. Across the four ratings the self-rating goes with ticking more of the general security entries (ρ = +0.34) more than with the AI-specific ones (+0.14), so part of what a higher self-rating predicts is ticking more boxes. That is the Dunning–Kruger shape at the top of the scale, but it rests on three people and on a checklist without decoys, where an unticked box may be oversight as easily as ignorance. The self-rated experts also register the narrowest bands, ±28 against ±60 for those who do not know what a system prompt is, and that part is not theirs: the page proposes a narrower band to a higher self-rating, and on the rows that stored the proposal the visitors' own change is unrelated to the rating (ρ = +0.07). The certainty belongs to the formula.
The p(doom) of the visitors who ticked a decoy
Expert-quiz takers who took the check, identical repeats excluded, split by whether they ticked any decoy. One mark per submission; a hollow mark kept the quiz's proposal, and the tall tick is the group median. The visitors who fell short and took the recommended quiz are left out, because the medium quiz proposes much higher numbers than the expert quiz and would make the comparison about the quiz.
Verdict: one Dunning–Kruger-like pattern, in what people claim to recognise, and none in how certain they are.Those who spot fewer real terms are about twice as likely to claim a fake one, which is the overclaiming half of the effect, and it holds on a measure built to catch it. Certainty does not follow. Among people who claimed expertise, those with the weakest vocabulary set wider bands, not narrower; among those who rate their own system-prompt skill highly, the narrow bands are the page's. The popular reading of the effect, that the people who know least are the most sure, is not what this data shows. Two limits apply to all of it. The check measures vocabulary, which the site's own definition calls an approximation of hands-on expertise, since vocabulary can be read up on. And nobody here estimated their own score, which is the measurement the effect is defined on; the directions section proposes adding it.
Fifteen things this export supports
- The busiest day was a one-off, and the fortnight after it looks like September. 58 submissions from 2 to 14 October, 4.5 a day against September's 4.3, at a median of 54% against September's 49%. 1 October, now 37 rows at 23% with one late arrival counted, is still nearly twice any other day. Two later days with five or more rows had medians under 25%, so the low tail is still there, but at the rate of a few rows a week rather than 33 in an evening.
- Visitors who submit quickly keep the page's number; visitors who take their time move it. 63% of the rows submitted within a minute of the proposal kept it, against 14% of those that took more than three minutes, on 40 and 21 rows of the 164 that record the time. The pattern holds inside each level. Among movers, time does not predict how far they move. This is the first measurement the site has of how a proposal is received, and it cannot say whether quick keepers agree or are in a hurry.
- A mover's commonest landing place is under 10%. 44 of 208 movers registered less than 10%, more than in any other band of ten points, from proposals with a median of 64%. Only 2 of 106 kept rows are that low. The share has sat between 15% and 20% of movers in every period except 1 October, when it was 41%. The movers' median sits between two kinds of visitor: those who adjust the proposal and those who reject it.
- Split by level, the pooled monthly median hides a falling expert line. On every row the expert median went 59%, 32%, 23% from August to October; beginner went 72%, 48%, 56%; medium 69%, 63%, 40%. Among movers the levels converged in September at 31%, 27% and 28%, and in October medium and expert went on down to 20% and 22% while 16 beginner movers registered 54%. The beginner figure is small and partly near-keepers; the falling expert line is the steadier of the two.
- The page proposes a number, and 34% of quiz-takers register it with the midpoints unchanged. 106 of 314 quiz submissions, identical repeats excluded. Their medians by level, 72%, 81% and 70%, are the formula's output. Of the 208 visitors who did move a midpoint, 168 moved down; the median move across all of them is 29 points down, and among those who moved down it is 37. The kept share has fallen to 24% from 2 to 14 October, mostly on the expert quiz, where 1 of 15 kept. What "kept" cannot say is that nothing was touched: 52 of the 55 kept rows that stored the proposed band changed it.
- Once the proposal is taken out, the three quiz levels land close together over the whole export. Pooled medians: beginner 55%, medium 63%, expert 38%. Among visitors who set their own numbers, repeats excluded: 36%, 33% and 31%, on 54, 75 and 79 rows. The spread the pooled figures show is the three quizzes proposing different numbers and being kept at different rates, 42%, 39% and 20%.
- The readers who passed the vocabulary check expect powerful AI, and discount the step from danger to catastrophe more than anyone else. Their chain runs 90%, 80%, 50% among the 60 who set their own numbers. No other quiz group goes that low on the last step: beginners who set their own leave it at 73%, medium takers at 75%, self-declared experts at 93%. Only the visitors from before any quiz existed sit there too, at 50%. Two things could produce it, and this export cannot separate them: a considered view, or an expert quiz that asks about mechanisms and never once mentions an outcome.
- Passing the vocabulary check goes with a lower number than claiming expertise did. Pooled, 32% (n = 74) against 68% (n = 22). Among visitors who set their own number the gap is 25% against 44%. That is a different comparison, not a decomposition: it leaves out the kept rows on both sides, and so changes who is compared as well as what. Neither figure isolates the proposal's effect, and the two groups are months apart.
- The drop in names recognised by verified experts was the 1 October crowd. The 18 verified that day recognised a median 2 of the 12 names in the field; the 14 verified since recognise a median 4, the same as the 34 verified before 20 September. Within the expert quiz it is the campaign organisations that go with the number: a median 54% for those who recognise two or three, 23% for those who recognise none.
- Visitors who spot fewer real AI terms are twice as likely to claim a fake one; they are not more certain. Of the check-takers who spotted 12 or fewer of the 15 real terms, 8 of 24 ticked a decoy, against 5 of 32 who spotted 13 or 14 and 5 of 30 who spotted all 15. That is the overclaiming half of the Dunning–Kruger effect. The certainty half is absent: among expert-quiz takers the check score is unrelated to the band they set (ρ = −0.07), and the medium quiz's self-rated system-prompt experts owe their narrow bands to the page's formula. The 13 expert-quiz takers who ticked a decoy registered a median 56% against 25%, a gap a permutation test puts at p ≈ 0.07 on so few people. Nobody is asked to estimate their own score, so the classic test cannot be run.
- The expert gate is a recommendation, and three people have now declined it. 86 finished the check: 74 scored 26 or better and went through, nine scored below and took the quiz it recommended, and three scored 24 or 25 and continued to the expert quiz anyway, registering 27%, 0.4% and 10%. The expert share since the check is 30% against 33% before it, and 26% in the fortnight after 1 October.
- How many boxes a visitor ticks does not sort the number they set, at any level, and the orderings keep changing. Among movers, from the third that ticked most to the third that ticked fewest: beginner 40%, 40%, 31%; medium 35%, 22%, 49%; expert 42%, 20%, 31%. The medium ordering is different at every export. Two single lists hold their direction across exports: the beginner catastrophe list, inversely (25%, 47%, 52% among movers), and the expert campaign-organisations list.
- What a medium-quiz taker recognises has no measurable association with the number they choose. The vulnerability checklist is ρ = +0.18 across all rows and −0.02 among the 75 who set their own number, repeats excluded, down from +0.12 at the last report. Having seen AI go off the rails is +0.07 among movers. What still correlates is the three opinion questions, which is close to tautological: they ask about competition, governance and reaction speed, and they also choose the band the proposal starts in.
- P(powerful AI) is rated highest of the three links and the extremes are used. Its median is 88% against 81% and 80% for the two conditionals, yet 76 of 424 submissions (18%) put it at 50% or below. 18 submissions put the final number at exactly 0% and 14 at 99% or above; the p10–p90 band has a median width of 35 points. One of the zeros and three of the 99%-or-above rows are new since the last report. The band is not an interval around the midpoint: the page draws 4,000 samples per factor from a bell curve centred on the midpoint with a standard deviation of half the slider's ± spread, rejects draws outside 0 to 1, multiplies, and takes the 10th and 90th percentiles. Truncation near 0 or 1 skews that distribution, so the midpoint product can sit outside the band, and on 133 of the 424 submissions it does. Before 20 August 2026 the draws were uniform across each band; rows from before then keep the band they were submitted with.
- Impatient double-submits have stopped. Zero identical repeat rows among the 222 since the 2 September fix, against six in the fortnight before it. Eleven browsers did submit more than once since, and all eleven changed something in between; the newest took the beginner quiz three times in six and a half minutes on 2 October with the same answers and three different slider settings, which is a person using the site rather than a button being pressed twice.
What to change next: the page can now time a decision, and should count what happens inside it
Each report ends with the changes the data argues for, and marks what happened to the ones before it. Nothing that changes what the page collects has shipped since the 1 October report, so the four measured items are the same four, measured again on 59 more rows, and all four still hold. One item is new, and it comes from the time-on-sliders finding. The items that are about the site rather than the data, the stats page and the About page among them, are kept in the open-items note.
- MeasuredTell kept rows from moved rowsWherever submissions are drawn one by one, the strips and the proposed-start chart, a round dot is a row whose midpoints are where the quiz put them and an arrow is one that moved; the timeline and the factor panels do not make the distinction. The section on the proposed start, the level chart's second view, the bars beside each question, and the findings give the moved rows on their own. The recomputation checks out: all 107 kept rows, 106 without the one identical repeat, sit exactly where the page would have put them, and seven rows with three equal midpoints at a value no proposal produces are counted as moved.
- MeasuredStore the proposed starting point with the submissionThe page has sent a
calibrationobject with every quiz submission since 12 September, and 164 rows now carry it. On all 164, the stored proposal and the one the build script recomputes from the answers agree to within a tenth of a point. Itsapplied_attimestamp, stored for bookkeeping, turned out to be the source of this report's time-on-sliders finding. - MeasuredRefuse to register the same prediction twiceSix identical repeat rows in the fortnight before the fix; none in the 222 rows since. Eleven browsers submitted more than once in that window and every one of them changed something, which is the case the fix was meant to let through.
- MeasuredReject rows without a level, and version the pageAll 222 rows since 2 September carry a level, a signature and a page version, across sixteen commits. No row without a level has arrived since.
- OpenRecord where each visitor came fromStill the most urgent item. 1 October was 37 rows at a median of 23%, the fortnight after it 58 rows at 54%, and two later days had medians under 25% on a handful of rows each. Whether those are the same source trickling on or new ones, nothing in any column says. Storing
document.referrer, and a?cohort=tag when the link carries one, as a diagnostic column outside the signed string would let the next report tie a burst to its source. - NewCount slider interactions, not only the final positionThe time from proposal to submit sorts kept rows from moved ones better than any quiz answer does, but it cannot tell a visitor who agreed with the proposal from one who never engaged with it. Storing, outside the signed string, how many times each midpoint was dragged and whether it ever left the proposal before coming back would separate the two. It would also show whether the 44 movers who landed under 10% got there in one sweep or by working down. It costs one counter per slider.
- NewAsk check-takers to estimate their own score, and check a random sample of everyoneThe Dunning–Kruger effect is defined on the gap between how well people think they did and how well they did, and the site records only the second. One question after the check and before the verdict, "how many of the 30 do you think you got right?", stored with the row, would supply the first. And because only visitors who click the expert card are checked, the visitors who underrate themselves never appear; offering the check to a random tenth of beginner and medium choosers, after their quiz so it cannot change their path, would. With both, the next report could run the test properly, with the regression-to-the-mean correction the method needs.
- OpenShow the proposal to a random halfEvery statement here about the proposed number sets kept rows beside moved rows, which compares two kinds of visitor and not two treatments. The time finding makes the case sharper: a quick submitter keeps the number in front of them, so the number in front of them matters most for exactly the visitors who look at it least. Showing neutral sliders to a random half of quiz-takers, with the proposal stored but not applied and the arm stored in the row, is the one change that lets a report say "because". It is specified in the second kind of instrument.
- OpenSeed flagged decoys into all three self-selected quizzesEvery list in the beginner and medium quizzes can only be answered correctly, so the only thing it measures is how many boxes someone ticks, and in this export the movers' thirds on the medium list changed order for the third report running. The beginner catastrophe list is ticked 89 to 91 times of 93 on its top four answers. The expert check shows what a decoy does. The case is set out in the note on the beginner quiz.
- OpenAdd one recognition question about catastrophe pathways to the expert quizThe readers who pass the check put the last link of the chain at 50%, against 73% for beginners and 75% for medium takers, counting only visitors who set their own numbers. The expert quiz asks about mechanism and names and never mentions an outcome, so the cut could be rehearsal rather than belief. The note on it proposes one scored question; the version that would settle it shows the question to a random half of expert-quiz takers with the calibration rule held fixed.
- OpenKeep the expert quiz's reasoning questions, and watch themThey still tell the two expert cohorts apart: 43 of 74 who passed the check ticked all four self-improvement answers, against 22 of 22 who declared themselves experts. The verified share has held at a little over half for three exports.
- OpenAsk one identical question at every levelThe three quizzes share no question, so the level lines in this report's new chart cannot be compared without the confound of three different question sets and three different proposals. One shared question would fix that. The beginner gut-reaction question sorts its cohort from 34% to 72% across three answers, and 27% to 48% among movers, and would be a candidate, if the aim is to compare rather than to test.
- OpenStamp each submission with the option set it was shownThe page version pins it exactly. What remains is teaching the build script to read the denominators from the stamp instead of the hand-kept start dates. This build had to be told by hand, again, that two new page versions carry the current term list.
- OpenOffer a one-minute survey after submissionA psychologist's question, who submits 5% and who submits 90%, cannot be answered from quiz answers about films and vulnerabilities. The 2 September proposal picks eleven to thirteen items by hypothesis, places them after the submit button where they cannot cost a row, and stores them in their own table.
These are directions the data supports, not a verdict on the design. Several rest on small cells: twelve people turned away by the check, sixteen October beginner movers, time bands of 21 to 42 rows, thirds of 11 to 32 people, and one day that still supplies 37 of the 424 rows.
Since 2 September, not one identical repeat in 222 rows
This section is bookkeeping for the people who run the site: it checks that the rows above are what they claim to be. Each browser holds a non-extractable P-256 key and signs a canonical string of its submission. The report verifies every signature against the public key the row carries, so "these rows came from one browser" is checked rather than asserted. What a key cannot show is that two different keys are two different people.
Signatures
Browsers that submitted more than once
| Rows | Within | Levels | p(doom) | Reading |
|---|
A visitor who presses submit twice used to produce two rows. Since a fix on 2 September, none has.All six confirmed repeat rows in the table predate it, and every one of the 222 rows since is signed, verifies, and carries the commit the page was built from, across sixteen page versions. Eleven browsers submitted more than once in that window and every one of them changed something in between, which is the case the fix was meant to let through: three went beginner then medium, one of them thirteen days apart; two went beginner then cleared the check and took the expert quiz; one went medium then beginner; one took the medium quiz twice and then the expert one; and four took the same quiz more than once with the same answers and different sliders or bands. The one new browser since the last report is the last kind: on 2 October it took the beginner quiz, registered 52%, then moved the sliders twice more in the next six minutes and registered 19% and 20%. The 37 rows of 1 October come from 35 browsers. The rule looks at the whole payload, so it cannot catch a visitor who registers the same number twice with only an uncertainty band changed, and on its own terms it should not. All 253 signatures verify, and every field the signature covers agrees with the row: the p(doom) and its range, the three factors, the timestamp, the quiz level, the check score, the signing key and its counter. The signature does not cover the quiz answers, the terms ticked on the check, the proposed number, the time it was applied or the page version, so those columns are trusted, not checked. A key is a browser's signing identity, not a person: two keys can be one person, and one key is whoever holds that browser.