/
Navigation
C
Chronicles
Browse all articles
C
E
Explore
Semantic exploration
E
R
Research
Entity momentum
R
N
Nexus
Correlations & relationships
N
~
Story Arc
Topic evolution
S
↻
Drift Map
Semantic trajectory animation
D
P
Posts
Analysis & commentary
P
Browse
@
Entities
Companies, people, products, technologies
◇
Domains
Browse by publication source
☉
Handles
Browse by social media handle
Detection
?
Concept Search
Semantic similarity search
!
High Impact Stories
Top coverage by position
+
Sentiment Analysis
Positive/negative coverage
*
Anomaly Detection
Unusual coverage patterns
Analysis
vs
Rivalry Report
Compare two entities head-to-head
/\
Semantic Pivots
Narrative discontinuities
!!
Crisis Response
Event recovery patterns
Connected
Nav: C E R N
Search: /
Command: ⌘K
Embeddings: large
VOICE ARCHIVE

Elizabeth Barnes

@bethmaybarnes
7 posts
2025-08-08
Wow that was not a great example of factualness. Famous common misconception [image]
2025-08-08 View on X
VentureBeat

OpenAI touts GPT-5's scores on math, coding, and health benchmarks: 94.6% on AIME 2025 without tools, 74.9% on SWE-bench Verified, and 46.2% on HealthBench Hard

After literally years of hype and speculation, OpenAI has officially launched a new lineup of large language models (LLMs) …

View original
The good news: due to increased access (plus improved evals science) we were able to do a more meaningful evaluation than with past models, and we think we have substantial evidence that this model does not pose a catastrophic risk via autonomy / loss of control threat models.
2025-08-08 View on X
VentureBeat

OpenAI touts GPT-5's scores on math, coding, and health benchmarks: 94.6% on AIME 2025 without tools, 74.9% on SWE-bench Verified, and 46.2% on HealthBench Hard

After literally years of hype and speculation, OpenAI has officially launched a new lineup of large language models (LLMs) …

View original
Wow that was not a great example of factualness. Famous common misconception [image]
2025-08-08 View on X
Simon Willison's Weblog

GPT-5 hands-on: it exudes competence but doesn't feel like a dramatic leap ahead of other LLMs, and the pricing is aggressively competitive with other providers

And It Changes Everything Tyler Cowen / Marginal Revolution : GPT-5, a short and enthusiastic review GPT-5 : GPT-5  —  Our hands-on review of OpenAI's newest model based on weeks o...

View original
The good news: due to increased access (plus improved evals science) we were able to do a more meaningful evaluation than with past models, and we think we have substantial evidence that this model does not pose a catastrophic risk via autonomy / loss of control threat models.
2025-08-08 View on X
Simon Willison's Weblog

GPT-5 hands-on: it exudes competence but doesn't feel like a dramatic leap ahead of other LLMs, and the pricing is aggressively competitive with other providers

And It Changes Everything Tyler Cowen / Marginal Revolution : GPT-5, a short and enthusiastic review GPT-5 : GPT-5  —  Our hands-on review of OpenAI's newest model based on weeks o...

View original
2025-07-12
1. One clear takeaway is that expert forecasts and even user self-reports are not reliable indicators about AI capabilities. Actually Measuring Things is very valuable!
2025-07-12 View on X
Second Thoughts

METR study: experienced open-source developers using Cursor, Claude, and other AI tools were 19% slower to complete tasks, despite thinking they were 20% faster

Devs estimated time saved with AI before & after each task, while screen recording to measured real-world AI productivity gains. @shimminykricket : A randomized controlled study fo...

View original
The lack of speedup is a bit less surprising to me considering high developer experience with the codebases (>5 years and >1500 PRs on average) and high quality bar - anecdotally I've heard AI tools don't usually speed up experts in their core areas/on very familiar codebases
2025-07-12 View on X
Second Thoughts

METR study: experienced open-source developers using Cursor, Claude, and other AI tools were 19% slower to complete tasks, despite thinking they were 20% faster

Devs estimated time saved with AI before & after each task, while screen recording to measured real-world AI productivity gains. @shimminykricket : A randomized controlled study fo...

View original
2. It's not easy to get current models to quickly write [high-quality code in mature codebases that meets stringent requirements]. We haven't totally ruled this out, but developers don't benefit from AI [starting from Cursor's basic setup] even with 30+ hours of practice.
2025-07-12 View on X
Second Thoughts

METR study: experienced open-source developers using Cursor, Claude, and other AI tools were 19% slower to complete tasks, despite thinking they were 20% faster

Devs estimated time saved with AI before & after each task, while screen recording to measured real-world AI productivity gains. @shimminykricket : A randomized controlled study fo...

View original