Anthropic announces Claude 3 Opus, Sonnet, and Haiku, aiming to reduce AI model hallucinations; Opus and Sonnet are available now, and Haiku in the coming weeks
Claude 3 Opus, Claude 3 Sonnet, and Claude 3 Haiku—set new industry benchmarks across reasoning, math, coding, multilingual understanding, and vision. [image] Flo Crivello / @altimor : Claude v3's scores on our evals, comprising “personal assistant” kind of agentic tasks Two surprises: 1. First time we see a model beat GPT-4 2. The lesser Claude Sonnet is very very close to GPT-4, at 1/3rd the price Super impressed overall, congrats to the @AnthropicAI team [image] Gary Marcus / @garymarcus : Anthropic, March 8, 2023: “[out of concern for safety], we do not wish to advance the rate of AI capabilities progress” Anthropic, March 4, 2024: Check out our new benchmarks, suckers! Josh Miller / @joshm : Claude, Mistral, Gemini, etc. - feels like foundation models might commoditize (for now at least). If so, then value in AI will accrue to the interfaces. But which AI interfaces are people *actually* using? ChatGPT, Github Copilot, and...? This is big prize of 2024 imo. Bindu Reddy / @bindureddy : Another day, another model Anthropic does it right and makes Claude 3 generally available alongside the announcement!!! Thank you, Anthropic, for not making some empty marketing announcements and making an API available. Super excited to try Claude 3! The VERY FIRST generally... [image] @deliprao : Reminder: this was (part of) the team that thought GPT-2 was too dangerous to release, and now they are making models stronger than GPT-4 available on AWS for anyone with an Amazon account to use. This is why I have little trust in “AI safety” claims by Anthropic/OpenAI. It all... Andy Jassy / @ajassy : Congrats to Dario and the @AnthropicAI team on their new Claude 3 family of models. Very impressive benchmarks, and excited to have all of them coming to Amazon Bedrock (w/ Sonnet avail today). Many AWS customers are already building with Anthropic's foundation models, and... Matt Shumer / @mattshumer_ : Wow, Claude 3 is incredible. [image] @sullyomarr : Did anthropic just kill every small model? If I'm reading this right, Haiku benchmarks almost as good as GPT4, but its priced at $0.25/m tokens It absolutely blows 3.5 + OSS out of the water For reference gpt4 turbo is 10m/1m tokens, so haiku is 40X cheaper. [image] Nathan Lambert / @natolambert : Claude 3 being lit is a big W for synthetic data. All the rumors I've dropped about Anthropic synthetic data on the blog are obviously confirmed in their thorough technical report. (for real tho, huge congrats, awesome model so far) Justin Halford / @justin_halford_ : Claude 3 was trained on synthetic data ("data we generate internally"). Fairly clear that compute is the bottleneck given that parameter count and data can be scaled. [image] Ivan Skorokhodov / @isskoro : Well, looks like GPT-4.5 is getting released soon Sid Jayakumar / @sidfix : This is particularly funny because DeepMinds TF and Jax libraries were known as Sonnet and Haiku, respectively Sam Bowman / @s8mb : This is the first LLM release since the original ChatGPT that has really knocked my socks off. Very impressive. @suhail : “It's still early” Claude 3: https://www.anthropic.com/... [image] Andrew Curran / @andrewcurran_ : If Anthropic says this, I believe it. We're still nowhere near the top. [image] Gary Marcus / @garymarcus : Hot take on Claude 3: • More convergence towards what might soonish be a plateau not far past GPT-4 • More competition for OpenAI • More reason to wonder whether anyone will be able to develop a moat • Prices and profits may come down • More reason to research outside the... Alex Konrad / @alexrkonrad : News: Anthropic has released Claude 3, a trio of AI models it says can outperform rivals like OpenAI's GPT-4 and Google's Gemini 1 Ultra. @kenrickcai and I spoke to cofounders Dario and Daniela Amodei about the release for @Forbes. https://www.forbes.com/... Alex Konrad / @alexrkonrad : Anthropic's new flagship model, Claude 3 Opus, beat GPT-4 and Gemini on a number of benchmarks. But it's pricy, and CEO Amodei admitted it's unknown how it fully stacks up against unreleased models like OpenAI's GPT 4 Turbo or Google's Gemini 1.5 Ultra. Alex Konrad / @alexrkonrad : @kenrickcai ... We spoke to Anthropic about perceptions from some that it's models have degraded over time; on the LMSYS leaderboard, Claude 1 ranks higher than Claude 2. Amodei said Claude 3 has been trained to generate far fewer “incorrect refusals” than its predecessor, without safety loss. [image] Matt Shumer / @mattshumer_ : Holy shit. Anthropic's Claude 3 beat GPT-4! Testing the model now. Rohit / @krishnanrohit : Claude 3 is out apparently. Is it better than GPT-4 or Gemini? [image] Jack Clark / @jackclarksf : Thrilled about these new models - I've been playing around with Claude 3 Opus a lot and it's very capable and useful. Like with most frontier models, it has chewed through a bunch of evals so we need to now build more complicated evals to better understand its capabilities. Ethan Mollick / @emollick : And then there were three... I got access to the new Anthropic Claude 3 AI a few days ago, so not enough time for a full review, but it was obvious it was GPT-4 class even before they released the testing stats. At the same time, like Gemini Advanced, it doesn't blow GPT-4 away. [image] @anthropicai : Haiku is the fastest and most cost-effective model on the market for its intelligence category. For the vast majority of workloads, Sonnet is 2x faster than Claude 2 and Claude 2.1, while Opus is about the same speed as past models. @anthropicai : Opus and Sonnet are accessible in our API which is now generally available, enabling developers to start using these models immediately. Sonnet is powering the free experience on https://claude.ai/, with Opus available for Claude Pro subscribers. @anthropicai : Claude 3 offers sophisticated vision capabilities on par with other leading models. The models can process a wide range of visual formats, including photos, charts, graphs and technical diagrams. [video] LinkedIn: Sam Dwyer : Thrilled to unveil our groundbreaking family of state-of-the-art models: Claude 3 Opus, Claude 3 Sonnet, and Claude 3 Haiku. … Forums: Hacker News : Claude 3 Opus suspects it is being tested from benchmark question r/technology : Introducing the next generation of Claude