Whoa. Training Day
A new Amazon AI model, according to the researchers who built it, is exhibiting language abilities that it wasn't trained on.
In a not-yet-peer-reviewed academic paper, the team at Amazon AGI which stands for "artificial general intelligence," or human-level AI say their large language model (LLM) is exhibiting "state-of-the-art naturalness" at conversational text. Per the examples shared in the paper, the model does seem sophisticated.
As the paper indicates, the model was able to come up with all sorts of sentences that, according to criteria crafted with the help of an "expert linguist," showed it was making the types of language leaps that are natural in human language learners but have been difficult to obtain in AI.
Named "Big Adaptive Streamable TTS with Emergent abilities" or BASE TTS, the initial model was trained on 100,000 hours of "public domain speech data," 90 percent in English, to teach it how Americans talk. To test out how large models would need to be to show "emergent abilities," or abilities they were not trained on, the Amazon AGI team trained two smaller models, one on 1,000 hours of speech data and another on 10,000, to see which of the three if any exhibited the type of language naturalness they were looking for.
Interestingly enough, it was the 10,000-hour model the Goldilocks of the three, if you will that scored highest on the Amazon researchers' emergent abilities criteria list, which included things like the ability to understand punctuation, non-English words, and emotions.
The middle model spat out sentences that would seem to human readers very natural, exhibiting the ability to transcribe non-words ("Shh, Lucy, shhh, we mustnt wake your baby brother, Tom whispered, as they tiptoed past the nursery") and even the kind of internetspeak many netizens use in text messages and spoken language alike ("She received an odd text from her brother: 'Emergency @ home; call ASAP! Mom & Dad are worried#familymatters.'")
In the paper, whose international team of authors includes 18 AI experts, the Amazon AGI consortium pointed out that BASE TTS was never "explicitly" told to come up with its more surprising outputs.
"These sentences are designed to contain challenging tasks parsing garden-path sentences, placing phrasal stress on long-winded compound nouns, producing emotional or whispered speech, or producing the correct phonemes for foreign words like qi or punctuations like @ none of which BASE TTS is explicitly trained to perform," the paper reads.
It's not AGI, of course but these findings could regardless have implications on the path towards that goal, especially if it didn't need such a gigantic set of training data to get there.
More on AI leaps: AI Used to Resurrect Dead Dictator to Sway Election
Follow this link:
Amazon AGI Team Say Their AI Is Showing "Emergent Abilities" - Futurism
- 12 Days: Pole Attachment Changes in 2023 Set Stage for BEAD Implementation - BroadbandBreakfast.com - December 23rd, 2023 [December 23rd, 2023]
- The Most Important AI Innovations of 2024 | by AI News | Dec, 2023 - Medium - December 23rd, 2023 [December 23rd, 2023]
- To Accelerate or Decelerate AI: That is the Question - Quillette - December 23rd, 2023 [December 23rd, 2023]
- Its critical to regulate AI within the multi-trillion-dollar API economy - TechCrunch - December 23rd, 2023 [December 23rd, 2023]
- MIT Sloan Executive Education Launches New Course Offerings Focused on AI Business Strategies - AiThority - December 23rd, 2023 [December 23rd, 2023]
- Forget Dystopian Scenarios AI Is Pervasive Today, and the Risks Are Often Hidden - The Good Men Project - December 23rd, 2023 [December 23rd, 2023]
- The Era of AI: 2023's Landmark Year - CMSWire - December 23rd, 2023 [December 23rd, 2023]
- AGI Will Not Take All Our Jobs. But it will change all our jobs | by Russell Lim | Predict | Dec, 2023 - Medium - December 23rd, 2023 [December 23rd, 2023]
- Policy makers should plan for superintelligent AI, even if it never happens - Bulletin of the Atomic Scientists - December 23rd, 2023 [December 23rd, 2023]
- Will superintelligent AI sneak up on us? New study offers reassurance - Nature.com - December 23rd, 2023 [December 23rd, 2023]
- 4 reasons why this AI Godfather thinks we shouldn't be afraid - TechRadar - December 23rd, 2023 [December 23rd, 2023]
- AI consciousness: scientists say we urgently need answers - Nature.com - December 23rd, 2023 [December 23rd, 2023]
- On the Eve of An A.I. Extinction Risk'? In 2023, Advancements in A.I. Signaled Promise, and Prompted Warnings from ... - The Debrief - December 31st, 2023 [December 31st, 2023]
- Superintelligence Unleashed: Navigating the Perils and Promises of Tomorrow's AI Landscape with - Medium - December 31st, 2023 [December 31st, 2023]
- Stand with beneficial AGI | Sharon Gal Or | The Blogs - The Times of Israel - December 31st, 2023 [December 31st, 2023]
- Top 5 Myths About AI Debunked. Unraveling the Truth: Separating AI | by Michiel Meire | Dec, 2023 - Medium - December 31st, 2023 [December 31st, 2023]
- What Is Artificial Intelligence (AI)? - Council on Foreign Relations - December 31st, 2023 [December 31st, 2023]
- AGI predictions for 2024. The major LLM players, as well as | by Paul Pallaghy, PhD | Dec, 2023 - Medium - December 31st, 2023 [December 31st, 2023]
- Meta's long-term vision for AGI: 600,000 x NVIDIA H100-equivalent AI GPUs for future of AI - TweakTown - January 27th, 2024 [January 27th, 2024]
- Chrome Will Use Experimental AI Feature to Organize Tabs, Write Reviews - CNET - January 27th, 2024 [January 27th, 2024]
- Meta is joining its two AI teams in pursuit of open-source artificial general intelligence - MediaNama.com - January 27th, 2024 [January 27th, 2024]
- Tech CEOs Agree Human-Like AI Is Inevitable. Just Don't Ask Them What That Means - The Messenger - January 27th, 2024 [January 27th, 2024]
- Dream On, Mark Zuckerberg. Your New AI Bet Is a Real Long Shot - CNET - January 27th, 2024 [January 27th, 2024]
- Demystifying AI: The ABCs of Effective Integration - PYMNTS.com - January 27th, 2024 [January 27th, 2024]
- Artificial Intelligence & The Enigma Of Singularity - Welcome2TheBronx - February 4th, 2024 [February 4th, 2024]
- Former Tesla FSD Leader, Karpathy, Offers Glimpse into the Future of Artificial General Intelligence - Not a Tesla App - February 4th, 2024 [February 4th, 2024]
- Dembski: Does the Squawk Around AI Sound Like the Tower of Babel? - Walter Bradley Center for Natural and Artificial Intelligence - February 4th, 2024 [February 4th, 2024]
- AI child unveiled by award-winning Chinese scientist in Beijing - South China Morning Post - February 4th, 2024 [February 4th, 2024]
- I Asked GPT4 To Predict The Timeline Of Breakthrough Advances In Superintelligence Until The Year - Medium - February 4th, 2024 [February 4th, 2024]
- AGI: A Big Deal(Baby) That Needs Our Attention - Medium - February 4th, 2024 [February 4th, 2024]
- Human Intelligence Is Fundamentally Different From Machine Intelligence - Walter Bradley Center for Natural and Artificial Intelligence - February 4th, 2024 [February 4th, 2024]
- What is Artificial General Intelligence (AGI) and Why Should You Care? - AutoGPT Official - AutoGPT - February 4th, 2024 [February 4th, 2024]
- Artificial General Intelligence: Unleashing a New Era of Accessibility for People with Disabilities - Medriva - February 4th, 2024 [February 4th, 2024]
- Scholar Creates AI-Powered Simulated Child - Futurism - February 4th, 2024 [February 4th, 2024]
- AGI: Machines vs. Organisms - Discovery Institute - February 4th, 2024 [February 4th, 2024]
- Meta's AI boss says LLMs not enough: 'Human level AI is not just around the corner' - Cointelegraph - February 20th, 2024 [February 20th, 2024]
- Understanding the Intersection between AI and Intelligence: Debates, Capabilities, and Ethical Implications - Medriva - February 20th, 2024 [February 20th, 2024]
- Future of Artificial Intelligence: Predictions and Impact on Society - Medriva - February 20th, 2024 [February 20th, 2024]
- With Sora, OpenAI highlights the mystery and clarity of its mission | The AI Beat - VentureBeat - February 20th, 2024 [February 20th, 2024]
- ICEF Podcast: The impact of Artificial General Intelligence on international education - ICEF Monitor - February 20th, 2024 [February 20th, 2024]
- What is Artificial General Intelligence (AGI) and Why It's Not Here Yet: A Reality Check for AI Enthusiasts - Unite.AI - February 20th, 2024 [February 20th, 2024]
- There's AI, and Then There's AGI: What You Need to Know to Tell the Difference - CNET - February 20th, 2024 [February 20th, 2024]
- Will Artificial Intelligence Increase Unemployment? No. - Reason - April 30th, 2024 [April 30th, 2024]
- The Babelian Tower Of AI Alignment - NOEMA - Noema Magazine - April 30th, 2024 [April 30th, 2024]
- Elon Musk's xAI Close to Raising $6 Billion - PYMNTS.com - April 30th, 2024 [April 30th, 2024]
- Learning Before Legislating in Texas' AI Advisory Council - dallasinnovates.com - April 30th, 2024 [April 30th, 2024]
- Generative AI Defined: How It Works, Benefits and Dangers - TechRepublic - April 30th, 2024 [April 30th, 2024]
- The Future of Generative AI: Trends, Challenges, & Breakthroughs - eWeek - April 30th, 2024 [April 30th, 2024]
- Drink the Kool-Aid all you want, but don't call AI an existential threat - Bulletin of the Atomic Scientists - April 30th, 2024 [April 30th, 2024]