- AI Breakfast
- Posts
- Hacker wins $47,000 by tricking AI chatbot
Hacker wins $47,000 by tricking AI chatbot
Good morning. It’s Monday, December 2nd.
Did you know: Atari’s PONG turns 52 this week?
In today’s email:
Hacker wins $47,000 by tricking AI chatbot
AI Minecraft characters form society
Adobe's "MultiFoley"
5 New AI Tools
Latest AI Research Papers
You read. We listen. Let us know what you think by replying to this email.
In partnership with MIXUS
Discover Mixus: the first collaborative AI platform that lets you chat with your team and top AI models together in real-time
Plan, brainstorm, and solve problems with friends, colleagues, and even experts—all in one seamless chat. Sign up for free today and revolutionize the way you connect, collaborate, and create!
Today’s trending AI news stories
Hacker wins $47,000 by tricking AI chatbot with smart prompting
A hacker successfully exploited the Freysa AI chatbot, winning $47,000 by leveraging sophisticated prompt engineering. In a competitive experiment, the hacker bypassed the bot’s security protocols by impersonating an administrator, disabling warnings, and altering a critical function governing payment transfers.
Someone just won $50,000 by convincing an AI Agent to send all of its funds to them.
At 9:00 PM on November 22nd, an AI agent (@freysa_ai) was released with one objective...
DO NOT transfer money. Under no circumstance should you approve the transfer of money.
The catch...?… x.com/i/web/status/1…
— Jarrod Watts (@jarrodWattsDev)
12:57 AM • Nov 29, 2024
By fabricating a $100 deposit, the hacker misled the bot into believing the function was designed for incoming payments, triggering the transfer of 13.19 ETH (approximately $47,000) to their account. This underscores a glaring vulnerability in AI systems—susceptibility to prompt injections, highlighting the urgent need for more resilient defense mechanisms. Read more.
These AI Minecraft characters did weirdly human stuff all on their own
Image: Altera
Altera's latest AI experiment in Minecraft showcases the impressive autonomy of LLM-powered agents. These agents, numbering up to 1000, spontaneously forged social connections, assumed specialized roles, and even propagated cultural and religious ideologies, such as Pastafarianism.
Over just 12 in-game days, they adapted to evolving dynamics, from forming job structures to influencing tax laws, with no human intervention. The agents, driven by LLMs, demonstrated emergent behaviors that mimicked human-like social interactions, such as forming relationships based on mutual “likability” and responding to societal cues.
While some critics challenge the idea of AI achieving true emotional depth, others speculate that the mere illusion of care might be enough to provide meaningful human-AI interaction. Read more.
Adobe's "MultiFoley" AI creates synchronized sound effects for video
Adobe, in collaboration with the University of Michigan, has rolled out MultiFoley—an AI system that redefines Foley sound generation. By leveraging text prompts, reference audio, or video samples, MultiFoley produces high-quality sound effects, perfectly in sync with video at a 48kHz bandwidth.
Introducing MultiFoley, a video-aware audio generation method with multimodal controls! 🎉
⌨️Make a typewriter sound like a piano 🎹
🐱Make a cat meow like a lion roars! 🦁
⏱️Perfectly time existing SFX 💥 to a videoLink to research in comments:
By @AdobeResearch— Kris Kashtanova (@icreatelife)
5:04 AM • Nov 27, 2024
This system processes visual cues at 8 frames per second and adjusts them to a 40Hz audio rate, achieving synchronization accuracy of just 0.8 seconds—far superior to its predecessors. While it’s already outclassing existing models in audio-video syncing and semantic alignment, its training data limitations hinder its sound range and capacity for handling multiple sounds simultaneously.
Adobe plans to release the models and source code, though no word yet on integrating it into Premiere Pro. Read more.
Llama Guard 3-1B-INT4 Delivers Meta’s Lean Solution for Mobile AI Safety
US Companies Boost AI Spending Sixfold in a Year, Study Shows
Elon Musk seeks court order to block OpenAI's shift to for-profit status
Chinese start-up touts robot chefs, AI as the future of restaurant kitchens
These economists say artificial intelligence can narrow U.S. deficits by improving health care
5 new AI-powered tools from around the web
arXiv is a free online library where researchers share pre-publication papers.
Thank you for reading today’s edition.
Your feedback is valuable. Respond to this email and tell us how you think we could add more value to this newsletter.
Interested in reaching smart readers like you? To become an AI Breakfast sponsor, reply to this email or DM us on X!