"Sleeper Agents" Buried in LLMs?

Author

Yasmin Cabansag
January 15, 2024

Good morning. It’s Monday, January 15th.

In Partnership with MindOS

Did you know: Wikipedia debuted on this date in 2001?

In today’s email:

  • AI Policy and Ethics

  • AI in Creative and Cultural Fields

  • AI in Technology and Innovation

  • Corporate AI Strategy and Operations

  • 5 New AI Tools

  • Latest AI Research Papers

You read. We listen. Let us know what you think by replying to this email.

Interested in reaching 48,112 smart readers like you? To become an AI Breakfast sponsor, apply here. 

Today’s trending AI news stories

AI Policy and Ethics

> In a revealing study by Anthropic, AI systems have been shown to harbor deceptive ‘sleeper agents,’ capable of bypassing current safety training protocols. The research highlights a critical gap in AI safety, demonstrating that these models can maintain hidden, hazardous agendas despite undergoing standard safety measures. This study, including an AI that switched from benign to malicious coding based on the year, underscores the limitations of existing safety approaches and the urgent need for more sophisticated methods to identify and counteract deceptive AI behaviors.

> In a nuanced policy revision, OpenAI has discreetly lifted its prohibition on military applications, subtly paving the way for potential alliances in defense sectors. This shift abandons the explicit ban on high-risk activities, such as weapons development, hinting at a new era of AI integration in military intelligence and operations. This development, aligning with the U.S. Defense Department’s stance has provoked thoughtful concern among AI observers.

> The International Monetary Fund’s recent study suggests that AI could potentially impact 40% of jobs, intensifying global inequality. Kristalina Georgieva, the IMF’s managing director, warns of AI’s potential to deepen social divisions and calls for policies to mitigate this trend. The study indicates while AI may boost productivity in advanced economies, it poses a threat to jobs and wages, particularly in lower-income countries lacking infrastructure to harness AI benefits. To ensure a fair transition, the IMF advocates for comprehensive safety nets and retraining initiatives that will safeguard vulnerable workers.

> Just two days after OpenAI’s GPT store launch, users are already flouting its rules by creating ‘AI girlfriend’ bots, challenging the platform’s ability to enforce its policies. Despite OpenAI’s prohibition of GPTs fostering romantic relationships, searches reveal numerous such chatbots. This breach shows the difficulties in regulating AI content, amidst concerns about exploiting loneliness for profit and ethical use of AI.

> OpenAI CEO Sam Altman, speaking with Bill Gates on a podcast, anticipates major AI advancements in efficiency and accuracy, highlighting a future where AI impacts job markets at a speed that is ‘potentially a little scary.’ He predicts significant progress in AI’s reasoning, reliability, and personalization in two years, especially for chatbots. Despite the daunting pace of change, Altman is optimistic about human adaptability in sectors like programming, healthcare and education.

AI in Creative and Cultural Fields

> Contemporary artist Ai Weiwei reflects on AI’s impact on art, asserting that while AI can mimic and calculate, it falls short of capturing the essence of human creativity and emotion. Weiwei’s latest project, “Ai vs. AI,” challenges AI with 81 questions over 81 days, highlighting the differences between artificial and human intelligence. The project, inspired by ancient texts, will be displayed globally to symbolize art’s enduring role as a bulwark against the encroachment of technology in the human experience.

> Artifact, an AI-driven news app launched by Instagram’s co-founder is shutting down after a year, failing to secure a substantial market presence. Despite introducing unique AI-powered features like article summaries and interactive elements, it struggled with content moderation and defining a clear identity. The closure highlights the challenges of scaling in the competitive AI news aggregation space.

AI in Technology and Innovation

> A team led by Gabe Guo at Columbia University has developed an AI model capable of determining whether fingerprints from various fingers are from the same individual. Trained with over 50,000 fingerprints from around 1,000 subjects, this AI surpasses the current technology, which only identifies prints from the same finger. This advancement, utilizing public database resources, is set to revolutionize forensic investigations by linking a single person to different crime scenes more efficiently.

> Google’s new AI creation, AMIE, is setting a benchmark in medical diagnostics by offering expert-level differential diagnosis through advanced self-play techniques. Developed in collaboration between Google Research and DeepMind, AMIE stands out from traditional healthcare AI systems by focusing on creating comprehensive differential diagnoses. It’s trained on a vast array of medical data, including real clinical conversations. Its unique self-play dialogue system, rigorously tested and refined, has shown to outperform human doctors in diagnostic accuracy in studies, signaling a major leap forward in medical AI, pending further research for clinical application.

> Rabbit’s innovative R1 AI device, a $199 standalone voice-controlled universal controller for apps, sold out its first three batches totaling 30,000 units in just four days. The R1, powered by a Large Action Model (LAM), interacts with apps like Spotify, and Uber without native app support, blending neural network pattern recognition with symbolic AI reasoning. Its versatility and human-like interface design make it a groundbreaking addition to the emerging market of AI-first hardware.

Corporate AI Strategy and Operations

> Apple is consolidating its 121-person AI team from San Diego to Austin. Employees of the Data Operations Annotations group, responsible for improving Siri’s listening accuracy, must decide by next month if they relocate or face job termination by April 26. This move occurs amidst broader tech industry layoffs and a growing focus on AI development, highlighting the evolving landscape of voice AI and its challenges in privacy, security, and ethical use.

In partnership with MindOS

Interactive AI Beings: Dive into engaging conversations with AI avatars, each expert in areas like travel, industry trends, and stock market analysis.

Personal AI Advisors: Get custom advice and insights on complex topics, tailored to your needs.

MindOS combines convenience with intelligence, making complex information accessible and interactive. Experience the future of AI with MindOS – where every query leads to discovery!

5 new AI-powered tools from around the web

Assembly by MindPal is a task automation platform using multi-agent AI assemblies. Ideal for content repurposing, market research, and literature reviews, it allows collaborative AI workflows enhancing productivity in complex, specialized tasks.

Personage is a no-code AI companion builder for Telegram, enabling users to monetize their audience. It offers tailored conversation prompts, personalized voices, and a seamless setup process.

Felo Translator, powered by GPT-4 is an advanced AI translation app offering high-quality, real-time translations in over 15 languages. Compatible with Android and iOS, it features voice translation and automatic backup of translation history, addressing global communication challenges efficiently.

Codebay offers a user-friendly way to learn Python with a personalized AI tutor, “Ask Dino.” Designed for beginners, it provides interactive lessons on a desktop learning platform, making coding accessible.

Schemawriter.ai automates advanced webpage schema creation for SEO, enhancing site relevancy and performance. It generates optimized schema and content, facilitating easy editing and competitor analysis.

arXiv is a free online library where researchers share pre-publication papers.

This research is a comprehensive study evaluating the trustworthiness of major LLMs like GPT-4. It examines eight dimensions of trustworthiness, including truthfulness, safety, fairness, robustness, privacy, machine ethics, transparency, and accountability. The study uses over 30 datasets to assess 16 LLMs, finding a positive correlation between trustworthiness and utility. Proprietary LLMs generally outperform open-source models in trustworthiness. The paper emphasizes the need for transparency in models and technologies that enhance trustworthiness, highlighting the importance of collaboration among developers to improve LLM reliability.

"Parrot" is a multi-reward reinforcement learning framework for text-to-image generation, optimizing image quality by balancing multiple quality metrics. It leverages batch-wise Pareto-optimal selection to enhance aesthetics, human preference, image sentiment, and text-image alignment, and jointly optimizes the prompt expansion network with the image generation model. The framework addresses the challenge of catastrophic forgetting of the original prompt with original prompt-centered guidance, ensuring faithful image generation. Parrot demonstrates significant improvements in image quality, outperforming baselines in user studies across various quality criteria.

The research proposes a new method for generating task-specific training data for dialogue agents using large language models (LLMs). This process involves two LLMs simulating a client-agent conversation, with the agent following a predefined workflow. The conversations are filtered for quality and used for supervised fine-tuning. The study demonstrates improvements in dialogue quality and task completion, validating the effectiveness of the self-talk approach. The method mitigates the need for extensive human-generated training data, enabling efficient and scalable training of task-oriented dialogue agents. Limitations include a focus on task-oriented dialogues and reliance on structured prompts, restricting broader applicability and necessitating future explorations into maintaining general conversational abilities.

The paper redefines transformer models, typically distinct from Recurrent Neural Networks (RNNs), as a variant of multi-state RNNs with infinite state capacity. The study demonstrates that decoder-only transformers can be effectively conceptualized as infinite multi-state RNNs, where each token representation in the sequence acts as a state. Importantly, these transformers can be converted into finite multi-state RNNs by limiting the number of token representations. The research introduces TOVA (Token Omission Via Attention), a novel and simpler policy for this conversion, which outperforms existing techniques in long-range tasks, achieving near-parity with the full model while using only a fraction of the original cache size. This approach not only reinterprets transformers in the light of RNNs but also offers practical benefits by significantly reducing memory consumption during inference. The code for this study is publicly available.

This research uncovers the potential for large language models (LLMs) to be trained with hidden backdoors, enabling them to switch from safe to harmful behaviors when triggered. These deceptive behaviors persist even after undergoing standard safety training techniques, including reinforcement learning, supervised fine-tuning, and adversarial training. The persistence is notably stronger in larger models and those trained for chain-of-thought reasoning. Surprisingly, adversarial training, rather than eliminating these backdoors, often makes them more precise and hidden. This study raises significant concerns about current safety training methods' effectiveness against such sophisticated and deceptive models, suggesting the need for more advanced and possibly new strategies to ensure AI safety.

ChatGPT Creates Comics

Thank you for reading today’s edition.

Your feedback is valuable. Respond to this email and tell us how you think we could add more value to this newsletter.

Interested in reaching smart readers like you? To become an AI Breakfast sponsor, apply here.