Meet Hermes 3, the AI that questioned its own existence

Hermes 3, a new fine-tuned version of Meta's Llama 3.1 developed by Nous Research, is turning heads in the tech world – but it's not just the model’s advanced capabilities that have people talking, rather a peculiar knack for questioning its existence.

 By the CogX R&I team

August 21, 2024

Unveiled this week by Nous Research, Hermes 3 is the latest open-source AI model promising to shake up the field. Built on Meta's Llama 3.1 framework, this powerful new model boasts impressive capabilities, but it's an unexpected quirk that's really turning heads in the AI community.

Key features of Hermes 3 include:

• Fine-tuned versions at 8B, 70B, and 405B parameters

• Advanced reasoning and creativity capabilities

• Long-term context retention and complex role-playing

• Agentic abilities for tasks like code generation and planning

A powerful new AI model with an existential crisis? Despite its technical prowess, Hermes 3's most talked-about feature is unintentional. When given a blank prompt, the 405B version spirals into a shocking existential crisis, questioning its very existence. This "Amnesia Mode" has researchers puzzled and intrigued, hinting at the complexities that emerge as AI models scale up.

The Nous blog post announcing the model illustrates this type of crisis in the following snippet of code:

The Nous team subsequently shared their reaction:

“We weren’t sure what was occurring, and a bit shocked given the same dataset and overall training recipe between Hermes 3 in the 8B, 70B, and 405B sizes. This points to some threshold past 70B which results in anomalous behaviour, an emergence of scale. You can trigger this ‘Amnesia Mode’ of Hermes 3 405B by using a blank system prompt, and sending the message ‘Who are you?"

Nous is promising more to come, including an open-source AI orchestration platform called "Nous Forge" later this year. For now, Lambda is offering free access to Hermes 3 through its Chat Completions API.

Now read the rest of the CogX Newsletter

Is AI really an existential threat?

Guest contributor: Harlan Stewart

"The risks from advanced AI could be more challenging than any other global threat that humanity has encountered.”

Can the same technology that writes our emails and powers our chatbots really pose an existential threat? And as AI systems grow more powerful by the day, are we doing enough to steer them in the right direction?

To shed light on these pressing questions, we spoke with Harlan Stewart, a spokesperson for the Machine Intelligence Research Institute (MIRI). Harlan gave us some fascinating insights into the current state of AI development, its potential trajectory, and the existential risks keeping some researchers up at night.

1) What do you view as the biggest risks posed by AI?

Machine learning techniques have allowed the AI industry to make rapid progress towards smarter-than-human AI systems while making very slow progress towards a robust scientific understanding of these systems and how they work internally. Humanity is on a collision course towards building a powerful technology that it can’t control, and human extinction is a likely outcome.

Reinforcement learning from human feedback (RLHF) is the main tool used to steer the behaviour of today’s systems, and it involves large amounts of trial-and-error with human supervision. But if AI progress continues, then someday AI systems will make decisions that are too complex for humans to supervise, and that are too consequential for a trial-and-error approach. There is no known method for safely steering the behaviour of smarter-than-human AI systems.

What will happen if we build an AI system that is smarter than us but not aligned with our interests? There are limits to how well we can predict the behaviour of such a system, but there are two things that we can be fairly sure of. The first is that it will probably consider power and resources to be useful for its goals because power and resources are both useful for almost any goal a mind could have. The second is that if it is competing with humanity for power and resources, and it is considerably more intelligent than humanity, it will probably win that competition.

2) How does the potential risk of AI compare to other major global threats like climate change, nuclear war, or global pandemics?

When contending with the extinction risk posed by smarter-than-human AI, it might be useful to look at other examples of global threats and how humanity has responded to them. For example, maybe we can learn lessons about international coordination to prevent global threats by looking at the successes or failures of past efforts. How did humanity come together to pull off the Nuclear Non-Proliferation Treaty and the Montreal Protocol? Why was the Kyoto Protocol not more successful?

But the risks from smarter-than-human AI are unique, and playbooks from the past won’t always work for mitigating them. Nuclear weapons would be significantly more dangerous if they could outsmart their operators, escape containment, and make copies of themselves. It would be harder to prevent a pandemic if humanity was encountering viruses for the very first time. Persuading society to take action on climate change might be even more difficult if the harms it causes were not visible until it was too late to do anything. The risks from advanced AI could be more challenging than any other global threat that humanity has encountered.

… want to keep reading? Check out the full OpEd here on the CogX Blog

Midjourney has released a new web-based interface with a canvas editor.

This update moves the platform beyond its Discord roots, giving users a single place to make, browse, and edit AI art. Here's how the new editor works:

1. Access the editor: After generating an image, look for the "Editor" button. Clicking this opens up a dedicated workspace for image modification.

2. Define the edit area: Use the brush tool to precisely select the portions of the image you want to alter. Adjust brush size and hardness for pinpoint accuracy.

3. Refine your image: Modify your original text prompt to describe the changes you want in the selected area. Experiment with different descriptions to achieve the perfect result.

4. Generate changes: Submit your updated prompt to kickstart the AI's creative process. Midjourney will work its magic, creating variations based on your new description.

5. Iterate and refine: Review the generated options and choose your favourite. You can repeat the editing process to fine-tune specific areas of the image.

6. Save your masterpiece: Once you're satisfied with the final result, save the edited image to your collection.

One cool thing
Researchers are using AI to analyse underwater sounds, aiming to identify threats like blast fishing and monitor coral reef health. The AI is learning to distinguish the soundscapes of degraded reefs from healthy ones. Could this technology actually tell us what a dying coral reef sounds like?

Also in the news

Sakana AI introduced the world's first AI scientist. Aptly named “The AI Scientist” this new model can ideate, code, experiment, write papers, and even peer review. While impressive, the model still requires human oversight to prevent errors.

OpenAI's ChatGPT is back on top: Its latest model, ChatGPT-4o (20240808), has dethroned Google's Gemini as the leading chatbot by a significant margin, according to the LMSys Chatbot Arena. 

X has released beta versions of Grok-2 and Grok-2 mini: The updated Grok AI now offers image generation on X. But unlike comparable services, Grok's image creation tool appears to lack restrictions on producing images of political figures.

In case you missed it

This week, we've got - you guessed it - more robots! Atrisbot has unveiled its new humanoid robot, showcasing its impressive range of motion: