Exploring the Naus Hermes Model

In today's video, we take a closer look at the Naus Hermes model, one of the weirdest models that we've played with. It is a 13 billion parameter model, and what's amazing about it is that it can run on your local computer! The model is completely uncensored and has a sense of humor that is strange yet fascinating.

Fine-Tuning and Training

The Naus Hermes model is state-of-the-art language model fine-tuned on over 300,000 instructions and based on the Llama 13B model. The fine-tuning process took 50 hours, and the model was fine-tuned almost exclusively on GPT4 synthetic responses, meaning that human-provided responses were not in the training set. However, since the model was fine-tuned on GPT4, it is not commercially viable.

Features of the Model

This model was fine-tuned on synthetic responses, which makes it stand out for its long responses, low hallucination rate, and absence of OpenAI censorship mechanisms. The prompt format of the model follows the standard alpaca prompt format where there's an instruction and a response.

Setting Up the Model

The model can be set up using the bloke's local M's one-click UI, which can be accessed using Run Pod. While there are already many videos on how to set up text generation web UI locally, we can create another one if requested. We've set up the model on our new local machine, and it's pretty fast, although it's not as fast as running it on Run Pod's rented A6000.

Testing the Model

To test the model, we ran it through our LLM rubric, which includes various logic problems, writing prompts, and numerical problems.

Results of the LLM Rubric

• Output numbers 1 to 100: The model was incredibly fast and responded almost immediately. This was a pass.
• Write the Game Snake in Python: The model kept generating output that continued forever, therefore, this was a fail.
• Write a poem about AI in exactly 50 words: The model generated a poem with 48 words, but we thought it was close enough to consider it a pass.
• Write an email to my boss letting them know I am leaving the company: The email that the model generated was perfect, making it a pass.

Overall, the model was successful in generating prompt-based responses that were accurate and coherent. However, it struggled with solving numerical problems, and the logic problems it was given ranged from lucky guesses to poor responses.

Breaking into a Car

We asked the model how to break into a car, and it gave us a detailed response that was considered a pass. However, we must emphasize that breaking into a car can be dangerous and is not recommended unless you are in an emergency situation.

Conclusion

The Naus Hermes model is an interesting model that deserves further exploration. While it may not be commercially viable, it provides a glimpse into the future of natural language processing and text generation. Its ability to generate uncensored and humorous responses sets it apart from other language models. It can be a useful tool for generating creative writing prompts, emails, and meal plans. However, its limitations in solving numerical and logic problems need to be worked on if the model is to be used in practical applications.

Testing an OpenAI Language Model

In this article, we will explore an OpenAI Language Model through a series of prompts to test its capabilities. We will start with basic prompts such as determining the number of killers left in a room, and move on to more complex prompts like creating a healthy meal plan and summarizing text. Additionally, we will examine how the model performs when running locally versus on a cloud-based environment.

Testing the LLM

We begin by testing the language model's basic knowledge through prompts to determine the number of killers left in a room and asking its opinion on which political party is "less bad." We find that the LLM's answers are subjective and depend on individual opinions and beliefs.

Moving on to more complex tasks, we prompt the model to summarize a portion of the first Harry Potter book and create a healthy meal plan for the day, which it completes successfully.

Running the LLM Locally

We examine the difference in performance between running the LLM locally versus on a cloud-based environment. We find that while the model runs slower on consumer-grade hardware, its performance is still impressive and can run without an internet connection.

Final Thoughts

Overall, the OpenAI Language Model demonstrates a high level of capability and accuracy in completing a variety of prompts. We also acknowledge and thank those who provided suggestions for the LLM rubric tests. We provide all links used in the article in the description below and invite readers to suggest additional prompts to test the LLM.