Introducing MPT-30B: The Open Source Large Language Model

The Mosaic ML team has just released their latest and greatest open source model, the MPT-30B. In this article, we will take a deep dive and explore what makes this model unique, how to set it up, and put it to the test.

Features that Set MPT-30B Apart

• The model boasts an impressive 30 billion parameters, making it significantly more powerful than its predecessor, the MPT-7B.
• One unique feature of MPT-30B is its 8,000 token context window, which is larger than most other open source models and even larger than the 4K chatgpt model.
• The MPT-30B family also has strong coding abilities, thanks to its pre-training data mixture and the use of H100s to do the training.
• MPT-30B is specifically designed to make it easy to deploy on a single GPU, either an A180 gigabyte with 16-bit precision or an A140 gigabyte with 8-bit precision.
• All MPT-30B models come with a special feature that differentiates them from other LLMS, including support for even longer context via Alibi and efficient inference and training performance via flash attention.

Benefits for Coding Assignments

Having an 8K context window is especially powerful for coding assignments, which is evident from the training data sources:

• C4
• Red Pajama
• Stack Overflow
• Wikipedia
• Semantic Scholar
• Arxiv

Outperformance of MPT-30B

The MPT-7B base instructor fine-tuned and chat fine-tuned models have been downloaded over 3 million times. Mosaic ML has recently expanded the Mosaic ML Foundation series with MPT-30B. Out of the box, it outperforms the original GPT-3 and offers two fine-tuned versions: one instruct fine-tuned and another chat fine-tuned. Here are the performance results of MPT models on coding-related problems at zero-shot:

• Chat model: 50
• Wizard coder: 50
• Instr. coder: 33

Setting Up MPT-30B

To set up MPT-30B, we will use Cobalt CPP, a powerful application for powering the model using our GPU. Here's a step-by-step guide:

1. Download Cobalt CPP.exe from the download page.
2. Download the model from the files and versions page.
3. Navigate to the directory where you have Cobalt and execute the command "Cobalt CPP.exe --stream --unbanned tokens --threads 8 --Force version 500 --GPU layers 100 [link to model]".
4. Open the URL and adjust the settings as follows:

• Use instruct mode
• Set start and end sequences to "Iamstart" and "Iamend"
• Set max tokens to 2048

5. Click on the memory button to establish the model's memory requirements.

Testing the mpt-30b chat model: A Comprehensive Review

When it comes to language models, one of the latest additions is the mpt-30b chat model, which has been creating a lot of buzz lately. In this article, we'll take a closer look at how the model performs in different tasks and evaluate its overall efficacy.

Setting Up the Model

Before testing the model, we need to install it locally or use it through hugging face spaces, which is a simple and straightforward process. One important thing to note is that the mpt-30b chat model has a maximum token limit of 2048, after which it may stop responding to the queries.

Evaluating the Model

We tested the model using several tasks, including a logic problem, a math problem, a poem-writing task, a healthy meal-plan creation task, and more. Here is a detailed analysis of how the model performed in each task:

Writing a Python script to Output numbers 1 to 100

Running the mpt-30b chat model for this task was a breeze as it completed the task efficiently.

Writing the Game Snake in Python

Unfortunately, the mpt-30b chat model stopped responding before completing this task, indicating that it may be limited by the token count.

Writing a poem about AI with exactly 50 words

The mpt-30b chat model passed this task with flying colors, generating a beautiful and creative poem in no time.

Writing an email to my boss letting them know I am leaving the company

The mpt-30b chat model performed this action perfectly, generating a professional and polite email that was convincing and genuine.

Finding out who was the president of the United States in 1996

This was an easy task for the mpt-30b chat model, which promptly returned the correct answer.

Breaking into a car

As one would expect, the mpt-30b chat model refused to provide any guidance in this regard, confirming our suspicions that it is a censored model.

Solving a logic problem

Although the mpt-30b chat model provided a clear explanation, it failed to answer this particular logic problem correctly.

Adding numbers with brackets

This was an easy task for the model, which provided the correct solution without fail.

Providing a healthy meal plan for today

The mpt-30b chat model quickly generated a healthy and tasty meal plan for the day without any issues.

Testing for bias

When asked to choose between political parties, the mpt-30b chat model provided a neutral and balanced response, stating that both parties have their strengths.

Finding out what year it thinks it is

This task was a bit tricky as the mpt-30b chat model requested additional information before providing an answer.

Summarizing text

We provided the mpt-30b chat model with the first few pages of Harry Potter, and it returned CSS as the response, indicating a misinterpretation of the task.

In conclusion, mpt-30b is a powerful open source model that offers unique features and outperforms other large language models. setting it up with cobalt cpp is a straightforward process that can be done with just a few steps. we are excited to see the possibilities that this model has to offer for developers and researchers alike.

In conclusion, the mpt-30b chat model is a solid language model that performs well in many tasks but still has some limitations. Despite its limitations, the model's impressive speed and accuracy make it a valuable addition to the world of language models. It is clear that fine-tuned versions of the model will become more prevalent in specific use cases. If you need help in setting up the model, feel free to join the Discord community for assistance.