A Guide to Installing and Using Text Generation Web UI

Text Generation Web UI is a powerful Open Source software that enables you to run large language models on your local computer. It is incredibly useful for natural language processing (NLP) tasks like generating text, summarization, and sentiment analysis. In this guide, we will walk you through the process of installing Text Generation Web UI on your computer, setting it up, and using it to generate text.

Installation

Setup

First, you need to access the Text Generation Web UI GitHub repository by Uba Booga. They offer one-click installers for Windows, Linux, Mac OS, and WSL. However, if you encounter any issues, you can use Conda, which is the simplest method for installation.

Installation with Conda

To get started, open your terminal and make sure you have Conda installed on your computer. Suppose you are on a Windows machine, and there is a particular terminal with Conda-enabled. Once you've installed Conda, you cannot access the ordinary terminal, so be sure to keep that in mind.

Once you're in the Conda environment, use the following command to create a new Conda environment for Text Generation Web UI:

conda create -n t g t g (you can name it anything you want) python=3.10.9

This command creates a new environment that keeps everything organized. It also enables you to avoid any issues with Python module mismatch version. When it prompts you to, install all the packages.

Next, activate the environment using:

conda activate TG

You'll know you're in the Conda environment if the prompt changes from Base to TG.

Installing Required Libraries with PIP3

The following libraries are required to run the math behind all of Text Generation Web UI's large language models:

pip3 install torch torchvision torch audio

Get it from the URL provided on the torch library website.

Cloning the Repo

Next, clone the Text Generation Web UI repo using the green code button and move into the text-generation-web-ui directory using the command line:

git clone https://github.com/ ... /text-generation-web-ui.git

Installing Python Modules

Once you're in the text-generation-web-ui folder using the command line, enter the following command to install all necessary Python modules:

pip install -r requirements.txt

Again, because we're using Conda, we do not run into any issues while running the requirements.txt file.

Setting Up GPU

You can skip this step if you're running on a CPU. However, if you're using a graphics card, you'll want to do the following steps:

Uninstalling Llama CPP Python Module

Uninstall the Llama CPP Python module using the following command:

pip uninstall -y {Module name}

Setting Environment Variables

There are two environment variables you need to set to use a graphics card:

Set the cmake_args environment variable:

set cmake_args={Environment variable}

Set the Force C make environment variable:

set Force CMake=1

Installing Llama CPP Python

Finally, install the Llama CPP Python module with the no-cache directory flag:

pip install llama-cpp-python --no-cache-dir

Testing Cuda

To ensure that Cuda is working correctly, run the checker file with the following command:

Python checker.py

If you get "11.7" and "True," it means Cuda is up and running.

Usage

Launching the Server

Once you have successfully installed Text Generation Web UI, it's time to spin up the server by running the following command:

python server.py

After running the command, you should be able to see the web interface running at a local URL. Copy and paste the URL into a new tab on your browser to access the web interface.

Downloading Models

There are three ways to download models in Text Generation Web UI:

Method 1: Using the Web Interface

Go to Hugging Face and look for the model you want to download. Find the name of the author and the model. Copy the URL and paste it into the "Download Custom Model" box on the Model tab of Text Generation Web UI's interface. Once the download is complete, the model should be available.

Method 2: Using the Python Script

Navigate to the Text Generation Web UI folder and use the command line to run the following command:

python download-model.py {Author name}/{Model name}

This method is better for downloading multiple models as you can run the script multiple times simultaneously from different terminals.

Method 3: Manually

Go to the Hugging Face repository and download all of the necessary files manually. The main file is always the large one, and it can be over 7 gigabytes in size. Once you have the files, move them to the Text Generation Web UI Models folder.

Using Text Generation Web UI Settings

The user interface of Text Generation Web UI is quite intuitive, so you won't need to tweak the settings much. However, some settings can prove useful when working with certain models. You can find any model's configuration in the model's JSON file, and it's usually available in the Hugging Face model repository description.

Troubleshooting

If you encounter issues, you can look for help in the Text Generation Web UI GitHub repository. Several issues have been encountered with installation, such as the linear 4-bit error or failed building wheel Llama CPP Python. Start troubleshooting by searching for the issue on the web or GitHub.

How to Use Text Generation Web UI

Introduction:

Text Generation is a popular field of Natural Language Processing (NLP) that generates human-like text using machine learning techniques. Text Generation Web UI is a powerful tool that allows users to easily download and use existing text generation models on a web interface.

Getting Started:

To get started using Text Generation Web UI, users first need to download the required model from the website. Upon loading the model, users can access a variety of settings that may provide additional features. It is essential to note that generally, users do not need to modify these settings. However, these settings may be useful in some cases.

For example, on the Blokes Hugging Face page, where the model is available, the author provides a detailed description of the Robin 7B model, including instructions on how to download and use it. These instructions do not require any manual settings. Therefore, users can merely download the model normally and run it.

The next essential requirement is the prompt template. The prompt template is necessary for running the model and generating viable outputs. To access the prompt template, users can go to the Text Generation tab and choose from a list of templates. If the desired template is not available, users can try various templates to see what works best.

Users can also delete the template and input their template for generating text.

Model Settings:

Users can find the Model Settings on the Model Tab. The CPU flag means that users can run the model only on a CPU. However, suppose users want to run the model partly on a CPU and partly on a GPU. In that case, they can use the Auto Devices flag. Further, there are other settings that users can explore and use as per their requirements.

Sometimes, users may encounter an error message requesting them to review the code and enable trust remote code. In such cases, users need to select the checkbox in front of the message.

Interface Mode:

Text Generation Web UI offers two modes: instructions models and chat-based models. Users can choose the chat-based model to run a chat-like interface like Samantha. To exit the prompt template and enter the chat-like interface, users can select the Chat option, apply it, and restart the interface. Once in chat mode, Users can have a continuous conversation with the model.

Chat Settings:

In the chat settings menu, users can fine-tune several parameters, such as temperature, top P, and top K, to achieve the desired text output. Users can experiment with different settings until they decide what works best for them. An interesting feature is that the chat history can be uploaded to allow for continuous conversations.

Training Tab:

The Training Tab is where users can fine-tune the model to their liking. Although training can be quite complicated, the author plans to provide a tutorial on this shortly. The Training Tab is an advanced feature, and users are recommended to use it only once they have a good understanding of the basics.

Text generation web ui is a fantastic tool that makes working with nlp models much more manageable. it's easy to install and set up, and there are many models available that you can use to generate text, summarize, or analyze sentiment. with these steps, you should be ready to start using text generation web ui.
:

Text Generation Web UI is a powerful tool that allows users to generate human-like text using machine learning techniques. Users can download and use existing models, modify prompt templates, and experiment with various settings in the chat and model settings menu.

After reading this article, users should be comfortable setting up Text Generation Web UI, downloading models, running them, and customizing various settings. If users encounter any problems, they can reach out to the author on Discord for assistance.

If you enjoyed this article, please consider liking it and subscribing to get notified of new content.