MrTuxedo/llama-cpp-python-djs-bot
1
0
Fork 0
forked from snxraven/llama-cpp-python-djs-bot
Code Pull Requests Activity
https://abetlen.github.io/llama-cpp-python/
34 Commits 1 Branch 0 Tags 195 KiB
JavaScript 88%
Shell 7.1%
Dockerfile 4.9%
5a56251e20
Go to file
Raven Scott 5a56251e20 Adding cache to server env
2023-04-16 13:22:17 +02:00
assets dockerize 2023-04-12 16:17:18 +02:00
server dockerize 2023-04-12 16:17:18 +02:00
.gitignore first 2023-04-09 18:07:32 +02:00
default.env Adding cache to server env 2023-04-16 13:22:17 +02:00
docker-compose.yml dockerize update 2023-04-12 16:19:27 +02:00
Dockerfile dockerize 2023-04-12 16:17:18 +02:00
llamabot.js dockerize 2023-04-12 16:17:18 +02:00
package.json dockerize 2023-04-12 16:17:18 +02:00
README.md update readme 2023-04-12 18:37:02 +02:00

README.md

llama-cpp-python-djs-bot

THIS CODE IS MEANT TO BE SELF HOSTED USING THE LIBRARY: https://abetlen.github.io/llama-cpp-python/

Description

This code is for a Discord bot that uses OpenAI's GPT-3 language model (self hosted at home) to generate responses to user messages. It listens for messages in two specified Discord channels, and when a user sends a message, it appends it to the conversation history and sends it to the GPT-3 API to generate a response. The response is then sent back to the user in the same channel. The bot uses the Node.js discord.js library to interact with the Discord API and the node-fetch library to make HTTP requests to the GPT-3 API.

Here is a summary of the main parts of the code:

Import required modules and set environment variables using dotenv.

Create a new Client instance and set the intents and partials.

Define two channel IDs that the bot will listen to.

Create a Map to store ongoing conversations with users.

Define functions to update the bot's presence status, check if any conversation is busy, and set a conversation as busy or not busy.

Listen for the ready event and update the bot's presence status.

Listen for the messageCreate event and respond to messages that are sent in the specified channels.

When a message is received, check if any conversation is busy. If so, delete the message and send a busy response to the user.

If no conversation is busy, append the user message to the conversation history and send it to the GPT-3 API to generate a response.

If the response is not empty, send it back to the user in the same channel. If it is empty, send a reset message and delete the conversation history for that user.

Define a generateResponse function that sends a request to the GPT-3 API to generate a response. If the request times out or an error occurs, handle it accordingly.

Call the generateResponse function within the messageCreate event listener function.

demo

Backend REQUIIRED

The HTTP Server from https://abetlen.github.io/llama-cpp-python/ is required to use this bot.

llama-cpp-python offers a web server which aims to act as a drop-in replacement for the OpenAI API. This allows you to use llama.cpp compatible models with any OpenAI compatible client (language libraries, services, etc).

To install the server package and get started:

pip install llama-cpp-python[server]

export MODEL=./models/your_model.py

python3 -m llama_cpp.server

Navigate to http://localhost:8000/docs to see the OpenAPI documentation.

Static Usage

  1. Use npm i

  2. Create a .env file cp default.env .env

  3. Edit .env for your needs

  4. Go into https://discord.com/developers/applications and enable Privileged Intents.

  5. Run the bot node llamabot.js

Docker Compose

This will automatically configure the API for you as well as the bot in two seperate containers within a stack.

  1. git clone https://git.ssh.surf/snxraven/llama-cpp-python-djs-bot.git

  2. cp default.env .env

  3. Set DATA_DIR in .env to the exact location of your model files.

  4. Edit docker-compose.yaml MODEL to ensure the correct model bin is set

  5. docker compose up -d

Want to make this better? Issue a pull request!