Go to file
2024-08-07 02:16:30 -04:00
backend-server Update max length according to new NCTX configured 2024-08-07 02:16:30 -04:00
bot edit guild, name headers for resets 2024-08-06 20:08:24 -04:00
llama-cpp-python first commit 2024-08-03 02:19:51 -04:00
README.md Update Readme 2024-08-04 11:58:33 -04:00

RayAI and Backend Server

This project contains the RayAI bot source code as well as the backend server that supports its operations.

Note: A llama-cpp-python OpenAI Emulation Server is required alongside the backend server. The server in this code is configured to run on 127.0.0.1:8002.

Prerequisites

  • Node.js installed
  • Python installed
  • llama-cpp-python server

Running the llama-cpp-python Server

  1. Install llama-cpp-python with server capabilities:

    pip install llama-cpp-python[server]
    
  2. Start the server:

    python3 -m llama_cpp.server --model <model_path>
    

Cloning the Repository

  1. Clone the repository:

    git clone https://git.ssh.surf/snxraven/rayai
    cd rayai
    

Backend Server Setup

  1. Navigate to the backend server directory and install dependencies:

    cd backend-server
    npm install
    
  2. Create a .env file in the backend-server directory with the following environment variables:

    PROMPT=<initial-prompt>
    ABUSE_KEY=<abuse-ipdb-api-key>
    MAX_CONTENT_LENGTH=2000 (optional)
    API_KEY=<my-mc-api-key>
    PATH_KEY=<my-mc-path-key>
    
  3. Start the backend server:

    node backend-server.js
    

Discord Bot Setup

  1. Navigate to the bot directory and install dependencies:

    cd ../bot
    npm install
    
  2. Create a .env file in the bot directory with the following environment variables:

    THE_TOKEN=<your-discord-bot-token>
    CHANNEL_IDS=<comma-separated-list-of-channel-ids>
    ROOT_IP=<root-ip-address>
    ROOT_PORT=<root-port-number>
    API_PATH=<api-path>
    MAX_CONTENT_LENGTH=8000 (optional, default is 8000)
    MAX_TOKENS=<max-tokens>
    REPEAT_PENALTY=<repeat-penalty>
    OVERFLOW_DELAY=3 (optional, delay in seconds between message chunks)
    
  3. Start the Discord bot:

    node discord-bot.js
    

Summary

  1. Run the llama-cpp-python server to emulate the OpenAI API.
  2. Set up and run the backend server to handle API requests, conversation history, and additional functionalities.
  3. Set up and run the Discord bot to interact with users and utilize the backend server for processing.

Ensure all environment variables are correctly configured, dependencies are installed, and both servers are running to enable full functionality of RayAI and its backend server.