HuggingFace Deploy Instructions

This commit is contained in:
Raven Scott 2023-05-06 01:41:03 +02:00 committed by MrTuxedo
parent 6531b633f7
commit c28bf9c022
4 changed files with 65 additions and 0 deletions

View File

@ -0,0 +1,38 @@
FROM ubuntu:latest
RUN apt update
RUN DEBIAN_FRONTEND=noninteractive apt install curl sudo -y
RUN curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
RUN DEBIAN_FRONTEND=noninteractive apt install nodejs -y
RUN npm i pm2 -g
RUN mkdir -p /code/.pm2
RUN mkdir -p /.pm2
RUN chmod 0777 /code/.pm2
RUN chmod 0777 /.pm2
RUN DEBIAN_FRONTEND=noninteractive apt install wget python3 python3-pip -y
WORKDIR /code
RUN pip install --no-cache-dir llama-cpp-python[server]
COPY . .
RUN npm i
ENV HOST localhost
ENV PORT 7860
ENV MODEL=/code/ggml-vic7b-q4_0.bin
ENV CACHE=1
ENV PM2_HOME=/code/.pm2
RUN wget https://huggingface.co/eachadea/ggml-vicuna-7b-1.1/resolve/main/ggml-vic7b-q4_0.bin
CMD /bin/bash /code/start.sh

View File

@ -0,0 +1,13 @@
# How to Deploy on Hugging Face
1) Put all of the bots files within your HuggingFace Git Repo.
2) Move the files Dockerfile - start.sh - startServices.json. (Overwrite the docker file found in the root of the project)
3) Edit Docker file as you need, the dockerfile is set to automatically download Vicuna 1.1 7B
4) Push the changes
You should then see the bot being built and deployed on HuggingFace
pm2 log will run automatically so you can see frontend and backend logs.

View File

View File

@ -0,0 +1,14 @@
{
"apps": [
{
"name": "AI-API",
"script": "node /code/llama.js",
"args" : ""
},
{
"name": "DiscordBot",
"script": "python3 -m llama_cpp.server",
"args" : ""
}
]
}