HuggingFace Deploy Instructions

huggingface-config/Dockerfile

@@ -0,0 +1,38 @@
+FROM ubuntu:latest
+
+RUN apt update
+
+RUN DEBIAN_FRONTEND=noninteractive  apt install curl sudo -y
+
+RUN curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash - 
+
+RUN DEBIAN_FRONTEND=noninteractive  apt install nodejs -y
+
+RUN npm i pm2 -g
+
+RUN mkdir -p /code/.pm2
+
+RUN mkdir -p /.pm2
+
+RUN chmod 0777 /code/.pm2 
+
+RUN chmod 0777 /.pm2
+
+RUN DEBIAN_FRONTEND=noninteractive apt install wget python3 python3-pip -y
+
+WORKDIR /code
+
+RUN pip install --no-cache-dir llama-cpp-python[server]
+
+COPY . .
+
+RUN npm i
+
+ENV HOST localhost
+ENV PORT 7860
+ENV MODEL=/code/ggml-vic7b-q4_0.bin
+ENV CACHE=1
+ENV PM2_HOME=/code/.pm2
+RUN wget https://huggingface.co/eachadea/ggml-vicuna-7b-1.1/resolve/main/ggml-vic7b-q4_0.bin
+
+CMD /bin/bash /code/start.sh

huggingface-config/README.md

@@ -0,0 +1,13 @@
+# How to Deploy on Hugging Face
+
+1) Put all of the bots files within your HuggingFace Git Repo.
+
+2) Move the files Dockerfile - start.sh - startServices.json. (Overwrite the docker file found in the root of the project)
+
+3) Edit Docker file as you need, the dockerfile is set to automatically download Vicuna 1.1 7B
+
+4) Push the changes
+
+You should then see the bot being built and deployed on HuggingFace
+
+pm2 log will run automatically so you can see frontend and backend logs.

huggingface-config/startServices.json

@@ -0,0 +1,14 @@
+{
+    "apps": [
+        {
+            "name": "AI-API",
+            "script": "node /code/llama.js",
+            "args" : ""
+        },
+        {
+            "name": "DiscordBot",
+            "script": "python3 -m llama_cpp.server",
+            "args" : ""
+        }
+    ]
+}