sandbox-web-ide/README.md

# GitWit Sandbox 📦🪄

<img width="1799" alt="GitWit Sandbox Demo" src="https://pouch.jumpshare.com/preview/js7qIRq5agZAJBvcAls-5y3b5P_EW-sBIl6k6hntpHC1mu2NayXbcs7_zTNNo1zICNLf34jx8ljK0mbpgXRQd95onG16D6GWKsLmV8kzcUI" />

Sandbox is an open-source cloud-based code editing environment with custom AI code generation, live preview, real-time collaboration and AI chat.

For the latest updates, join our Discord server: [discord.gitwit.dev](https://discord.gitwit.dev/).

## Running Locally

Notes:

- Double check that whatever you change "SUPERDUPERSECRET" to, it's the same in all config files.
- Right now we are loading project templates from a custom Cloudflare bucket which isn't covered in this guide, but that be updated/fixed very soon.

### 0. Requirements

The application uses NodeJS for the backend, NextJS for the frontend and Cloudflare workers for additional backend tasks.

Needed accounts to set up:

- [Clerk](https://clerk.com/): Used for user authentication.
- [Liveblocks](https://liveblocks.io/): Used for collaborative editing.
- [E2B](https://e2b.dev/): Used for the terminals and live preview.
- [Cloudflare](https://www.cloudflare.com/): Used for relational data storage (D2) and file storage (R2).

A quick overview of the tech before we start: The deployment uses a **NextJS** app for the frontend and an **ExpressJS** server on the backend. Presumably that's because NextJS integrates well with Clerk middleware but not with Socket.io.

### 1. Initial setup

No surprise in the first step:

```bash
git clone https://github.com/jamesmurdza/sandbox
cd sandbox
```

Run `npm install` in:

```
/frontend
/backend/database
/backend/storage
/backend/server
/backend/ai
```

### 2. Adding Clerk

Setup the Clerk account.
Get the API keys from Clerk.

Update `/frontend/.env`:

```
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY='🔑'
CLERK_SECRET_KEY='🔑'
```

### 3. Deploying the storage bucket

Go to Cloudflare.
Create and name an R2 storage bucket in the control panel.
Copy the account ID of one domain.

Update `/backend/storage/src/wrangler.toml`:

```
account_id = '🔑'
bucket_name = '🔑'
key = 'SUPERDUPERSECRET'
```

In the `/backend/storage/src` directory:

```
npx wrangler deploy
```

### 4. Deploying the database

Create a database:

```
npx wrangler d1 create sandbox-database
```

Use the output for the next setp.

Update `/backend/database/src/wrangler.toml`:

```
database_name = '🔑'
database_id = '🔑'
KEY = 'SUPERDUPERSECRET'
STORAGE_WORKER_URL = 'https://storage.🍎.workers.dev'
```

In the `/backend/database/src` directory:

```
npx wrangler deploy
```

### 5. Applying the database schema

Delete the `/backend/database/drizzle/meta` directory.

In the `/backend/database/` directory:

```
npm run generate
npx wrangler d1 execute sandbox-database --remote --file=./drizzle/0000_🍏_🍐.sql
```

### 6. Configuring the server

Update `/backend/server/.env`:

```
DATABASE_WORKER_URL='https://database.🍎.workers.dev'
STORAGE_WORKER_URL='https://storage.🍎.workers.dev'
WORKERS_KEY='SUPERDUPERSECRET'
```

### 7. Adding Liveblocks

Setup the Liveblocks account.

Update `/frontend/.env`:

```
NEXT_PUBLIC_LIVEBLOCKS_PUBLIC_KEY='🔑'
LIVEBLOCKS_SECRET_KEY='🔑'
```

### 8. Adding E2B

Setup the E2B account.

Update `/backend/server/.env`:

```
E2B_API_KEY='🔑'
```

### 9. Adding AI code generation

In the `/backend/ai` directory:

```
npx wrangler deploy
```

Update `/backend/server/.env`:

```
AI_WORKER_URL='https://ai.🍎.workers.dev'
```

### 10. Configuring the frontend

Update `/frontend/.env`:

```
NEXT_PUBLIC_DATABASE_WORKER_URL='https://database.🍎.workers.dev'
NEXT_PUBLIC_STORAGE_WORKER_URL='https://storage.🍎.workers.dev'
NEXT_PUBLIC_WORKERS_KEY='SUPERDUPERSECRET'
```

### 11. Running the IDE

Run `npm run dev` simultaneously in:

```
/frontend
/backend/server
```

## Setting up Deployments

The steps above do not include steps to setup [Dokku](https://github.com/dokku/dokku), which is required for deployments.

**Note:** This is completely optional to set up if you just want to run GitWit Sandbox.

Setting up deployments first requires a separate domain (such as gitwit.app, which we use).

We then deploy Dokku on a separate server, according to this guide: https://dev.to/jamesmurdza/host-your-own-paas-platform-as-a-service-on-amazon-web-services-3f0d

The Sandbox platform connects to the Dokku server via SSH, using SSH keys specifically generated for this connection. The SSH key is stored on the Sandbox server, and the following environment variables are set in /backend/server/.env:

```bash
DOKKU_HOST=
DOKKU_USERNAME=
DOKKU_KEY=
```

## Creating Custom Templates

We're working on a process whereby anyone can contribute a custom template that others can use in the Sandbox environment. The process includes:

- Creating a [custom E2B Sandbox](https://e2b.dev/docs/sandbox-template) including the template files and dependencies
- Creating a file to specify the run command (e.g. "npm run dev")
- Testing the template with Dokku for deployment

Please reach out to us [on Discord](https://discord.gitwit.dev/) if you're interested in contributing.

## Contributing

Thanks for your interest in contributing! Review this section before submitting your first pull request. If you need any help, feel free contact us [on Discord](https://discord.gitwit.dev/).

### Structure

```
frontend/
├── app
├── assets
├── components
└── lib
backend/
├── server
├── database/
│   ├── src
│   └── drizzle
├── storage
└── ai
```

| Path               | Description                                                  |
| ------------------ | ------------------------------------------------------------ |
| `frontend`         | The Next.js application for the frontend.                    |
| `backend/server`   | The Express websocket server.                                |
| `backend/database` | API for interfacing with the D1 database (SQLite).           |
| `backend/storage`  | API for interfacing with R2 storage. Service-bound to `/backend/database`. |
| `backend/ai`       | API for making requests to Workers AI .                      |

### Development

#### Fork this repo

You can fork this repo by clicking the fork button in the top right corner of this page.

#### Clone repository

```bash
git clone https://github.com/<your-username>/sandbox.git
cd sandbox
```

#### Create a new branch

```bash
git checkout -b my-new-branch
```

### Commit convention

Before you create a Pull Request, please check that you use the [Conventional Commits format](https://www.conventionalcommits.org/en/v1.0.0/)

It should be in the form `category(scope or module): message` in your commit message from the following categories:

- `feat / feature`: all changes that introduce completely new code or new
  features

- `fix`: changes that fix a bug (ideally you will additionally reference an
  issue if present)

- `refactor`: any code related change that is not a fix nor a feature

- `docs`: changing existing or creating new documentation (i.e. README, docs for
  usage of a lib or cli usage)

- `chore`: all changes to the repository that do not fit into any of the above
  categories

  e.g. `feat(editor): improve tab switching speed`

---

## Tech stack

### Frontend

- [Next.js](https://nextjs.org/)
- [TailwindCSS](https://tailwindcss.com/)
- [Shadcn UI](https://ui.shadcn.com/)
- [Clerk](https://clerk.com/)
- [Monaco](https://microsoft.github.io/monaco-editor/)
- [Liveblocks](https://liveblocks.io/)

### Backend

- [Cloudflare Workers](https://developers.cloudflare.com/workers/)
  - [D1 database](https://developers.cloudflare.com/d1/)
  - [R2 storage](https://developers.cloudflare.com/r2/)
  - [Workers AI](https://developers.cloudflare.com/workers-ai/)
- [Express](https://expressjs.com/)
- [Socket.io](https://socket.io/)
- [Drizzle ORM](https://orm.drizzle.team/)
- [E2B](https://e2b.dev/)