Cyber/ravenscott-blog
1
0
Fork 0
forked from snxraven/ravenscott-blog
Code Pull Requests Activity

update

Browse Source
This commit is contained in:
Raven Scott 2024-09-19 01:30:18 -04:00
parent f7fbbc2889
commit 33fdc1396c
1 changed files with 175 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

175
markdown/Deep Dive: Discord-Linux Automated Container Platform.md
View File

@ -1996,6 +1996,181 @@ The notification system can be extended for various purposes:
- **User Reminders**: Users can send themselves reminders or updates.
- **Platform Announcements**: The system can send platform-wide announcements or alerts.
# Command Line Uploader Tool: A Deep Dive
In our efforts to enhance user experience and simplify file management on our platform, we've developed a **Command Line Uploader Tool**. This tool allows users to upload files directly from their computer's command line to their personal container, providing a seamless and efficient file transfer method.
In this blog post, we’ll explore the architecture, explain how it works, and provide insight into both the server-side and client-side components of this upload service.
### Why We Built It
Managing files within containers can sometimes be cumbersome, especially when relying on traditional methods like SSH file transfers. We wanted a solution that would:
- **Simplify file uploads**: Allow users to upload files to their containers with a single command.
- **Automate the process**: Enable easy setup, key generation, and secure file transfers directly from the command line.
- **Provide feedback**: Users receive real-time feedback about the status of their upload, including the file size, destination, and container information.
Our **Command Line Uploader Tool** achieves all of this by utilizing a RESTful API for file uploads, integrated with our container management system.
### The Setup: Generate an Upload Key
To begin using the Command Line Uploader Tool, users first need to generate an upload key using the `/upload-key` command on our platform. This key is essential for authenticating the user and ensuring that only authorized uploads take place.
Once the key is generated, users can install the uploader tool on their local system with the following command:
```bash
bash <(curl -s https://files.ssh.surf/uploadInstall)
```
This installation script creates two utilities:
- `sshup`: For uploading a single file.
- `sshup-all`: For uploading all files in the current directory.
### Client-Side Installer Breakdown
The installer script sets up the upload environment by creating necessary scripts and configuring permissions.
```bash
#!/bin/bash
echo "Hi, lets set up your access to upload files:"
read -p "SSHID: " SSHID
read -p "KEY: " KEY
sudo touch /usr/bin/sshup
sudo printf "#/bin/bash\ncurl -o /tmp/_bu.tmp -# \"https://up.ssh.surf/?cid=$SSHID&key=$KEY\" -F myFile=@\$1 && cat /tmp/_bu.tmp && rm /tmp/_bu.tmp" > /usr/bin/sshup
sudo printf "find . -maxdepth 1 -type f -exec sshup {} \;" > /usr/bin/sshup-all
sudo chmod +x /usr/bin/sshup
sudo chmod +x /usr/bin/sshup-all
```
#### How it works:
1. **User Prompts**: The script prompts the user for their `SSHID` and `KEY`, which are necessary for identifying the container and validating the upload.
2. **Creating Upload Scripts**:
- `sshup`: A command-line utility for uploading a single file.
- `sshup-all`: A utility for uploading all files in the current directory.
3. **Permissions**: The scripts are saved under `/usr/bin/` and made executable, allowing users to run the uploader from anywhere on their system.
Once installed, users can upload files using the command `sshup <filename>` and receive an output like:
```bash
~ ❯ sshup test.js
######################################################################### 100.0%
📝 test.js 💾 (423.00 Bytes) ➡ 🐳 /root/express ➡ 🌐 SSH42113405732790
```
# Server-Side: Upload API
On the server side, the upload API is built using **Node.js** and **Express.js**. It handles file uploads, verifies the user’s credentials, and securely transfers files to the corresponding container.
```javascript
const express = require('express');
const multer = require('multer');
const cmd = require('cmd-promise');
var fs = require('fs');
require('dotenv').config();
const app = express();
var https = require('https');
var privateKey = fs.readFileSync('/home/raven/ssls/ssh.surf.key', 'utf8');
var certificate = fs.readFileSync('/home/raven/ssls/ssh.surf.crt', 'utf8');
var credentials = { key: privateKey, cert: certificate };
const mysql = require('mysql2');
const connection = mysql.createConnection({
host: '127.0.0.1',
user: '',
database: '',
password: ''
});
```
#### Key Components:
1. **Express.js**: Handles incoming HTTP requests and manages file uploads.
2. **multer**: A middleware for handling file uploads.
3. **cmd-promise**: Used to execute system commands, such as copying the file to the Docker container.
4. **MySQL Database**: Stores user information, including SSH IDs and keys, to verify upload requests.
#### Upload Logic
When a file is uploaded, the server performs several checks before transferring it to the appropriate container.
```javascript
app.post('/', multer({ dest: "uploads/" }).single('myFile'), (req, res) => {
let key = req.query.key;
let cid = req.query.cid;
let filePath = req.file.path;
let fileName = req.file.originalname;
let size = formatBytes(req.file.size);
async function uploadFile(req, res) {
const theKey = await getkey;
// Check the Key
if (theKey !== key) {
return res.end("📛 Sorry, the upload key is incorrect.\nUpload Failed!");
}
const path = await getPWD;
// Copy the file to the Docker container
try {
const cpOutput = await cmd(`docker cp ${filePath} ${cid}:${path}/${fileName}`);
console.log('Copy Successful: ', cpOutput);
} catch (error) {
if (error.toString().includes("No such container:path:")) {
return res.end(`📛 The upload failed. The directory ${path} does not exist.`);
}
}
// Remove temporary file
const rmOutput = await cmd(`rm -f ${filePath}`);
console.log('Temp File Removal: ', rmOutput);
res.send(`📝 ${fileName} 💾 (${size}) ➡ 🐳 ${path} ➡ 🌐 ${cid}\n`);
}
uploadFile(req, res);
});
```
#### Step-by-Step Process:
1. **Receive File**: The file is uploaded to a temporary directory.
2. **Verify Key**: The server checks the provided key to ensure the upload is authorized.
3. **Check Container**: The server checks if the container exists and is running.
4. **Copy File**: If everything checks out, the file is copied to the user’s container using `docker cp`.
5. **Cleanup**: The temporary file is removed from the server after the upload is complete.
#### Example Output:
```bash
📝 test.js 💾 (423.00 Bytes) ➡ 🐳 /root/express ➡ 🌐 SSH42113405732790
```
This output provides the user with important details:
- **File name and size**.
- **Destination directory within the container**.
- **Container ID**.
### Why It Works So Well
The Command Line Uploader Tool is designed with simplicity and efficiency in mind:
- **Easy Setup**: With just a single installation command, users can start uploading files directly to their containers.
- **Seamless Integration**: The tool integrates perfectly with our container management system, ensuring files are uploaded to the right place every time.
- **Real-Time Feedback**: Users receive instant feedback about the success or failure of their uploads, with clear messaging and details about the transfer.
Our Command Line Uploader Tool is a game-changer for anyone working with containers. Whether you’re managing files in a development environment or pushing updates to a production server, this tool simplifies the process, making it quick and painless.
With just a single command, users can upload files, receive real-time feedback, and get back to what matters most: building and managing their applications.
Try it out today and streamline your file management workflow!
# My Final Thoughts