MageSail - Holesail Tunnel Integration for Magento 2

Full documentation: docs/README.md (installation, configuration, NGINX, routing plugins, troubleshooting).

Overview

MageSail is a Magento 2 extension that integrates Holesail (P2P tunneling with HyperDHT / Hyperswarm) so you can expose a local Magento dev site or TCP backends without traditional port forwarding or a central SaaS relay. A shareable key (e.g. hs://…) allows remote clients to reach your local HTTP/HTTPS listener (Website tunnels) or a service port resolved from app/etc/env.php (Service tunnels, e.g. MySQL, Redis).

Version: package 1.0.4 — see Magesail/composer.json and Magesail/etc/module.xml (setup_version).

Key features

• Secure tunneling — Private (authenticated) vs public Holesail modes, mapped to Secure Tunnel Yes/No in Admin.
• Admin lifecycle — Create, start, stop, and remove tunnels under MageSail → Holesail Tunnel → Manage Tunnel; status polling and per-tunnel log tail via AJAX.
• Website tunnels — Type: Website under Add tunnel: each tunnel gets a generated store code (e.g. mgtun_…) on the website / store group you choose, with base URLs you supply. Remove tears down the store and NGINX map line; Stop only stops the Node process (with safety rules if that store is the website default on delete).
• Service tunnels — Type: Service: TCP forward to SSH on 127.0.0.1 (presets 22 / 2223 only if a TCP probe finds a listener; custom port always) and/or endpoints discovered from env via DeploymentConfig. No store view, no NGINX map row; Remove drops the registry row and PID/key only. See docs/admin-ui.md.
• NGINX global map — Optional append/remove of map $http_host $MAGE_RUN_CODE / $MAGE_RUN_TYPE lines for website tunnels only, plus nginx -t and reload (or wrapper script). Admin screen: MageSail → Holesail Tunnel → NGINX Global Map.
• Storefront / admin routing — For website tunnels, plugins detect HTTP traffic via header X-MageSail-Store-Code, query magesail_store, ___store, or hostname vs persisted base URLs (HTTP_HOST plus X-Forwarded-Host / HTTP_X_FORWARDED_HOST when the client host differs from what PHP sees); adjust MAGE_RUN_*, backend front name resolution, generated base URLs, and skip redirect-to-base where appropriate. Service tunnels do not participate in Magento HTTP routing.
• Tunnel storefront content — Each new website tunnel store copies CMS home / no-route / no-cookies / front paths and theme from that website’s default store view, and mirrors cms_page_store / cms_block_store so pages and blocks assigned to the default storefront appear on the tunnel (see docs/tunnel-store-and-multistore.md).
• PID and logs — Per tunnel: var/magesail_tunnels/{id}.pid, {id}.key, var/log/magesail_tunnel_{id}.log; optional DB rows in magesail_logs.
• Cron — Daily job checks each tunnel’s PID file. If the process is dead and Auto-restart is No, it removes that tunnel — same end state as Remove: for website tunnels that means store (when allowed), NGINX map line, registry, PID/key; for service tunnels, registry and PID/key only. If Auto-restart is Yes, it clears stale PID/key and starts Node again (saved port and secure mode).

Good for client demos, remote QA, and team access without deploying to staging.

Private vs public keys (Holesail)

Holesail supports private (default, authenticated) and public (open) modes. In Admin, Secure Tunnel = Yes means private mode, No means public.

Private mode

• A seed derives a keypair; the server keeps authentication material; peers need the shared secret to connect.
• Treat shared seeds like SSH private keys: only share with trusted parties.

Public mode

• A discoverable public address; anyone with the key can reach the tunnel endpoint.
• Protect sensitive data with Magento’s own auth (customer login, admin ACL, etc.).

Private mode is recommended for most dev sharing.

Requirements

• Magento 2: 2.4.x (see Composer constraints in Magesail/composer.json).
• PHP: 8.1, 8.2, 8.3, or 8.4.
• Node.js: 18+ for start-server.js.
• npm: Install dependencies under Magesail/scripts/.
• OS: Linux/macOS typical; Windows untested (WSL may work). Shell tools: ps, kill, nohup.
• Composer: For packaged installs.

Holesail and npm dependencies are pulled when you run npm install in the scripts directory (see docs/scripts.md).

Continuous integration

• Tier 1 (default): stub-based PHPUnit, PHPStan, and PHPCS — see .github/workflows/ci.yml.
• Tier 2 (optional): full Magento CE in Docker, real nginx nginx -t, real Node/npm + holesail — runs when the COMPOSER_AUTH secret is set. See docs/ci-full-stack.md.

Installation

1. Install the module

MageSail is not on Packagist (no public composer require from a registry). Install by cloning the repository into app/code, or by adding a path / VCS repository in your Magento composer.json and requiring magesail/magesail from that source.

Git clone under app/code (recommended):

cd <magento-root>/app/code/
git clone https://git.ssh.surf/snxraven/MageSail.git MageSail

Module root: app/code/MageSail/Magesail/.

Set MageSail → Holesail Tunnel → Configuration → Scripts directory to the absolute path of the folder that contains start-server.js, e.g. <magento-root>/app/code/MageSail/Magesail/scripts.

Composer from a clone or VCS: add a repositories entry (path to the clone, or {"type": "vcs", "url": "https://git.ssh.surf/snxraven/MageSail.git"}), then composer require magesail/magesail:@dev (or a tagged version). Scripts directory example when installed as a Composer package: <magento-root>/vendor/magesail/magesail/scripts.

2. Enable the module

php bin/magento module:enable MageSail_Magesail
php bin/magento setup:upgrade
php bin/magento setup:di:compile
php bin/magento cache:flush

3. Verify

• php bin/magento module:status MageSail_Magesail → enabled.
• Admin: MageSail → Holesail Tunnel (Manage Tunnel, NGINX Global Map, Configuration).
• Database: table magesail_logs.

4. Node dependencies (one-time)

cd <magento-root>/app/code/MageSail/Magesail/scripts
# or: cd <magento-root>/vendor/magesail/magesail/scripts
npm install

Configuration

All settings: MageSail → Holesail Tunnel → Configuration (standard Magento config UI; not listed under Stores → Configuration).

Tunnel settings

Field	Default	Description
Secure Tunnel	Yes	Private vs public Holesail mode.
Scripts directory	(empty)	Absolute path to start-server.js folder. Empty = default under app/code/MageSail/Magesail/scripts from Magento root.
Node.js binary	(empty)	Empty = node on PHP’s PATH. Set a full path if Node is not on PATH (nvm, fnm, etc.).
Auto-restart on crash	No	If Yes, the monitor cron restarts each tunnel whose process died (saved port and secure mode). If No, the monitor tears down tunnels with a stale PID (for website: store + map + registry; for service: registry + PID/key), not only the PID files.
Monitor cron schedule	0 0 * * *	Magento cron expression for the tunnel health job (default: daily at midnight). Edit in Configuration → Settings.

Scoped globally / per website / per store view. After changes: php bin/magento cache:flush config.

NGINX configuration

Used when you manage MAGE_RUN_CODE / MAGE_RUN_TYPE via a shared global map file (e.g. Ansible). Fields include Global map file path, nginx binary path, Temporary PID file (for isolated nginx -t), Reload wrapper script (when PHP cannot nginx -s reload), map variable names, MAGE_RUN_TYPE value, and Disable base URL redirect for tunnel. See docs/configuration.md and docs/nginx-global-map.md.

Advanced (app/etc/config.php): you can set magesail/settings/* and magesail/nginx/* keys to match system.xml paths.

Usage

Admin menu

• MageSail → Holesail Tunnel → Manage Tunnel — start/stop, base URLs, key display, log snippet.
• MageSail → Holesail Tunnel → NGINX Global Map — edit/validate/reload the configured map file (requires config ACL).
• MageSail → Holesail Tunnel → Configuration — tunnel defaults, scripts path, Node binary, monitor cron schedule, auto-restart, NGINX paths (requires config ACL; opens standard system config for section magesail).

Starting a tunnel

1. Open Manage Tunnel.
2. Under Add tunnel, set Type:
   • Website (default) — Choose Store group, Local port (preset 443, 80, 8080, or Custom), and Tunnel hostname (unsecure) (required), e.g. dev.myshop.local — no http://; Magento stores http://…/ automatically. It must match the host in the browser / Holesail client vhost. Optionally set Tunnel hostname (secure), Use secure URLs on storefront and in admin, and Holesail secure tunnel. Full http:// / https:// URLs still work if pasted.
   • Service — Pick a Service from the dropdown (endpoints discovered from app/etc/env.php); optional label; Holesail secure tunnel as needed. Port and target host come from env (no store group or tunnel hostnames). If nothing is discoverable, submit is disabled.
3. Click Create and start tunnel.

For Website, the module creates a new store view with a generated code (e.g. mgtun_…), sets scoped base URLs, copies CMS + theme defaults from the website default store and mirrors CMS page/block store assignments, persists var/magesail_tunnels.json, may update the NGINX map, then starts Node. For Service, it only writes the registry row (plus service_key, target host, local_port) and starts Node — no store, no map line.

PID and key live under var/magesail_tunnels/{id}.pid and .key. The UI polls magesail/tunnel/status (optionally with tunnel_id) until the key appears.

For an existing stopped tunnel, use Start on that row (no re-provision).

If the map was updated but nginx could not reload (common for the PHP user), follow the on-screen hint or docs/nginx-global-map.md.

Proxy / client routing

For website tunnels, so Magento resolves the tunnel store, your proxy or browser can use the tunnel’s store code from Admin (example: mgtun_abc12345):

• Header: X-MageSail-Store-Code — value = that store code
• Query: magesail_store= + store code
• Or native: ___store= + store code

If the request host matches the provisioned tunnel base URL host, detection can work without those hints. When PHP sees an internal Host (e.g. 127.0.0.1) but the user browses a tunnel hostname, configure your proxy to send X-Forwarded-Host with that hostname. Details: docs/request-routing-and-plugins.md.

Sharing access and making connections

Recommended: Holesail Browser (holesail-browser) — a third-party graphical client that speaks Holesail natively. For Magento storefront and admin over a tunnel it is usually the simplest option: Virtual Host–style mapping from your tunnel hostname to 127.0.0.1:<local port> (the port from Manage Tunnel), plus a Use TLS (secure connection) toggle for the hop to localhost when Magento (or a proxy in front of it) serves HTTPS on that port—without installing nginx, Apache, or NGINX Proxy Manager on every workstation.

• When Local port is 443 (or anything that expects HTTPS on localhost), enable Use TLS for that Virtual Host in Holesail Browser. For plain HTTP on port 80 or 8080, leave TLS off for the localhost hop. That TLS setting is only between the Holesail client and your local listener; it does not replace Base URL / Base URL (Secure) on the tunnel store—they must still match what users type in the browser (scheme and host).
• CLI: use the holesail package from npm with the same hs://… or key string shown in Manage Tunnel when the process is running. Example with a local port override: holesail hs://… --port 1234 then curl http://127.0.0.1:1234/.
• Advanced: reverse-proxy examples (nginx, Apache, NGINX Proxy Manager, cURL smoke tests) and security notes live in Admin under MageSail → Holesail Tunnel → How-To, and in this repo’s docs/ (e.g. docs/request-routing-and-plugins.md).

Also:

• Copy the hs://… (or key string) for whichever client you use.
• You may add the shop hostname to /etc/hosts pointing at 127.0.0.1 when using a local listener.
• TLS to localhost often uses dev certificates; browsers may show warnings until you trust the cert or use a dev CA.

Stopping or removing a tunnel

• Stop — Kills the Node process and removes that tunnel’s PID/key files. The registry row stays so you can Start again (website: store and NGINX map line also stay).
• Remove — Website: stops the process, removes the NGINX map line, deletes the store view when allowed, and removes the tunnel from the registry. Service: removes the registry row and PID/key only (no store, no map).

Manual: kill $(cat var/magesail_tunnels/<id>.pid) and remove stale *.pid / *.key under that directory if needed.

Monitoring

• UI — running state, PID, key.
• File log — tail -f var/log/magesail_tunnel_<id>.log (per tunnel)
• Database — magesail_logs for start/stop/cron messages.

Cron

• Schedule: configurable under Configuration → Settings → Monitor cron schedule (stored as crontab/default/jobs/magesail_monitor/schedule/cron_expr; default 0 0 * * *). Magesail/etc/crontab.xml points at that path; default is also in Magesail/etc/config.xml.
• Behavior: For each tunnel, if its PID file exists but the process is dead: with Auto-restart on crash = No, the job runs full teardown for that tunnel (same as Remove — website: store/map/registry when allowed; service: registry + PID/key). With Auto-restart = Yes, it removes stale PID/key, then starts Node using that tunnel’s saved local port and Holesail secure flag. A warning is logged either way.

Troubleshooting

Short table — expanded in docs/troubleshooting.md.

Issue	Suggestion
Start fails / “Command failed”	Node.js binary in Configuration if node is not on PHP’s PATH; run npm install in Scripts directory.
“Tunnel hostname is required”	For Website type, fill unsecure hostname before start.
No services in Service dropdown	env.php has no discoverable TCP endpoints (or incomplete host/port); add or fix config.
Invalid / unknown service on create	Env changed after page load; refresh Admin and pick a current service.
Invalid port	Use a local port between 1 and 65535 (presets or custom). If two tunnels share a port, the second Start may fail unless your OS/stack allows multiple binds.
Stale PID	Delete var/magesail_tunnels/<id>.pid manually or use Start after a clean stop. If you wait for daily cron with Auto-restart = No, the module may remove the whole tunnel (not only the PID file). Check for zombie start-server.js.
301 to production URL	Tunnel plugins + NGINX map + reload; see docs/tunnel-store-and-multistore.md.
504 / Gateway Timeout on tunnel	Often slow or failing catalog image resize: ensure pub/media/catalog/product/cache exists and is writable by the PHP user; check var/log/exception.log. Increase proxy fastcgi_read_timeout if first-hit generation is slow.
Too few arguments / DI errors after upgrade	Run php bin/magento setup:di:compile and flush caches after pulling MageSail changes that change constructors.
Reload / map permission errors	Map file writable by PHP user; wrapper + sudoers — Magesail/scripts/README-NGINX-RELOAD.md.
start-server.js / node_modules errors on start	Scripts directory must contain start-server.js and a node_modules folder after npm install.

Development / CI

From the repository root (not only inside Magesail/):

composer install
php vendor/bin/phpunit -c phpunit.xml.dist
php vendor/bin/phpstan analyse --memory-limit=512M
php vendor/bin/phpcs --standard=phpcs.xml

Optional code coverage (Xdebug or PCOV): XDEBUG_MODE=coverage php vendor/bin/phpunit -c phpunit.xml.dist --coverage-text --coverage-filter Magesail

PHPUnit loads the module via Composer PSR-4 and uses minimal Magento (and PSR) stubs under Magesail/Test/Stubs/ so tests run without repo.magento.com. Test/bootstrap.php defines BP (Magento root for path resolution in tests). What still requires a full Magento app is listed in docs/phpunit-tier4-deferred.md; optional integration-test placement is described in docs/magento-integration-tests.md.

License

This project is licensed under the GNU General Public License v3.0 or later — see LICENSE and Magesail/composer.json.