ravenscott-blog/markdown/MageSail: P2P Tunnels for the Commerce World.md

Real Peer-To-Peer for Magento At Last!

Repo: https://git.ssh.surf/snxraven/MageSail

You need someone to see your local Magento: a client, QA, or a teammate three time zones away. Staging is slow. Traditional port forwarding is fragile. Hosted tunnel products add another subscription and another place secrets live. MageSail is a Magento 2 module that closes that gap by pairing your store with Holesail peer to peer networking. You create tunnels from Admin, get an hs:// key, and optionally wire NGINX so each tunnel hostname lands in the right store view.

If you build or operate Magento for agencies, extension shops, or internal commerce teams, this post is the project tour: what problem it solves, what ships in the box, and how requests get from a browser to the correct scope without a redirect to production.

MageSail Demo

Why MageSail sits on Holesail

Disclosure: I help develop Holesail, so I am hardly a neutral observer. I also work with Magento stacks a lot at work, and that is where MageSail came from: I wanted Admin side tunnel controls that respect multistore rules while staying on the Holesail model I already trust when working within my own development environment at work.

Relay based “share localhost” tools are a great fit for plenty of teams. They give you dashboards, stable URLs, and someone else runs the hard parts. Holesail is peer to peer first: both sides join with a shared connection key, the same hs:// style string MageSail surfaces in Manage Tunnel. That key is how peers find each other (only someone with the secret should be able to), how they authenticate without usernames or passwords, and how the pipe stays encrypted end to end. Once the session is up, Holesail carries TCP and UDP like the two machines were on one LAN. There is no central coordination server, no relay path, and no metadata collection in the middle: traffic goes directly between peers.

Under the hood Holesail is open source, built on Pear, and uses a distributed hash table with Kademlia style routing. The DHT is the practical trick that lets two laptops behind NAT and flaky home IPs still rendezvous: you hand someone a capability (the key), the swarm helps you meet, then the conversation is directly you and them.

Encryption is always on (Holesail leans on libsodium). There are no accounts, no identity database, no connection logs, and no usage tracking baked into that model. It is meant to stay boring and small from a trust perspective.

If you want to dig in: holesail.io, holesail.io/features, docs.holesail.io.

That lines up nicely with commerce workflows:

• You and a client: hand them a key, they use Holesail Browser or the CLI, and they land on your dev store without you both adopting another vendor just for the preview.
• You and another dev: quick pairing for bugs, branches, or extensions when spinning full staging for every question is overkill.
• A smaller middle: when you want fewer hops between laptops, that is the P2P trade in a nutshell.

MageSail does not replace any of that. It plugs Holesail into Magento: tunnels from Admin, store views and optional NGINX maps for Website rows, and Service tunnels for env.php ports with the same hs:// flow.

At a glance

• Tunnels from Admin: create, start, stop, and remove tunnels under MageSail → Holesail Tunnel → Manage Tunnel; keys and logs stay visible to authorized admins.
• Website tunnels: each tunnel can provision its own store view and base URLs so multistore rules stay honest over a tunnel host.
• Optional NGINX map: append host to MAGE_RUN_CODE lines in a shared map file so PHP boots in the right scope before Magento runs its usual pipeline.
• Service tunnels: expose TCP services discovered from app/etc/env.php (MySQL, Redis, AMQP, and more) as an hs:// URL, no storefront.
• Recommended client: Holesail Browser for Virtual Hosts on Website tunnels and Service Tunnels on Service tunnels (paste the same hs:// key from Manage Tunnel).

Why Magento makes tunneling harder than a static site

Magento is multistore by design: websites, store groups, and store views, often each with its own base URL. The stack tries to keep URLs canonical. In production that is a feature. On a tunnel you are almost always “wrong” from core’s point of view: the browser shows one host, config still names another, and the redirect to base URL path can send every click to a production domain.

A dumb TCP tunnel does not fix that. You need Magento to resolve store scope from the tunnel (hostname, header, query, or early NGINX MAGE_RUN_CODE), and you need plugins that relax base URL checks when tunnel traffic is detected. MageSail implements that pipeline for website tunnels. Service tunnels are a separate path: MageSail forwards to TCP backends from DeploymentConfig and exposes them with an hs:// key, no store view and no storefront routing.

What you get in the product

Manage Tunnel is the control center: tunnel list, lifecycle actions, log tail, copy helpers for store codes and keys. Add tunnel chooses Website or Service, then collects the fields each type needs. How To documents client setups, with Holesail Browser (Virtual Hosts for HTTP stores, Service Tunnels for hs:// TCP like MySQL) plus NGINX, Apache, NGINX Proxy Manager, and cURL for proxy based stacks. NGINX Global Map edits the map file your vhost includes, validates with nginx -t, and reloads NGINX when the server allows. Configuration stores script paths, Node binary, secure defaults, auto restart, cron schedule, and map paths.

Giving it a try

Treat MageSail like any other module you drop under app/code. There is no Packagist install; you clone the repo, run the usual Magento CLI sequence, point Admin at the Node scripts folder, and open Manage Tunnel.

Clone into app/code

From your Magento project, under app/code, run git clone https://git.ssh.surf/snxraven/MageSail.git MageSail.

Module root: app/code/MageSail/Magesail/.

Magento CLI

Use the Magento root (where bin/magento sits beside app/ and vendor/).

• php bin/magento module:enable MageSail_Magesail
• php bin/magento setup:upgrade — declarative schema and patches, including the magesail_logs table
• php bin/magento setup:di:compile — omit in developer mode if you never compile locally; typical for production mode
• php bin/magento cache:flush — picks up routes, DI, and the Admin menu

Check: php bin/magento module:status MageSail_Magesail — output should include Module is enabled.

Configuration and Node

• MageSail → Holesail Tunnel → Configuration → Scripts directory: absolute path to the directory that contains start-server.js. For this clone layout, that is usually …/app/code/MageSail/Magesail/scripts.
• In that scripts directory, once: npm install

First tunnel

Open MageSail → Holesail Tunnel → Manage Tunnel.

• Website: hostname and local port, start the tunnel, copy the hs:// key. In Holesail Browser, add a Virtual Host for that key.
• Service: choose a backend from env.php, copy the hs:// key. In the Browser, add a Service Tunnel so traffic lands on a local TCP port for your tools.

Full detail and edge cases: docs/installation.md, README.md.

How it works under the hood

PHP (everything inside Magento) covers Admin UI, ACL, the tunnel registry, website tunnel store provisioning, optional NGINX map updates via NginxGlobalMapManager, cron (Magesail/Cron/DailyMonitor.php), and plugins that keep request scope and URLs aligned with tunnel traffic.

Node runs Magesail/scripts/start-server.js, a thin wrapper around the holesail package. TunnelManager starts it with port, secure mode, key and log paths, and optional forwardHost (default 127.0.0.1). When the tunnel is ready, the process writes the hs:// URL to the key file so Admin can poll magesail/tunnel/status.

On disk: tunnel metadata lives in var/magesail_tunnels.json.

flowchart LR
  admin[Admin ManageTunnel]
  node[Holesail Node]
  registry[magesail tunnels json]
  provision[TunnelStoreProvisioner]
  resolver[EnvServicePortResolver]
  nginxTool[NginxGlobalMapManager]
  detector[TunnelRequestDetector]
  plugins[Store and URL plugins]
  admin --> node
  admin --> provision
  admin --> resolver
  admin --> nginxTool
  provision --> registry
  resolver --> registry
  detector --> plugins

Store routing on the storefront

For website tunnels the core question is which store view applies. TunnelRequestDetector considers, in order: the magesail_store query parameter, the X-MageSail-Store-Code header (and the matching CGI style server variable), a parse of the query string from the raw URI if needed, then hostname matching against tunnel base URLs in TunnelRegistry. Codes are validated with StoreRepository so stale NGINX map values or bookmarks do not take the site down.

StoreFromMageSailHeaderPlugin feeds MAGE_RUN_CODE and MAGE_RUN_TYPE from that result on the storefront. On admin routes it intentionally clears tunnel scope so the backend does not stay stuck in a tunnel store view and 404. The rest of the story is redirect suppression and link base URL rewriting so HTML stays on the tunnel host. The full plugin list and sort orders are documented in docs/request-routing-and-plugins.md; redirect behavior is explained in Magesail/doc/TUNNEL_BASE_URL_REDIRECT.md.

Recommended client: Holesail Browser

Holesail Browser (git clone https://git.ssh.surf/snxraven/holesail-browser) is a Chrome or Firefox extension plus a native host. You copy the hs:// string from MageSail → Manage Tunnel when the process is running, then wire it in the Browser UI depending on tunnel type.

• MageSail Website tunnel: add a Virtual Host in Holesail Browser and point it at that hs:// key. You browse a hostname you choose; PAC, local CONNECT / HTTPS proxies, SNI, and a trusted local CA handle HTTPS in the browser. That matches hostname based store view detection in Magento.

• MageSail Service tunnel: MageSail still gives you an hs:// URL for the exposed TCP service (for example MySQL). In Holesail Browser add a Service Tunnel: you attach that hs:// peer to a local TCP port so your laptop tools (mysql client, Redis CLI, and so on) can dial 127.0.0.1 and ride the tunnel.

Server Tunnels in Holesail Browser are the opposite direction (they publish a local port as an hs:// key). That is not how you consume a MageSail Service tunnel; use Service Tunnel there with the key from Admin.

Docs: README, TUNNELING, VIRTUAL-HOSTS, ARCHITECTURE.

For more on native messaging and browsers, see Native Messaging: Finally Bringing P2P to the Modern Browser.

Further reading

• docs/README.md: documentation index
• docs/overview.md: component overview
• docs/nginx-global-map.md: map file workflow
• LICENSE: GPL 3.0 or later