Monitoring and Health

These commands answer the question of whether a site is up, fast, sending mail and reachable over SSH.

monitor

captaincore monitor run <site|target>
captaincore monitor stats

A monitor run holds a PID lock at <path_tmp>/captaincore-monitor.lock. If the previous run is still alive the new one exits immediately and records a skipped run. If the lock is stale, the run proceeds and emails the admin address from captaincore_admin_email to report that the previous job died without cleanup.

URLs are built from each environment’s home URL plus the optional --page. Checks run through a bounded worker pool, and results are streamed to stdout as JSON one line at a time and appended to a timestamped log file under the system logs path, named YYYY-MM-DD_HH-MM_<random>.txt.

Failures are retried up to --retry times with a ten second pause between attempts. Early attempts use a fifteen second timeout, the final attempt uses sixty seconds. Retries use a transport that resolves through 1.1.1.1 directly, which rules out a local resolver problem being mistaken for a site outage.

Running state is kept in monitor.json under the system data path, tracking each URL’s HTTP code, HTML validity, check count, notify count and timestamps. Run-level statistics go to the monitor stats database, which is what monitor stats reads.

captaincore monitor run @production.monitor-on --parallel=20 --retry=3

monitor-check

captaincore monitor-check <url,name>

A single HTTP health check on one URL, with the display name after the comma. HTTP URLs are upgraded to HTTPS first and fall back to HTTP if the connection fails. The result is a JSON object with http_code, num_redirects, url, name, html_valid and, on failure, error.

captaincore monitor-check https://example.com,mysite

monitor-notify

captaincore monitor-notify <account-portal-id>

The command requires an account portal ID argument. In the current source its handler only prints Monitoring...; the notification email itself is sent through the Manager API by monitor run.

performance-monitor

captaincore performance-monitor activate <site>
captaincore performance-monitor deactivate <site>
captaincore performance-monitor fetch <site> [--hours=] [--format=raw]

activate deploys the sampling scripts into the site’s private directory and installs a crontab entry that runs the check every ten minutes:

*/10 * * * * bash <private_dir>/php-monitor-check.sh

deactivate removes both the scripts and the crontab line. fetch pulls the accumulated samples back and parses them; without --format=raw it aggregates rather than listing every sample.

captaincore performance-monitor fetch mysite-production --hours=24

scan-errors

captaincore scan-errors <site>

Runs Lighthouse headless against the environment’s home URL with --only-audits=errors-in-console and reports the JavaScript console errors it finds. The home URL must be a clean http or https URL; anything containing shell metacharacters is refused rather than executed.

captaincore scan-errors mysite-production

email-health

captaincore email-health generate <site|target>
captaincore email-health send <site> <token>
captaincore email-health response <site|target>

generate runs the check across one or more sites or a target group, retrying failures. send fires a single test email from a site using a token that identifies the check. response logs the result once the test message is observed.

captaincore email-health generate @production

logs

captaincore logs list <site>
captaincore logs get <site> --file=<filename> [--limit=<lines>]
captaincore logs archive <site|@target> [--parallel=] [--dry-run] [--skip-if-recent=]
captaincore logs archive-list <site>
captaincore logs archive-get <site> <file> [--expire=<hours>]

logs list and logs get read the live server logs over SSH. logs archive enumerates the rotated access and error logs on the server, then pipes each one through gzip straight into rclone rcat, so nothing is staged on disk locally. Files already present at the destination are skipped, and the run reports how many were uploaded versus already in B2. The destination path is deterministic:

<rclone_backup>/<site>_<site_id>/<environment>/logs/

archive-list lists what is stored there and archive-get returns a signed download URL.

captaincore logs archive @all --parallel=10 --skip-if-recent=24h

ssh-detect

captaincore ssh-detect <username> <address> <port> [--debug]

Connects with the configured default key and runs cd public; pwd, which both proves the connection works and reports the site’s home directory. The Manager uses this when provisioning a new site. --debug prints the assembled SSH command instead of running it.

captaincore ssh-detect myuser 1.2.3.4 12345 --debug

ssh-verify

captaincore ssh-verify <ssh-connection-string>

Verifies that a given SSH connection string is usable.

captaincore ssh-verify "[email protected]:12345"

Related security scans

Most malware and integrity checking is done with remote scripts rather than dedicated commands, for example detect-seo-spam, malware-hunt, find-nulled-components and php-in-uploads. See remote scripts. Capture-based injected-script detection is covered in backups and history.