Research workflows

Share a private research dashboard

Launch a small managed HTTP app, verify readiness and collaborator access, inspect failures, and stop it.

Create a tiny Python dashboard backed by synthetic measurements, register it as a managed project app, and open it with a collaborator. Python's standard library supplies the server; no web-framework installation is needed.

Use a project in which you are an owner or collaborator with runtime access. Viewer access cannot start or open project app servers. Managed apps use the project's shared trust model: other code and collaborators in that project are part of the same working environment. This tutorial creates an authenticated project app, not an anonymous public website.

Prepare the project and dashboard

Complete the CLI quickstart. Run the following in Bash on your own computer, where Python 3 and the CLI are installed:

export CLI_PROFILE=cocalc-ai
export PROJECT_ID='REPLACE_WITH_FULL_PROJECT_ID'
export DASHBOARD_DIR='/home/user/research-dashboard-demo'

cocalc --profile "$CLI_PROFILE" --json auth status --check
cocalc --profile "$CLI_PROFILE" project get --project "$PROJECT_ID"
cocalc --profile "$CLI_PROFILE" project exec --project "$PROJECT_ID" -- python3 --version
cocalc --profile "$CLI_PROFILE" project app list --project "$PROJECT_ID"

LOCAL_DASHBOARD=$(mktemp -d)
cd "$LOCAL_DASHBOARD"
cat > measurements.csv <<'CSV'
value
2
4
6
8
CSV

cat > dashboard.py <<'PYTHON'
import csv
import html
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import math
import os
from pathlib import Path
import statistics
from urllib.parse import urlsplit

DATA = Path(__file__).with_name("measurements.csv")

class Dashboard(BaseHTTPRequestHandler):
    def do_GET(self):
        route = urlsplit(self.path).path
        if route == "/health":
            content = b"ok\n"
            content_type = "text/plain; charset=utf-8"
        elif route == "/":
            try:
                with DATA.open(encoding="utf-8", newline="") as source:
                    reader = csv.DictReader(source)
                    if reader.fieldnames != ["value"]:
                        raise ValueError("Expected one column named value")
                    values = []
                    for row in reader:
                        if set(row) != {"value"} or row["value"] is None:
                            raise ValueError("Expected exactly one value per CSV row")
                        values.append(float(row["value"]))
                if not all(math.isfinite(value) for value in values):
                    raise ValueError("Measurements must be finite")
                mean = statistics.mean(values)
            except (OSError, ValueError, KeyError, statistics.StatisticsError):
                self.send_error(503, "Measurement data is unavailable or invalid")
                return
            rows = "".join(f"<li>{html.escape(str(value))}</li>" for value in values)
            content = (
                "<!doctype html><html lang='en'><meta charset='utf-8'>"
                "<meta name='viewport' content='width=device-width, initial-scale=1'>"
                "<title>Research measurements</title>"
                "<main><h1>Research measurements</h1>"
                f"<p>Count: {len(values)}; mean: {mean}</p><ul>{rows}</ul>"
                "<p>Refresh after updating measurements.csv.</p></main></html>"
            ).encode("utf-8")
            content_type = "text/html; charset=utf-8"
        else:
            self.send_error(404)
            return
        self.send_response(200)
        self.send_header("Content-Type", content_type)
        self.send_header("Content-Length", str(len(content)))
        self.send_header("Cache-Control", "no-store")
        self.end_headers()
        self.wfile.write(content)

if __name__ == "__main__":
    host = os.environ.get("HOST", "127.0.0.1")
    port = int(os.environ.get("PORT", "8765"))
    print(f"Dashboard listening on {host}:{port}", flush=True)
    ThreadingHTTPServer((host, port), Dashboard).serve_forever()
PYTHON

cocalc --profile "$CLI_PROFILE" project file put --project "$PROJECT_ID" \
  measurements.csv "$DASHBOARD_DIR/measurements.csv"
cocalc --profile "$CLI_PROFILE" project file put --project "$PROJECT_ID" \
  dashboard.py "$DASHBOARD_DIR/dashboard.py"

Adjust the remote home path if necessary. Use an unused directory and confirm that no existing app has ID research-dashboard-demo; choose another ID throughout this guide if it does. file put and app upserts replace existing content at their destinations.

The same Python source is available as dashboard.py in the example directory.

Register the managed app

Create this JSON spec locally, using the selected remote directory:

python3 - <<'PYTHON'
import json
import os
from pathlib import Path
spec = {
    "version": 1,
    "id": "research-dashboard-demo",
    "title": "Research measurements",
    "kind": "service",
    "lifecycle": {"mode": "managed"},
    "command": {
        "exec": "python3",
        "args": ["-u", "dashboard.py"],
        "cwd": os.environ["DASHBOARD_DIR"],
    },
    "network": {"listen_host": "127.0.0.1", "port": 8765, "protocol": "http"},
    "proxy": {
        "base_path": "/apps/research-dashboard-demo",
        "strip_prefix": True,
        "websocket": False,
        "open_mode": "proxy",
        "readiness_timeout_s": 30,
    },
    "wake": {"enabled": False, "keep_warm_s": 1800, "startup_timeout_s": 30},
}
Path("dashboard-app.json").write_text(json.dumps(spec, indent=2) + "\n")
PYTHON

cocalc --profile "$CLI_PROFILE" --json project app upsert \
  --project "$PROJECT_ID" --file dashboard-app.json
cocalc --profile "$CLI_PROFILE" --json project app start research-dashboard-demo \
  --project "$PROJECT_ID" --wait --timeout 30s
cocalc --profile "$CLI_PROFILE" --json project app status research-dashboard-demo \
  --project "$PROJECT_ID"

The app manager supplies HOST and PORT to the process, so the script and spec agree about the listener. The explicit remote cwd locates dashboard.py; it does not point to your laptop's files. strip_prefix lets the server handle / when a request arrives through the app's CoCalc URL. This example disables automatic wake-up: start it explicitly, and stopping it will not allow a later request to restart it automatically.

Require ok: true, data.state: "running", and data.ready: true. Readiness checks that the port accepts a connection; it does not verify the correctness of your dataset or page. Confirm the HTTP content separately:

cocalc --profile "$CLI_PROFILE" project exec --project "$PROJECT_ID" -- \
  python3 -c 'from urllib.request import urlopen; page=urlopen("http://127.0.0.1:8765/", timeout=5).read().decode(); assert "Count: 4; mean: 5.0" in page; print("PASS: dashboard serves four measurements with mean 5.0")'

This check runs inside the project. 127.0.0.1:8765 on your laptop refers to your laptop, not to CoCalc. If you choose a different port in the spec, update the verification URL too.

Managed local forwards

For local tools, cocalc project app forward APP_ID --project PROJECT_ID creates or reuses a managed SSH tunnel to a service app. It may start the app, ensure/install an SSH key, and write local SSH configuration. The default local bind address is loopback. Inspect the returned project_id, app_id, local_url, forward_id, and reused fields before using or stopping the tunnel; a reused tunnel may also serve another local task.

project app forward-list APP_ID --project PROJECT_ID lists matching managed local forwards.
project app forward-stop APP_ID --project PROJECT_ID can stop multiple matching forwards for that app.
project sync forward terminate FORWARD_ID targets the explicit local forward id. Record the id from the forward response when you need to limit cleanup to one tunnel.

Stopping a tunnel does not stop the app or remove installed SSH keys and configuration. App cleanup is separate, as described below. A reported running tunnel or a TCP-ready app does not establish correct HTTP output; check the application response separately. Local tunnel access also does not establish that a collaborator can open the authenticated app URL.

See the CLI command reference for command help and SSH access for connection setup.

Open the same project in CoCalc's full interface and select Apps.
Find Research measurements and use its open action. CoCalc constructs the authenticated project-host URL; do not paste a raw data.url path from the CLI into your laptop browser and assume it is a complete URL.
Confirm the heading Research measurements and Count: 4; mean: 5.0.
Add your colleague using project collaborators with a role that permits runtime access. Have them sign in, open that same project, and open the app from Apps themselves.
Confirm that both of you see the same measurements. This is the collaboration check; your own successful browser request alone does not verify their access.

Use CoCalc's authenticated opening flow for each person. Do not distribute bootstrap URLs containing temporary authentication tokens. A viewer role is suitable for reading supported project files, but is not enough to open this running service.

To change the example, edit your local measurements.csv, upload it to the same remote file, and refresh the page. The program reads that file on every request. Keep an original copy of the input if the update represents a new experiment.

Diagnose startup and data failures

cocalc --profile "$CLI_PROFILE" --json project app logs research-dashboard-demo \
  --project "$PROJECT_ID" --tail 50
cocalc --profile "$CLI_PROFILE" --json project app status research-dashboard-demo \
  --project "$PROJECT_ID"

Python or the script cannot be found: inspect the app spec's command and the remote file paths. Install/select a Python environment if needed.
The port is already in use: inspect existing project apps, choose an unused port in dashboard-app.json, upsert the spec, and restart this app. Do not kill another researcher's process just to reuse its port.
The page returns 503: the listener may still be ready. Restore a valid CSV with a value header and at least one numeric row, then refresh.
The CLI wait times out: inspect status and logs before starting again. The process may have started after the wait deadline.
A collaborator cannot open the app: verify their project membership and runtime role, then have them open it through their own signed-in Apps page.

After changing the Python program or spec, explicitly restart the managed app:

cocalc --profile "$CLI_PROFILE" --json project app restart research-dashboard-demo \
  --project "$PROJECT_ID" --wait --timeout 30s

Recheck both the page contents and the collaborator's access when those behaviors are affected. Captured stdout/stderr are useful diagnostics, not a durable record of the experiment; retain the inputs and analysis results as project files.

Stop and remove the scratch app

Close its browser tabs and stop the managed process:

cocalc --profile "$CLI_PROFILE" --json project app stop research-dashboard-demo \
  --project "$PROJECT_ID"
cocalc --profile "$CLI_PROFILE" --json project app status research-dashboard-demo \
  --project "$PROJECT_ID"

Confirm the app is stopped. When finished with the example, remove its registered spec:

cocalc --profile "$CLI_PROFILE" project app delete research-dashboard-demo \
  --project "$PROJECT_ID"

The uploaded Python and CSV files remain research files. Inspect and remove only research-dashboard-demo in Files if no longer needed; retain your local copies for reproduction. Stopping the app does not stop the project itself.