Run a research analysis from your laptop

Run an analysis in CoCalc from your laptop

Upload a small CSV and Python script, execute the script inside your project, and download a verified result. All commands below run in Bash on your own computer; project exec runs the specified process remotely. Python's standard library is sufficient on both computers.

This is a single-file transfer recipe. For many files or a large directory tree, use SSH and rsync. File uploads and downloads replace an existing destination file; choose a scratch project and unused paths.

Prepare the connection and local files

Complete the CLI quickstart, then replace the project placeholder with the full ID from project list. Keep this shell open:

export CLI_PROFILE=cocalc-ai
export PROJECT_ID='REPLACE_WITH_FULL_PROJECT_ID'
export REMOTE_DIR='/home/user/research-cli-demo'

cocalc --profile "$CLI_PROFILE" --json auth status --check
cocalc --profile "$CLI_PROFILE" project get --project "$PROJECT_ID"
cocalc --profile "$CLI_PROFILE" project exec --project "$PROJECT_ID" -- pwd
cocalc --profile "$CLI_PROFILE" project exec --project "$PROJECT_ID" -- python3 --version

LOCAL_RUN=$(mktemp -d)
cd "$LOCAL_RUN"
printf 'Local example directory: %s\n' "$LOCAL_RUN"

Confirm the account, project, and data.check.ok: true in the authentication result. Adjust REMOTE_DIR if your project's home differs from /home/user. LOCAL_RUN is on your computer; REMOTE_DIR is inside CoCalc. The remote process working directory is selected by project exec --path, not --cwd.

Create these two local files:

cat > measurements.csv <<'CSV'
value
2
4
6
8
CSV

cat > analyze.py <<'PYTHON'
import csv
import hashlib
import io
import json
import math
from pathlib import Path
import statistics
import sys

source, destination = map(Path, sys.argv[1:3])
raw = source.read_bytes()
rows = csv.DictReader(io.StringIO(raw.decode("utf-8")))
if rows.fieldnames != ["value"]:
    raise ValueError("Expected a CSV with one column named value")
values = []
for row in rows:
    if set(row) != {"value"} or row["value"] is None:
        raise ValueError("Expected exactly one value per CSV row")
    values.append(float(row["value"]))
if not values or not all(math.isfinite(value) for value in values):
    raise ValueError("Expected at least one finite measurement")
result = {
    "count": len(values),
    "mean": statistics.mean(values),
    "input_sha256": hashlib.sha256(raw).hexdigest(),
}
destination.write_text(json.dumps(result, indent=2) + "\n", encoding="utf-8")
print(f"Wrote {destination}: count={result['count']}, mean={result['mean']}")
PYTHON

Upload and run

file put takes a local source followed by a remote destination. It creates remote parent directories by default. Use the absolute remote paths shown here:

cocalc --profile "$CLI_PROFILE" project file put --project "$PROJECT_ID" \
  measurements.csv "$REMOTE_DIR/measurements.csv"
cocalc --profile "$CLI_PROFILE" project file put --project "$PROJECT_ID" \
  analyze.py "$REMOTE_DIR/analyze.py"
cocalc --profile "$CLI_PROFILE" project file list --project "$PROJECT_ID" \
  "$REMOTE_DIR"

cocalc --profile "$CLI_PROFILE" --json project exec --project "$PROJECT_ID" \
  --path "$REMOTE_DIR" --timeout 60 -- \
  python3 analyze.py measurements.csv result.json > execution.json

python3 - <<'PYTHON'
import json
from pathlib import Path
response = json.loads(Path("execution.json").read_text())
assert response["ok"], response
assert response["data"]["exit_code"] == 0, response["data"]
print(response["data"]["stdout"], end="")
PYTHON

Expected stdout is Wrote result.json: count=4, mean=5.0. A successful JSON response has ok: true, but you must also check data.exit_code == 0: a remote program can fail even when the CLI successfully retrieves its result. Read data.stderr for Python errors. See scripting and results.

Download and verify the artifact

file get takes a remote source followed by a local destination. It retrieves one file, not a recursive directory:

cocalc --profile "$CLI_PROFILE" project file get --project "$PROJECT_ID" \
  "$REMOTE_DIR/result.json" downloaded-result.json

python3 - <<'PYTHON'
import hashlib
import json
from pathlib import Path
result = json.loads(Path("downloaded-result.json").read_text())
expected_hash = hashlib.sha256(Path("measurements.csv").read_bytes()).hexdigest()
assert result["count"] == 4, result
assert result["mean"] == 5.0, result
assert result["input_sha256"] == expected_hash, result
print("PASS: four measurements, mean 5.0, input checksum matches")
PYTHON

The checksum connects the result to the exact input bytes you uploaded. Keep analyze.py, measurements.csv, execution.json, and the downloaded result together when handing the analysis to another researcher. To reproduce the work in a fresh project, see reproduce an analysis.

Recover from a failed step

• Wrong project or path: inspect project get and file list before uploading again. A relative local path is interpreted on your computer; the paths passed after --path and as upload destinations are remote.
• Python is unavailable: select a suitable project environment or install it using your project's software workflow. A local Python installation does not install Python in CoCalc.
• Remote execution fails: inspect execution.json, fix the local source, upload that file again, and rerun. This example deterministically replaces only result.json; adapt retry behavior before running analyses with other side effects.
• The connection times out: inspect the remote files before retrying. A client timeout does not prove the command made no changes.
• Downloaded results do not match: compare the remote input with your local input and rerun the validation. Do not treat an older result.json as evidence that the latest execution succeeded.

After retaining the result, use Files to inspect and delete only the scratch research-cli-demo directory if you no longer need it. Your local files remain in the directory printed as LOCAL_RUN; review them before deleting that directory. The analysis process exits on completion, so there is no service to stop.