What hitop run does
hitop run sends every request in a collection in order, runs the scripts before and after each one, and reports what passed. It works on a collection folder in your repository, so the pipeline tests exactly what is in the commit:
hitop run ./my-api --env staging
Each request shows its status, its response time and every check from its script after, with a mark for passed or failed and the reason for each failure. The run ends with a summary line such as Summary: 11/12 requests succeeded, 30/31 assertions passed.
A request passes when it was sent, the status is below 400, and every check in its script after passed.
Install it in the pipeline
Use the install script (curl -fsSL https://hitop.rws8.tech/install.sh | sh, a static binary from GitHub Releases, checked against its SHA-256), cargo install hitop, or the Docker image:
docker run --rm -v "$PWD:/work" -w /work ghcr.io/bohdaq/hitop:1.2.0 run ./my-api --env ci
On GitHub, the HITOP action does this in one step and adds a job summary. GitLab needs the image's -debug tag, which has a shell: see Run HITOP in GitLab CI.
Exit codes
Pipelines decide from the exit code, so hitop keeps them simple:
| Code | Meaning |
|---|---|
| 0 | Every request passed. |
| 1 | A request or a check failed: a wrong status, a failed assertion, a timeout or a server that did not answer. |
| 2 | Usage or configuration error: an unknown collection or environment, a bad flag, an unreadable environment or data file, a missing secret, or reports that could not be written. |
hitop run --help lists them too. Code 2 means the run never really started, so fix the command rather than the API.
Reports
Pick reporters with --reporter, separated by commas, and a folder with --out (default hitop-reports):
hitop run ./my-api --env ci --reporter cli,junit,json --out reports
- cli: progress and results in the log (the default).
- junit:
reports/junit.xml, one test suite per collection and one test case per request. Most CI systems show it as a test report: GitLabartifacts:reports:junit, Jenkins, Azure Pipelines, and test report actions on GitHub. - json:
reports/report.json, with totals, every request and every check. Its format is versioned ("reportVersion": 1) and documented in the run report spec, so your own scripts can read it safely. - html:
reports/report.html, one page for people: totals, every request with its timings, failure and checks. It is self-contained (no script, nothing loaded from the network) and follows the system's light or dark theme, so it works as a downloaded CI artifact. From hitop 1.0.0.
Secret values never reach a report. Wherever one would appear, in a URL, an error or a check, it is shown as ••••.
Run on a schedule
hitop schedule ./my-api --github --env prod writes a GitHub Actions workflow that runs the folder every day and on demand (see Run HITOP in GitHub Actions). On your own machine or a server, hitop schedule ./my-api --cron "*/30 * * * *" --env local prints a crontab line instead: it keeps every run's reports in a folder of their own and appends the output to a log; add it with crontab -e.
Environments and variables
Choose the variables for the run:
--env <name>: one of the folder'senvironments/files.--env-file <file.yaml>: an environment file from anywhere, in the same format, for example one your pipeline writes.--var key=value: one variable for this run; repeat the flag for more.
Later sources win: collection variables, then the environment, then the data row, then --var, then secrets. --var values are printed like any other value, so give secrets through HITOP_SECRET_<ENV>_<NAME> (for example HITOP_SECRET_CI_TOKEN for token in environment ci) or --secret NAME=value:
HITOP_SECRET_CI_TOKEN=$API_TOKEN hitop run ./my-api --env ci --var base=https://preview.example.com
Stop early or limit time
--bail: stop at the first request that fails. Without it every request runs, so one report shows every problem.--timeout <ms>: the longest a request may take. A slower one fails with a timeout and the run goes on.
Data driven runs
--data runs the collection once per row of a CSV or JSON file, with the row's values as variables:
user,expected
ada,200
mallory,403
hitop run ./my-api --env ci --data users.csv
A request can use ${user} and a check can use getVariable('expected'). A JSON file is an array of objects: [{"user": "ada"}, {"user": "mallory"}]. Each row is an iteration: the log shows Iteration 2 of 2, the JUnit report has one test suite per iteration with the row as properties, and the JSON report keeps the row with its results.
To repeat a run without data, use --iterations <n>.
Why run HITOP in CI?
- Collections in git: the requests reviewed in a merge request are the ones the pipeline runs.
- Clear exit codes: 0, 1 or 2, nothing to parse.
- Standard reports: JUnit for your CI's test view, versioned JSON for your own tools.
- Secrets stay secret: from environment variables, masked in every log and report.
- One small binary: no runtime to install.