# Running Tests ## Use --test Option Hurl is run by default as an HTTP client, returning the body of the last HTTP response. ```shell $ hurl hello.hurl Hello World! ``` When multiple input files are provided, Hurl outputs the body of the last HTTP response of each file. ```shell $ hurl hello.hurl assert_json.hurl Hello World![ { "id": 1, "name": "Bob"}, { "id": 2, "name": "Bill"} ] ``` For testing, we are only interested in the asserts results, we don't need the HTTP body response. To use Hurl as a test tool with an adapted output, you can use [`--test` option]: ```shell $ hurl --test hello.hurl assert_json.hurl hello.hurl: Success (6 request(s) in 245 ms) assert_json.hurl: Success (8 request(s) in 308 ms) -------------------------------------------------------------------------------- Executed files: 2 Executed requests: 10 (17.82/s) Succeeded files: 2 (100.0%) Failed files: 0 (0.0%) Duration: 561 ms ``` Or, in case of errors: ```shell $ hurl --test hello.hurl error_assert_status.hurl hello.hurl: Success (4 request(s) in 5 ms) error: Assert status code --> error_assert_status.hurl:9:6  |  | GET http://localhost:8000/not_found  | ...  9 | HTTP 200  | ^^^ actual value is <404>  | error_assert_status.hurl: Failure (1 request(s) in 2 ms) -------------------------------------------------------------------------------- Executed files: 2 Executed requests: 5 (500.0/s) Succeeded files: 1 (50.0%) Failed files: 1 (50.0%) Duration: 10 ms ``` > With or without `--test`, all asserts are always executed. `--test` adds a run recap and disables the output of the > last response. To ignore asserts execution, you can use [`--ignore-asserts`]. In test mode, files are executed in parallel to speed-ud the execution. If a sequential run is needed, you can use [`--jobs 1`] option to execute tests one by one. ```shell $ hurl --test --jobs 1 *.hurl ``` [`--repeat` option] can be used to repeat run files and do [performance check]. For instance, this call will run 1000 tests in parallel: ```shell $ hurl --test --repeat 1000 stress.hurl ``` ### Selecting Tests Hurl can take multiple files into inputs: ```shell $ hurl --test test/integration/a.hurl test/integration/b.hurl test/integration/c.hurl ``` ```shell $ hurl --test test/integration/*.hurl ``` Or you can simply give a directory and Hurl will find files with `.hurl` extension recursively: ```shell $ hurl --test test/integration/ ``` Finally, you can use [`--glob` option] to test files that match a given pattern: ```shell $ hurl --test --glob "test/integration/**/*.hurl" ``` ## Debugging ### Debug Logs If you need more error context, you can use [`--error-format long` option] to print HTTP bodies for failed asserts: ```shell $ hurl --test --error-format long hello.hurl error_assert_status.hurl hello.hurl: Success (4 request(s) in 6 ms) HTTP/1.1 404 Server: Werkzeug/3.0.3 Python/3.12.4 Date: Wed, 10 Jul 2024 15:42:41 GMT Content-Type: text/html; charset=utf-8 Content-Length: 207 Server: Flask Server Connection: close 404 Not Found

Not Found

The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

error: Assert status code --> error_assert_status.hurl:9:6  |  | GET http://localhost:8000/not_found  | ...  9 | HTTP 200  | ^^^ actual value is <404>  | error_assert_status.hurl: Failure (1 request(s) in 2 ms) -------------------------------------------------------------------------------- Executed files: 2 Executed requests: 5 (454.5/s) Succeeded files: 1 (50.0%) Failed files: 1 (50.0%) Duration: 11 ms ``` Individual requests can be modified with [`[Options]` section][options] to turn on logs for a particular request, using [`verbose`] and [`very-verbose`] option. With this Hurl file: ```hurl GET https://foo.com HTTP 200 GET https://bar.com [Options] very-verbose: true HTTP 200 GET https://baz.com HTTP 200 ``` Running `hurl --test .` will output debug logs for the request to `https://bar.com`. [`--verbose`] / [`--very-verbose`] can also be enabled globally, for every requests of every tested files: ```shell $ hurl --test --very-verbose . ``` ### HTTP Responses In test mode, HTTP responses are not displayed. One way to get HTTP responses even in test mode is to use [`--output` option] of `[Options]` section: `--output file` allows to save a particular response to a file, while `--output -` allows to redirect HTTP responses to standard output. ```hurl GET http://foo.com HTTP 200 GET https://bar.com [Options] output: - HTTP 200 ``` ```shell $ hurl --test . 301 Moved

301 Moved

The document has moved here. /tmp/test.hurl: Success (2 request(s) in 184 ms) -------------------------------------------------------------------------------- Executed files: 1 Executed requests: 2 (10.7/s) Succeeded files: 1 (100.0%) Failed files: 0 (0.0%) Duration: 187 ms ``` ## Stress and Performance Tests Hurl can be used to perform stress tests: - with [query duration]: ```hurl GET https://example.org/foo HTTP 200 [Asserts] duration < 1200 ``` - [response metrics] with [`--very-verbose`] or [`--json`] to get structured timing data: ```shell $ hurl --json foo.hurl | jq ... "timings": { "app_connect": 0, "begin_call": "2025-10-06T07:00:57.127794Z", "connect": 120915, "end_call": "2025-10-06T07:00:57.280724Z", "name_lookup": 92987, "pre_transfer": 121014, "start_transfer": 152721, "total": 152807 } ... ``` In performance uses-cases, it's important to know how Hurl is working to get the best request per second load. Each Hurl file is processed by its own libcurl instance (roughly speaking a living HTTP connection). Let's say we want to stress our application with 100,000 HTTP requests and see how it's behaving. The simplest idea would be to make a single Hurl file repeating 100,000 requests: ```hurl GET https://my-website/health [Options] repeat: 100000 ``` and run it: ```shell $ hurl --test perf.hurl ``` With a single Hurl file, we won't benefit from any parallel runs (files are run in parallel, requests _within_ a file are run sequentially). To benefit from parallel run, we can use [`--repeat`] as a CLI argument to repeat file execution. It's as if we've written 100,000 identical Hurl files and run them: ```hurl GET https://my-website/health ``` and run it: ```shell $ hurl --test --repeat 100000 perf.hurl ``` In this case, we're running 100,000 Hurl files in parallel. Now, we have another problem: as each file is basically a HTTP connection, we can run out of TCP port, a phenomenon known as [ephemeral ports exhaustion]. So, to recap: - running 100,000 requests in a single file won't benefit from parallel run - running 100,000 one request files can lead to TCP port resources issues The solution is to combine the two approaches: running 10 files of 10,000 HTTP requests. With 10 files of 10,000 requests, we're going to execute those files in parallel in their own worker. Each worker will be working sequentially, maximizing the usage and not running into TCP ports exhaustion. We can force jobs count to be exactly 10 so all workers start at the same time and no one is waiting to get a job done: ```hurl GET https://my-website/health [Options] repeat: 10000 ``` ```shell $ hurl --test --repeat 10 --jobs 10 perf.hurl ``` ## Generating Report In the different reports, files are always referenced in the input order (which, as tests are executed in parallel, can be different from the execution order). ### HTML Report Hurl can generate an HTML report by using the [`--report-html DIR`] option. If the HTML report already exists, the test results will be appended to it.
Hurl HTML Report
The input Hurl files (HTML version) are also included and are easily accessed from the main page.
Hurl HTML file
### JSON Report A JSON report can be produced by using the [`--report-json DIR`]. The report directory will contain a `report.json` file, including each test file executed with [`--json`] option and a reference to each HTTP response of the run dumped to disk. If the JSON report already exists, it will be updated with the new test results. ### JUnit Report A JUnit report can be produced by using the [`--report-junit FILE`] option. If the JUnit report already exists, it will be updated with the new test results. ### TAP Report A TAP report ([Test Anything Protocol]) can be produced by using the [`--report-tap FILE`] option. If the TAP report already exists, it will be updated with the new test results. ## Use Variables in Tests To use variables in your tests, you can: - use [`--variable` option] - use [`--variables-file` option] - define environment variables, for instance `HURL_VARIABLE_foo=bar` You will find a detailed description in the [Injecting Variables] section of the docs. [`--output /dev/null`]: /docs/manual.md#output [`--test`]: /docs/manual.md#test [`--report-html DIR`]: /docs/manual.md#report-html [`--report-json DIR`]: /docs/manual.md#report-json [`--report-junit FILE`]: /docs/manual.md#report-junit [`--report-tap FILE`]: /docs/manual.md#report-tap [`--test` option]: /docs/manual.md#test [`--glob` option]: /docs/manual.md#glob [`--variable` option]: /docs/manual.md#variable [`--variables-file` option]: /docs/manual.md#variables-file [Injecting Variables]: /docs/templates.md#injecting-variables [Test Anything Protocol]: https://testanything.org [`--jobs 1`]: /docs/manual.md#jobs [`--json`]: /docs/manual.md#json [`--error-format long` option]: /docs/manual.md#error-format [options]: /docs/request.md#options [`--verbose`]: /docs/manual.md#verbose [`--very-verbose`]: /docs/manual.md#very-verbose [`verbose`]: /docs/manual.md#verbose [`very-verbose`]: /docs/manual.md#very-verbose [`--output` option]: /docs/manual.md#output [`--repeat` option]: /docs/manual.md#repeat [query duration]: /docs/asserting-response.md#duration-assert [response metrics]: /docs/response.md#timings [`--ignore-asserts`]: /docs/manual.md#ignore-asserts [performance check]: /docs/running-tests.md#stress-and-performance-tests [ephemeral ports exhaustion]: https://blog.cloudflare.com/how-to-stop-running-out-of-ephemeral-ports-and-start-to-love-long-lived-connections/ [`--repeat`]: /docs/manual.md#repeat