This tool helps with comparing times different YAML parsers take to parse the same input.
## Synopsis
```
bench_compare time_parse
bench_compare run_bench
```
This will run either `time_parse` or `run_bench` (described below) with the given set of parsers from the configuration file.
## Parsers requirements
Parsers are expected to be event-based. In order to be fair to this crate's benchmark implementation, parsers should:
* Load the file into memory (a string, `mmap`, ...) **prior** to starting the clock
* Initialize the parser, if needed
* **Start the clock**
* Read events from the parser while the parser has not finished parsing
* Discard events as they are received (dropping them, `free`ing them or anything similar) so as to not grow their memory consumption too high, and allowing the parser to reuse event structures
* **Stop the clock**
* Destroy the resources, if needed/wanted (parser, file buffer, ...). The kernel will reap after the process exits.
## Parsers required binaries
This tool recognizes 2 binaries: `time_parse` and `run_bench`.
### `time_parse`
Synopsis:
```
time_parse file.yaml [--short]
```
The binary must run the aforementioned steps and display on its output the time the parser took to parse the given file.
With the `--short` option, the binary must only output the benchmark time in nanoseconds.
```sh
# This is meant to be human-readable.
# The example below is what this crate implements.
$> time_parse file.yaml
Loaded 200MiB in 1.74389s.
# This will be read by this tool.
# This must output ONLY the time, in nanoseconds.
$> time_parse file.yaml --short
1743892394
```
This tool will always provide the `--short` option.
### `run_bench`
Synopsis:
```
run_bench file.yaml <iterations> [--output-yaml]
```
The binary is expected to run `<iteration>` runs of the aforementioned steps and display on its output relevant information.
The `--output-yaml` instructs the binary to output details about its runs in YAML on its standard output.
The binary may optionally perform some warmup runs prior to running the benchmark. The time it took the binary to run will not be evaluated.
```sh
# This is meant to be human-readable.
# The example below is what this crate implements.