Add documentation for those tools, and make it so that `gen_large_yaml`
generates a predetermined set of files instead of outputting to its
standard output.
This removes all allocations in the `Scanner` code. The downside is that
the buffer is now stored in the `Scanner` structure, making it 48 bytes
larger. This however makes the code much more performant.
`self.buffer` is a `VecDeque<char>`, meaning that characters are stored
on 4B. When reading as we used to do, this means that every 1 byte
character we read was turned into 4 bytes, which was turned into 1 byte
in `String::extend`.
Instead of going through `self.buffer`, use a local `String` to store
the characters before pushing them to `string`.
Instead of doing a loop that goes:
* fetch from input stream
* push char into string
Make a loop that fetches characters while they're not a breakz and
_then_ extend the string. This avoids a bunch of reallocations.
If building release mode, remove debug code. Now, the `debug_print!`
macro resolves to nothing in release build.
In debug build, don't check the environment for each print. This has a
huge overhead. The environment is only checked once and next checks are
made against a simple boolean value.
This is a huge commit that cannot easily be broken down as it contains
fixes for the next ignored test in the suite which, one fixed, broke
tests that used to pass and were only then fixed.
There is also a substantial amount of comments that were added,
especially around `SimpleKey`. Minor improvements around the code were
added and I did not bother making a separate commit for them.
Overall, that commit fixes 7 tests from the matrix that were related to
the handling of simple keys.
