Also helps in some cases with #142, when the BOM is at the beginning of
the file (common), but not in corner case where the BOM is at the start
of a document which is not the first one.
Closes: #155
Add documentation for those tools, and make it so that `gen_large_yaml`
generates a predetermined set of files instead of outputting to its
standard output.
Add documentation for those tools, and make it so that `gen_large_yaml`
generates a predetermined set of files instead of outputting to its
standard output.
Add documentation for those tools, and make it so that `gen_large_yaml`
generates a predetermined set of files instead of outputting to its
standard output.
This removes all allocations in the `Scanner` code. The downside is that
the buffer is now stored in the `Scanner` structure, making it 48 bytes
larger. This however makes the code much more performant.
This removes all allocations in the `Scanner` code. The downside is that
the buffer is now stored in the `Scanner` structure, making it 48 bytes
larger. This however makes the code much more performant.
`self.buffer` is a `VecDeque<char>`, meaning that characters are stored
on 4B. When reading as we used to do, this means that every 1 byte
character we read was turned into 4 bytes, which was turned into 1 byte
in `String::extend`.
Instead of going through `self.buffer`, use a local `String` to store
the characters before pushing them to `string`.
`self.buffer` is a `VecDeque<char>`, meaning that characters are stored
on 4B. When reading as we used to do, this means that every 1 byte
character we read was turned into 4 bytes, which was turned into 1 byte
in `String::extend`.
Instead of going through `self.buffer`, use a local `String` to store
the characters before pushing them to `string`.
Instead of doing a loop that goes:
* fetch from input stream
* push char into string
Make a loop that fetches characters while they're not a breakz and
_then_ extend the string. This avoids a bunch of reallocations.
