When running tests that involve 2 runs of the parser, one with the
`StrInput` and another with the `BufferedInput`, the output of debug
prints can be confusing as it's unclear where one parser ends parsing
and the other starts. These 2 prints allow us to know when each is
started and, if one errors but not the other, to know which did.
This requires making the `input` mod public, as some items in
`char_traits` are referenced in `Input`'s function doccomments.
The items have been exposed in `input` directly, rather than in a
`saphyr::input::char_traits` submod.
This issue arises in the `yaml-test-suite` not in a test, but rather in
the test description files themselves. Since we now use the library
itself to load the test files, this made the `yaml-test-suite` fail.
Look ahead before testing for EOF.
This fixes panics in saphyr's test_multiline_trailing_newline and
test_multiline_leading_newline tests, in which `self.input.next_is_z`
would be called on an empty buffer and panic in `peek`
Previously, we often used the scanner state to infer the positions of
mappings. This is sometimes wrong, because the scanner has already
scanned ahead by the time the mapping is parsed.
This commit adds a check to the test suite, asserting that parser event
positions are all observed in order, and it fixes the scanner and parser
to make the new check pass.
We reserve bufmaxlen bytes in the string, but then proceed to push
up to bufmaxlen chars. Therefore it is possible that the string will
need to reallocate. A different solution would be to reserve
4*bufmaxlen bytes. Removing the assert is probably ok, because the
attempt to pre-allocate is just a performance optimization.
* Use `str::strip_prefix` to avoid using `str::from_utf8_unchecked`
* Avoid most uses of `extend_left` unsafe function
* Remove `Input::push_back` and remaining unsafe
This is, for some reason, a huge pessimization. `rustc` fails to
optimize it as well as it did when it was part of `Scanner`.
This is however kinda needed if I want to avoid having this code
duplicated in every implementation of the input.
The refactoring added `next_is` which takes a `&str` as parameter, while
we only use it with strings of lengths 2 and 3. Replacing this by 2
dedicated methods (which can be added to the trait interface and only
specialized if needed) removes almost all the overhead that was added by
`Input`.
Hiding character fetching behind this interface allows us to create more
specific implementations when is appropriate. For instance, an instance
of `Input` can be created for a `&str`, allowing for borrowing and more
efficient peeking and traversing than if we were to fetch characters one
at a time and placing them into a temporary buffer.
These are corner cases not tested by the `yaml-test-suite`.
Parsing for the reported input has been fixed, as well as other
similar-looking inputs which make use of nested implicit flow mappings
and implicit null keys and values.
Fixes#1.
Internally, `ScanError` stores a `String`. Having `new` take a `&str`
misleadingly indicates that it stores a `&str`. This commit changes
`new` so that it takes a `String`, and adds a convenience method,
`new_str` that takes a `&str` and allocates a `String` for it.
