This issue arises in the `yaml-test-suite` not in a test, but rather in
the test description files themselves. Since we now use the library
itself to load the test files, this made the `yaml-test-suite` fail.
Look ahead before testing for EOF.
This fixes panics in saphyr's test_multiline_trailing_newline and
test_multiline_leading_newline tests, in which `self.input.next_is_z`
would be called on an empty buffer and panic in `peek`
Previously, we often used the scanner state to infer the positions of
mappings. This is sometimes wrong, because the scanner has already
scanned ahead by the time the mapping is parsed.
This commit adds a check to the test suite, asserting that parser event
positions are all observed in order, and it fixes the scanner and parser
to make the new check pass.
We reserve bufmaxlen bytes in the string, but then proceed to push
up to bufmaxlen chars. Therefore it is possible that the string will
need to reallocate. A different solution would be to reserve
4*bufmaxlen bytes. Removing the assert is probably ok, because the
attempt to pre-allocate is just a performance optimization.
* Use `str::strip_prefix` to avoid using `str::from_utf8_unchecked`
* Avoid most uses of `extend_left` unsafe function
* Remove `Input::push_back` and remaining unsafe
This is, for some reason, a huge pessimization. `rustc` fails to
optimize it as well as it did when it was part of `Scanner`.
This is however kinda needed if I want to avoid having this code
duplicated in every implementation of the input.
The refactoring added `next_is` which takes a `&str` as parameter, while
we only use it with strings of lengths 2 and 3. Replacing this by 2
dedicated methods (which can be added to the trait interface and only
specialized if needed) removes almost all the overhead that was added by
`Input`.
Hiding character fetching behind this interface allows us to create more
specific implementations when is appropriate. For instance, an instance
of `Input` can be created for a `&str`, allowing for borrowing and more
efficient peeking and traversing than if we were to fetch characters one
at a time and placing them into a temporary buffer.
This would make more sense in user code:
```rs
Yaml::load_from_str("foo"); // Explicit that we're parsing YAML
load_from_str("foo"); // Too implicit, too generic, may be from another
lib
```
Plus, this mirrors `MarkedYaml`'s behavior.
A few changes have had to be made to `LoadableYamlNode`:
* The `From<Yaml>` requirement has been removed as it can be
error-prone. It was not a direct conversion as it is unable to
handle `Yaml::Hash` or `Yaml::Array` with a non-empty array/map.
* Instead, `from_bare_yaml` was added, which does essentially the same
as `From` but does not leak for users of the library.
* `with_marker` has been added to populate the marker for the `Node`.
The function is empty for `Yaml`.
`load_from_*` methods have been added to `MarkedYaml` for convenience.
They load YAML using the markers.
The markers returned from `saphyr-parser` are not all correct, meaning
that tests are kind of useless for now as they will fail due to bugs
outside of the scope of this library.
* Make `YamlLoader` generic on the type of the `Node`. This is required
because deeper node need to have annotations too.
* Add a `LoadableYamlNode` trait, required for YAML node types to be
loaded by `YamlLoader`. It contains methods required by `YamlLoader`
during loading.
* Implement `LoadableYamlNode` for `Yaml`.
* Take `load_from_str` out of `YamlLoader` for parsing non-annotated
nodes. This avoids every user to specify the generics in
`YamlLoader::<Yaml>::load_from_str`.
These are corner cases not tested by the `yaml-test-suite`.
Parsing for the reported input has been fixed, as well as other
similar-looking inputs which make use of nested implicit flow mappings
and implicit null keys and values.
Fixes#1.
