Rebuilt format: Meaning of tokens of length 0 or less

[simon-clematide]

@mromanello There are tokens with length 0 and even with negative values -1.
What does this encode?

e.g. GDL-1799-11-20-a-i0004 {'c': [1203, 2081, 14, 42], 's': 617, 'l': -1}
or GDL-1799-12-22-a-i0003 {'c': [1150, 1031, 28, 33], 's': 918, 'l': 0}

[mromanello]

The meaning of these fields is as described in the schema.

But the length (l) of a tokens should not have values < 1. From a quick look at the code I believe it's a bug having to do with the hyphenation. I close it here (it does not relate to the schema but to the data) and reopen it there where it belongs.

Thanks for spotting it!