Transform tokens + Minor bugfixes
Lark transformers can now visit tokens as well. Use like this:
class MyTransformer(Transformer):
def TOKEN1(self, tok):
return tok.upper()
def rule_as_usual(self, children):
return children
MyTransformer(visit_tokens=True).transform(tree)
Fixed a few regressions that I accidentally added to 0.7.4