Introduce the bruteforce parser (interpreter?). #10

eddyb · 2019-07-23T09:15:46Z

Check out src/lyg.rs for an example involving pattern-matching parse nodes, and the two tests (ported over from the gll crate) for how to use the interpret and what the output is like.

I'm open to suggestions on the API and formatting, it's a bit arbitrary at the moment.

CAD97

Everything looks fine. Obviously src/slow_bruteforce_interpreter.rs is the tricky part.

src/lyg.rs

eddyb · 2019-07-23T15:56:53Z

I think I'd like this to bake a while longer, at least to get to test it with wg-grammar and maybe try out some imperative disambiguation? Could result in a wildly different handle API.

@oli-obk also expressed interest in experimenting with the "web frontend" idea.

EDIT: some initial testing with wg-grammar results in a larger slowdown than expected, will investigate.
EDIT2: oops, that was a limitation in cyclotron, got 1000x speedup (or more) on some files.

src/slow_bruteforce_interpreter.rs

Force push: need to reread

CAD97

Generally: Everything outside slow_bruteforce_interpreter.rs looks good to me so far. I think some of the FIXME in that file can be pretty easily addressed, though. (Generally, it looks fine, but I've only really scanned that file so far.)

CAD97

Everything looks good here

src/slow_bruteforce_interpreter.rs

eddyb · 2019-10-10T16:51:27Z

tests/basic.rs

+    };
+}
+
+testcases![


Need to sync these with gll, additions were made there.

eddyb · 2019-10-10T16:52:26Z

src/rule.rs

-            Rule::RepeatMore(rule, None) => {
-                NodeShape::Split(rule, cx.intern(Rule::RepeatMany(rule, None)))
+            Rule::RepeatMany(..) | Rule::RepeatMore(..) => {
+                NodeShape::Alias(self.expand_repeats(cx))


This needs to be reverted/replaced with self.expand_repeats(cx).node_shape(cx, named_rules).

eddyb · 2019-10-10T16:53:10Z

src/forest.rs

@@ -304,6 +309,93 @@ impl<'i, G: GrammarReflector, I: Input> ParseForest<'i, G, I> {
    }
 }

+// TODO(eddyb) remove this entirely, only user of it left is `ListHandle`.


This needs to be dealt with ASAP.

eddyb · 2019-10-10T17:00:44Z

src/bruteforce.rs

+                        }
+                    }
+                }
+            }


So this is the nastiest part of parsing atm, an emulated stack to avoid blowing up the real one.

(Using a queue instead of a stack looks too much like GLL for my own comfort, otherwise I would've probably done it that way)

I'm wondering how I can make this more readable - maybe keep more things explicitly on the emulated stack? Use an enum with variants as an explicit state machine?

Cargo.toml

eddyb · 2021-03-17T19:37:18Z

src/bruteforce.rs

+#[derive(Debug)]
+enum SmallSet<T> {
+    One(T),
+    Many(BTreeSet<T>),
+}


Might be cheaper to use e.g. im::OrdSet for this.

eddyb force-pushed the interpret branch from 313de37 to af0c15e Compare July 23, 2019 09:24

eddyb mentioned this pull request Jul 23, 2019

what's gll? rust-lang/gll#130

Open

CAD97 previously approved these changes Jul 23, 2019

View reviewed changes

src/lyg.rs Outdated Show resolved Hide resolved

eddyb commented Jul 24, 2019

View reviewed changes

src/slow_bruteforce_interpreter.rs Outdated Show resolved Hide resolved

eddyb force-pushed the interpret branch from af0c15e to d684081 Compare July 25, 2019 14:25

eddyb mentioned this pull request Jul 27, 2019

rule: intern fields and store them in a tree matching the rule structure. #11

Merged

eddyb force-pushed the interpret branch from d684081 to 42ca2e1 Compare July 27, 2019 14:30

eddyb mentioned this pull request Aug 25, 2019

Assorted ParseError-related fixes. #12

Merged

eddyb force-pushed the interpret branch from 42ca2e1 to 6f223dc Compare August 25, 2019 03:52

eddyb force-pushed the interpret branch from 6f223dc to d5cc009 Compare September 3, 2019 13:44

CAD97 reviewed Sep 4, 2019

View reviewed changes

eddyb force-pushed the interpret branch from d5cc009 to ff06bb8 Compare October 4, 2019 19:53

CAD97 approved these changes Oct 6, 2019

View reviewed changes

src/slow_bruteforce_interpreter.rs Outdated Show resolved Hide resolved

eddyb force-pushed the interpret branch 2 times, most recently from 1d4f1f5 to 2a3bfe1 Compare October 10, 2019 16:48

eddyb changed the title ~~Introduce the slow_bruteforce_interpreter.~~ Introduce the bruteforce parser (interpreter?). Oct 10, 2019

eddyb commented Oct 10, 2019

View reviewed changes

tests/basic.rs

};

}

testcases![

Copy link

Member Author

eddyb Oct 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to sync these with gll, additions were made there.

eddyb commented Oct 10, 2019

View reviewed changes

eddyb force-pushed the interpret branch 3 times, most recently from 9f32647 to baec49d Compare October 19, 2019 13:28

Introduce the bruteforce parser (interpreter?).

40f19db

eddyb force-pushed the interpret branch from baec49d to 40f19db Compare October 19, 2019 13:48

aleksmelnikov reviewed Nov 9, 2019

View reviewed changes

Cargo.toml Show resolved Hide resolved

eddyb commented Mar 17, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce the bruteforce parser (interpreter?). #10

Introduce the bruteforce parser (interpreter?). #10

eddyb commented Jul 23, 2019 •

edited

Loading

CAD97 left a comment

eddyb commented Jul 23, 2019 •

edited

Loading

CAD97 left a comment

CAD97 left a comment

eddyb Oct 10, 2019

eddyb Oct 10, 2019

eddyb Oct 10, 2019

eddyb Oct 10, 2019 •

edited

Loading

eddyb Mar 17, 2021

Introduce the bruteforce parser (interpreter?). #10

Are you sure you want to change the base?

Introduce the bruteforce parser (interpreter?). #10

Conversation

eddyb commented Jul 23, 2019 • edited Loading

CAD97 left a comment

Choose a reason for hiding this comment

eddyb commented Jul 23, 2019 • edited Loading

CAD97 left a comment

Choose a reason for hiding this comment

CAD97 left a comment

Choose a reason for hiding this comment

eddyb Oct 10, 2019

Choose a reason for hiding this comment

eddyb Oct 10, 2019

Choose a reason for hiding this comment

eddyb Oct 10, 2019

Choose a reason for hiding this comment

eddyb Oct 10, 2019 • edited Loading

Choose a reason for hiding this comment

eddyb Mar 17, 2021

Choose a reason for hiding this comment

eddyb commented Jul 23, 2019 •

edited

Loading

eddyb commented Jul 23, 2019 •

edited

Loading

eddyb Oct 10, 2019 •

edited

Loading