[FEATURE] lightweight user-level concurrency

[gvwilson]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem?

Many languages now provide a lightweight user-level concurrency mechanisms, from fibers in Wren to goroutines in Go or async callbacks in JavaScript (the latter being the least usable of the three). Adding this to Dictu would enable instructors to teach concurrency.

Describe the solution you'd like

Co-operative concurrency (e.g., fibers in Wren) is straightforward to implement and will be familiar to most instructors.

Describe alternatives you've considered

1. JavaScript-style callbacks and/or promises have proven very hard to teach.
2. Linda-style tuple spaces are easy to teach and understand, but the pattern matching would require larger changes to Dictu.
3. Software Transactional Memory (STM) is easier to understand than locks or monitors (https://neverworkintheory.org/2011/08/24/is-transactional-programming-actually-easier.html) but again, would require larger changes to the VM and language than fibers.

Additional context

I'm looking for a small teaching language (think Pascal in the early 80s, or Python in the early 2000s); Dictu seems like a very promising candidate. I'm extracting requirements from https://third-bit.com/sdxjs/ and https://third-bit.com/sdxpy/.

[Jason2605]

This is something that i've been putting off in Dictu but in the back of my mind knowing I should not have 😄. I toyed with looking into libuv (#481) which would essentially just be a thin wrapper around the library, but as you say, will end up being quite verbose and definitely lead to callback hell.

I think fibers are probably the way to go but it does mean that users will have to roll their own event loops. We will also need to take care to add things that can allow us to be non-blocking (such as non-blocking sockets and things like socket.select() so that we can happily wait). It would also be interesting to see how this would work with cURL (as thats the foundation of the HTTP module).

tl;dr - I have thought about it, and I'm open to suggestions in the way we go. Wrens fibers will most likely be the easiest to implement as Dictu internal code is quite similar to Wren internal code (essentially since the core was written by the same person).

I'm extracting requirements from https://stjs.tech/, and would be happy to chat: https://third-bit.com/ for contact info.

This sounds great!! If there's anything I can do to help you with this let me know! The only thing I think which could potentially be an issue is the surrounding tooling for Dictu is almost non-existing. So think debuggers, IDE / Text editor extensions (apart from a very limited VSc extension), package managers, etc.

[gvwilson]

Bundle an event loop with the Dictu interpreter for the non-embedded case, but provide a hook so that embedded uses can provide their own functions? (I think that relying on libuv's event loop would make building a single-step/breakpoint debugger for Dictu harder, but I haven't tried, so that's just a guess.)

[Jason2605]

Yeah the event loop isn't really going to be complicated in user-land code (I don't think), it just may get a bit tedious having to implement it each time. Something provided with the interpreter would probably be useful, but I think getting Fibers into the language would be a good start!

Lua example for reference: https://www.lua.org/pil/9.4.html

[Jason2605]

Some helpful repositories:

• https://github.com/wren-lang/wren
• https://github.com/ThakeeNathees/pocketlang

We should go with implementing Fibers into the language

[briandowns]

I was going through the feature set of Wren and Pocketlang and Dictu is far more extensive in terms of features and capabilities. The one thing that's lacking is concurrency. With that, Fibers seems like an incredibly good choice and with that would position Dictu as the leader in this language space.

[Jason2605]

Yeah concurrency has something that has been in the back of my mind for a while, but something I've also just never got round to doing. The implementation in Wren will be very useful as the internals of Wren and the internals of Dictu are pretty similar. Having some sort of concurrency model is also one of the things in the back of my mind that is keeping the Dictu version < 1 as well.

Definitely one we're missing and will need to be done at some stage!

[Ezbob]

What about actors as an concurrency model?
https://en.wikipedia.org/wiki/Actor_model
Actors are basically classes where each instance run in it's own thread and can send messages to each other to share data. I know that this is the model that ponylang bets on.

You could implement the actor as a builtin type or as a library base class, that other classes can inherit from.

[Jason2605]

@briandowns I think in terms of priorities this one should be pretty high on the list. Concurrency is a major part that is missing from Dictu and "Fibers" implemented (similar to wren) would be a major milestone

[briandowns]

I agree. Any thoughts for the API? I like the idea of Fibers but I've been also thinking about the way Go handles concurrency in its runtime and was curious your thoughts around something similar.

[Jason2605]

I'd honestly be pretty happy with something that is pretty similar to Wren https://wren.io/concurrency.html. I think this will definitely be the easiest route for implementation, we'd just need to make sure that some of the built-ins have non-blocking variants (e.g sockets / http lib etc)

[gvwilson]

Updated initial issue post with links to books teaching software design in JS and Python that motivated the original request; happy to chat any time about requirements for teaching.

[Jason2605]

Thanks @gvwilson! Sorry the issue is as old! Will definitely give those a read and get back to you