build skeleton of end-to-end toolchain for next-generation L4 DSL through to REST API

[mengwong]

I have in mind something like

graph TD;
  DX["Developer"] --> IDE
subgraph IDE
  L4P("L4 'Program'")
  Infra
end
subgraph Application
  appServer["application API server"]
  RoyalFlush
end
  RoyalFlush --"orchestrator"-->JSClient

  L4P --> L4C("L4 compiler\nLam4?")
  L4C --"outputs"--> SimalaCode("Simala Code / AST")
  Infra --"ships"-->SimalaCode
  Infra --"to (PUT)"-->Servant
  JSClient["JS Client Lib"] --"JSON payload"--> asOIA
  Servant["powered by Servant"]
  Servant --"calls"--> SimalaRuntime["Simala Runtime"]
  SimalaRuntime --"loads previously PUTted"-->SimalaCode
  asOIA("protocol-compatible\nwith Oracle Intelligent Assistant\nBatch Assess API") -->Servant
  otherDE("other decision engine API") --> Servant
  forGPT("tuned for ChatGPT Actions API") -->Servant
  EU["End User"] --> ChatGPT
  ChatGPT --"Actions API?"--> forGPT

classDef highlight2 fill:#f9f,stroke:#333,stroke-width:3px;
classDef highlight1 fill:#9ff,stroke:#333,stroke-width:3px;
class L4C,BRE,Servant,RoyalFlush,asOIA,SimalaCode,SimalaRuntime highlight2;
class L4P,Infra,JSClient highlight1;

Loading

I'd like us to focus on getting the highlighted path working initially, for purposes of a canned demo.

PLAYGROUND DEMO

I start by talking to ChatGPT and asking a question that runs against the Servant API. I get a response.

I say, "hang on, that's not quite right. I need to edit that rule."

I open a browser tab to a web-based IDE, where the text of an L4 program is shown.

Ideally, that web-based IDE is not the Natural4 spreadsheet, but something based on Simala or Lam4.

I edit the text of the program, and hit save.

The edit will be very trivial, like, (walks && (eats || drinks)) becomes (walks || (eats && drinks))

That kicks off the highlighted toolchain, resulting in a refresh to the API server.

I go back to ChatGPT and repeat the question, and see that the answer has changed due to the change in logic.

ROYALFLUSH DEMO:

I use the IDE to edit the business logic of the RoyalFlush app.

After tweaking the rules, I go back to the RoyalFlush web app and upload a new PDF; the new logic runs on that PDF.

The IDE saves a new Lam4 program to the RoyalFlush backend using the saveOverrides API. YM is starting to see what Meng had in mind; YM is now leaning towards Meng's original proposal to just have the IDE update the decision service with the new program directly.

After the saveOverrides API saves the new Lam4 program, the RoyalFlush server saves the program, and PUT uploads it to the Servant backend which overwrites the program.

OUT OF SCOPE:
The edit is less trivial; I change the input terms to include dances.
I go back to ChatGPT and repeat the question, and need to give dances as an input.
This is out of scope because rebuilding the schema for OpenAI in an automated way is not working on their end right now.

We see that the answer to the question has changed, because the L4 program has changed.

future work:

graph TD;
  L4C --> Viz["Visualizations"]
  L4C --> DMN("DMN")
  L4C --> BPMN("BPMN")
  Viz --> Layman("Layman Allen? from sandbox/mengwong/layman")
  Viz --> Petri("Petri Net")
  Viz --> DFA("Others e.g. DFA?")
  Viz --> BDD("Binary Decision Diagram")
  Viz --> Ladder("Ladder Diagram")

Loading

[mengwong]

Concretely, that means that the currently-hardcoded compute_qualifies function will need to be integrated with the upstream toolchain and made dynamically refreshable.

dsl/lib/haskell/explainable/app/Server.hs

Lines 493 to 499 in fe2b758

	-- | Example function which computes whether a person qualifies for *something*.
	personQualifies :: Expr Double
	personQualifies =
	"qualifies"
	@|= MathPred
	( getvar "walks" |&& (getvar "drinks" ||| getvar "eats")
	)

dsl/lib/haskell/explainable/app/Server.hs

Line 573 in fe2b758

personQualifiesFunction :: Function

[mengwong]

In turn, that means we need to swap out the existing MathLang stuff and perhaps drop Simala in its place?

[ym-han]

I've been thinking of something like this for a while now, yeah, except that I've been thinking of the chat bot involvement more as a copilot than the servant api. With Langium / Lam4 (and of course with any other implementation that has a LSP), one can do the Yscript-style visual linking too

[mengwong]

Other things to look at:

OIA Batch API

https://docs.oracle.com/en/cloud/saas/b2c-service/opawx/using-batch-assess-rest-api.html

https://documentation.custhelp.com/euf/assets/devdocs/cloud19b/PolicyAutomation/en/Default.htm#Guides/Developer_Guide/DeterminationsAPI/Automate_decisions/Get_started_assess_service.htm

client to the OIA

https://github.com/smucclaw/royalflush/blob/main/experiments/meng/oiacall.py

existing Servant server

https://github.com/smucclaw/dsl/blob/main/lib/haskell/explainable/app/Server.hs

swagger docs generated by Servant

https://olive.well-typed.com/swagger-ui/#/default/runComputeQualifies

the input to be translated to lam4

• in word docx
  • https://github.com/smucclaw/royalflush/tree/main/oia-rules
• parsed to Haskell prolog-like AST
  • https://github.com/smucclaw/sandbox/blob/default/mengwong/parse-oia/business-status.hs
  • https://github.com/smucclaw/sandbox/blob/default/mengwong/parse-oia/system-rules-simple.hs

[ym-han]

The IDE saves a new Lam4 program to the RoyalFlush backend using the saveOverrides API.

I had been resisting the proposal to have the IDE update the decision service backend directly because I had been thinking of the IDE as a more generic 'playground' component, but I'm starting to see see how we might think of this direct updating of the Servant backend as a 'cloud integration' component of the IDE. I'm starting to lean towards that. I will update the diagram if I end up going with that.

[mengwong]

i revised the diagram slightly to highlight the parts that we are showing in today's internal tech demo.