Skip to content

Graph Queries

Jump to bottom
zalky edited this page May 1, 2023 · 6 revisions

There is a single function reflet.core/reg-pull for defining reactive graph queries. This function supports a subset of the EQL spec, with union queries replaced by the Reflet's polymorphic descriptions. It also supports a couple of other extensions beyond EQL. Much of this will be familiar to anyone who has used Datomic pull syntax.

To summarize, reg-pull supports the following operations:

reg-pull

Recall the graph music catalog from the Multi Model DB document:

{::db/data
 {[:system/uuid #uuid "a"]  {:system/uuid      #uuid "a"
                             :kr/name          "Miles Davis"
                             :kr.artist/albums #{[:system/uuid #uuid "b"]}}
  [:system/uuid #uuid "b"]  {:system/uuid     #uuid "b"
                             :kr/name         "Miles Smiles"
                             :kr.album/artist [:system/uuid #uuid "a"]
                             :kr.album/tracks [[:system/uuid #uuid "t1"]
                                               [:system/uuid #uuid "t2"]]}
  [:system/uuid #uuid "c"]  {:system/uuid #uuid "c"
                             :kr/name     "Wayne Shorter"}
  [:system/uuid #uuid "t1"] {:system/uuid       #uuid "t1"
                             :kr/name           "Orbits"
                             :kr.track/artist   [:system/uuid #uuid "c"]
                             :kr.track/album    [:system/uuid #uuid "b"]
                             :kr.track/duration 281
                             :kr/uri            "/audio/Miles Smiles - Orbits.mp3"}
  [:system/uuid #uuid "t2"] {:system/uuid       #uuid "t2"
                             :kr/name           "Footprints"
                             :kr.track/artist   [:system/uuid #uuid "c"]
                             :kr.track/album    [:system/uuid #uuid "b"]
                             :kr.track/duration 596
                             :kr/uri            "/audio/Miles Smiles - Footprints.mp3"}}}

Select an Attribute Set

The following query selects 4 attributes from the album specified by album-ref:

(f/reg-pull ::album
  (fn [album-ref]
    [[:system/uuid
      :kr/name
      :kr.album/artist
      :kr.album/tracks]
     album-ref]))

The reg-pull handler always returns a vector of one or two elements. The first element is always a pull spec. Here it is a vector of 4 attributes. The second element, if provided, is the root entity reference from which the query begins its data graph traversal.

(f/reg-pull ::query
  (fn [root-ref]
    [pull-spec
     root-ref]))

Besides the query syntax, there are two other important differences from regular re-frame subscriptions:

  1. The query handler does not depend on the db value. Instead it receives one or more arguments, usually references, which are used to construct the query.

  2. The body of the query handler returns a declarative query expression, unlike a re-frame subscription handler which imperatively computes the query result from its data dependencies.

This has a number of benefits:

  1. How often a query runs is now an implementation detail. With regular re-frame subscriptions, each time you write a handler, you have to explicitly choose the reactive dependency on all or part of the data to get the most efficient query. Did you choose the most specific subpath in the db? Is your subpath too general? You must design every handler correctly, or it will run more than it should. With reg-pull, the implementation decides when queries are run, so you can't make a mistake.

  2. The declarative pull specification is a visual reflection of the shape of your data: it is self-describing. With a regular re-frame subscription handler, you often have to parse imperative code to infer the shape of the data.

With your query defined, you can subscribe to the result in your components:

(defn component
  [{:keys [album-ref]
    :as   props}]
  (let [album @(f/sub [::album album-ref])]
    [:div 
     [:div (:kr/name album)]
     ...]))

Recall reflet.core/sub is just an alias for re-frame.core/subscribe. Subscribing to reg-pull will return a regular re-frame subscription object, where all the regular subscription semantics, like caching and automatic disposal, apply.

The value returned by the dereferenced example subscription is:

{:system/uuid     #uuid "b"
 :kr/name         "Miles Smiles"
 :kr.album/artist [:system/uuid #uuid "a"]
 :kr.album/tracks [[:system/uuid #uuid "t1"]
                   [:system/uuid #uuid "t2"]]}

Joins and Nested Data

As we can see, the above query spec did not traverse beyond the graph joins. To do so, we introduce a join expression to the pull syntax:

(f/reg-pull ::album
  (fn [album-ref]
    [[:system/uuid
      :kr/name
      {:kr.album/artist [:kr/name]}
      {:kr.album/tracks [:system/uuid
                         :kr/name
                         :kr.track/duration]}]
     album-ref]))

Here, the joins at both :kr.album/artist and :kr.album/track are resolved, and the associated track data is returned in denormalized form:

{:system/uuid     #uuid "b"
 :kr/name         "Miles Smiles"
 :kr.album/artist {:kr/name "Miles Davis"}
 :kr.album/tracks [{:system/uuid       #uuid "t1"
                    :kr/name           "Orbits"
                    :kr.track/duration 281}
                   {:system/uuid       #uuid "t2"
                    :kr/name           "Footprints"
                    :kr.track/duration 281}]}

Note that the cardinality and type of the join attributes :kr.album/artist and :kr.album/tracks is preserved with respect to the input data. The album had a single artist, and a vector of tracks. Remember there is no DB schema, so cardinality and type are runtime dependent.

Select a Single Attribute

Quite often you only want a single attribute value from your root entity. This is not part of EQL, but because it is so common, Reflet provides an extension. To get just the name of the above album, use a single attribute :kr/name instead of a vector as the top level pull spec:

(f/reg-pull ::album
  (fn [album-ref]
    [:kr/name album-ref]))

This will will return:

"Miles Smiles"

It also works with joins. To return the :kr.album/tracks of the album just do:

(f/reg-pull ::album
  (fn [album-ref]
    [{:kr.album/tracks [:system/uuid
                        :kr/name
                        :kr.track/duration]}
     album-ref]))

Which will return:

[{:system/uuid       #uuid "t1"
  :kr/name           "Orbits"
  :kr.track/duration 281}
 {:system/uuid       #uuid "t2"
  :kr/name           "Footprints"
  :kr.track/duration 281}]

As always the cardinality and type of the data is preserved.

To summarize, the root pull spec is always either a:

  • vector: select an attribute set
  • keyword: select a single attribute
  • map: select a single attribute but traverse the join

Select a Link Entry

Recall the data from the link entry example in the Multi Model DB document:

{::db/data
 {:active/user                [:system/uuid #uuid "user"]   ; <- cardinality one join
  [:system/uuid #uuid "user"] {:system/uuid #uuid "user"    ; <- merged in user
                               :kr/name     "Name"
                               :kr/role     "user"}}}

There is a natural extension to selecting a single attribute that applies to link entries. If you do not provide a root reference, then the db is taken as the start of the graph traversal, and then selecting a single attribute is effectively a link query:

(f/reg-pull ::active-user-link
  (fn []
    [:active/user]))   ; <- single attribute with no root reference

The above will return the link entry at :active/user:

[:system/uuid #uuid "user"]

As before, to actually traverse the join, use the join syntax with a single attribute selection, but this time exclude the root reference:

(f/reg-pull ::active-user-link
  (fn []
    [{:active/user    ; <- single attribute /w join
      [:system/uuid   ; <- nested select of attribute set
       :kr/name
       :kr/role]}]))
                      ; <- no root reference, so db is the "entity"

This will return:

{:system/uuid #uuid "user"
 :kr/name     "Name"
 :kr/role     "user"}

Wildcard Queries

To pull all the attributes of an entity, use the quoted '* symbol in an attribute set:

(f/reg-pull ::album
  (fn [album-ref]
    [[:system/uuid
      :kr/name
      {:kr.album/label [:kr/name]}
      {:kr.album/tracks ['*]}]
     album-ref]))

The above will pull all the attributes of each track. You can also combine a wildcard with other keywords in the attribute set:

(f/reg-pull ::album
  (fn [album-ref]
    [[:system/uuid
      :kr/name
      {:kr.album/label [:kr/name]}
      {:kr.album/tracks ['* {:kr.track/artist [:kr/name]}]}]
     album-ref]))

The above will pull all of the attributes of each track, and traverse the join of each :kr.track/artist attribute to get the artist name.

Infinite and Recursive Queries

Sometimes you need to query a graph data model that represents some recursive, tree-like structure. Here, recursive queries can save a lot of boilerplate:

(f/reg-pull ::recursive-infinite
  (fn [ref]
    [[:system/uuid
      :kr.node/name
      :kr.node/description
      {:kr.node/child '...}]
     ref]))

When the quoted ellipsis '... is used as the value in a join expression, it indicates that the currently scoped attribute set should be used to process all nested nodes in the traversal. Note that only the current attribute set participates in the recursion:

(f/reg-pull ::recursive-infinite
  (fn [ref]
    [[:kr/higher
      :kr/context
      {:kr/root-nodes [:system/uuid
                       :kr.node/name
                       :kr.node/description
                       {:kr.node/child '...}]}]
     ref]))

Here, the [:kr/higher :kr/context :kr/root-nodes] attribute set is not part of the pattern.

Of course the result and performance of this kind of recursive graph traversal very much depends on the shape of the data. Your query could potentially span the entire db, or even fail to terminate if there are loops in your graph structure.

So Reflet also provides a limited recursive query spec. Just replace the ellipsis ... with the maximum number of recursions you wish to allow:

(f/reg-pull ::recursive-2
  (fn [id]
    [[:kr/name
      :kr/description
      {:kr/join 2}]
     id]))

Here we allow at maximum two recursions before the query traversal stops.

Side Effects and Data Synchronization

While EQL supports mutations, the syntax is not flexible enough to support Reflet's extended query grammar. Instead, Reflet leverages EQL's parameterized query syntax, to implement query side-effects.

Any sub-expression within a pull structure can be wrapped with query parameters (sub-expr params):

(f/reg-pull ::album
  (fn [album-ref other-ref]
    [[:kr/name
      {:kr.album/tracks ([:system/uuid
                          :kr/name
                          :kr.track/duration]
                         {:id       ::my-pull-fx-method
                          :my-param other-ref})}]
     album-ref]))

Here, the sub-expression in the :kr.album/tracks join has been wrapped with a list (), and a map of parameters. These parameters are parsed by the reflet.core/pull-fx multimethod, which dispatches on the :id in the map. A warning will be issued if no handler has been defined for the given :id.

You can define your own pull-fx handler:

(defmethod f/pull-fx ::my-pull-fx-method
  [params context]
  (let [{:keys [id my-param]} params
        {:keys [db ref expr]} context]
    (do-some-side-effect ...)))

The multimethod function accepts both the params from the query expression, as well as the query execution context. The context map will contain both a :ref to the current node in the graph traversal, the sub query expression :expr that remains to be evaluated, and the immutable value of the db against which the query is being executed. For the ::album query example above, the :refwould be an entity reference to a track entity, and :expr would be:

[:system/uuid
 :kr/name
 :kr.track/duration]

If the :kr.album/tracks join resolves to cardinality many tracks, then a separate pull-fx call is made for each track entity.

Using this approach, the body of the pull-fx could then dispatch a synchronization event, or some other side-effect, the details of which are left up to user code.

But remember: the pull effects are called every time a query is run. So you should always keep them as simple as possible, ideally no more than dispatching a Re-frame event.

Result Functions

When you subscribe to reg-pull, it returns a regular subscription object. Just like other subscriptions, these can participate in layer-3 Re-frame subscriptions:

(require '[re-frame.core :as f*]
         '[reflet.core :as f])

(f/reg-pull ::track-duration
  ;; This single attribute query gets the track duration in numeric seconds
  (fn [track-ref]
    [:kr.track/duration track-ref]))

(f*/reg-sub
  ;; Layer-3 Re-frame sub
  (fn [[_ track-ref]]
    (f/sub [::track-duration track-ref]))
  (fn [secs]
    ;; Convert numeric seconds to a HH:mm:ss string representation
    (when (number? secs)
      (->> (-> secs
               (* 1000)
               ( js/Date.)
               (.toISOString)
               (.slice 11 19))
           (drop-while #{\0 \:})
           (apply str)))))

Because this kind of post-processing so common, Reflet allows you to append a result function to the reg-pull definition:

(f/reg-pull ::track-duration
  (fn [track-ref]
    [:kr.track/duration track-ref])
  (fn [secs]
    ;; Convert numeric seconds to a HH:mm:ss string representation
    (when (number? secs)
      (->> (-> secs
               (* 1000)
               ( js/Date.)
               (.toISOString)
               (.slice 11 19))
           (drop-while #{\0 \:})
           (apply str)))))

The main benefit of using a result function, aside from saving some boilerplate, is that they are traced along-side with their input queries in the Reflet debugger. Regular layer-3 subscriptions are not.

The graph query and the result function are run in separate reactions, and the combined pathway is returned and cached against the invoking subscription vector:

@(f/sub [::track-duration track-ref])

Which should return something like:

"4:31"

The first argument of the result function is always the result of the input query, but if you need it, the result function additionally accepts all of the arguments to the input query as context:

(f/reg-pull ::my-quer
  (fn [ref arg1 arg2]
    [[:attr-1 :attr-2 :attr-3]
     ref])
  (fn [result ref arg1 arg2]     ; <- Result + args from the input sub
    (do-something ...)))

Non-reactive Queries in Event Handlers

Often you will want to query graph data in event handlers. Because you cannot use reactive subscriptions in event handlers, three non-reactive, pure query functions are provided for this purpose.

reflet.db/getn and reflet.db/get-inn return graph data, with semantics similar to clojure.core/get and clojure.core/get-in. Importantly, they do not resolve entity references.

Assuming the following db:

{::db/data
 {[:system/uuid #uuid "a"]  {:system/uuid      #uuid "a"
                             :kr/name          "Miles Davis"
                             :kr.artist/albums #{[:system/uuid #uuid "b"]}}
  [:system/uuid #uuid "b"]  {:system/uuid     #uuid "b"
                             :kr/name         "Miles Smiles"
                             :kr.album/artist [:system/uuid #uuid "a"]
                             :kr.album/tracks [[:system/uuid #uuid "t1"]
                                               [:system/uuid #uuid "t2"]]}}}

Then:

(getn db [:system/uuid #uuid "a"])

Returns the whole entity:

{:system/uuid      #uuid "a"
 :kr/name          "Miles Davis"
 :kr.artist/albums #{[:system/uuid #uuid "b"]}}

While:

(get-inn db [[:system/uuid #uuid "a"] :kr/name])

Returns just the value:

"Miles Davis"

Full pull semantics, excluding side-effects are available via the non-reactive, pure reflet.db/pull:

(db/pull db
         [:system/uuid
          :kr.generic/name
          {:kr.artist/albums [:kr/name]}]
         [:system/uuid #uuid "a"])

This would return:

{:system/uuid      #uuid "a"
 :kr/name          "Miles Davis"
 :kr.artist/albums #{{:kr/name "Miles Smiles"}}

Gotcha: Unintentional Entity Unification

Take care which attributes you select in joins when the join type is a cardinality many set.

Consider this example graph data containing a location map and some geo data points:

{[:system/uuid 1] {:kr/type       :kr.type/map
                   :kr/title      "Demo Location Map"
                   :kr.map/points #{[:system/uuid 2]
                                    [:system/uuid 3]
                                    [:system/uuid 4]}}
 [:system/uuid 2] {:system/uuid     2
                   :kr/type         :kr.type/point
                   :kr.geo/latitude 40
                   :kr.geo/latitude 140}
 [:system/uuid 3] {:system/uuid     3
                   :kr/type         :kr.type/point
                   :kr.geo/latitude 44
                   :kr.geo/latitude 141}
 [:system/uuid 4] {:system/uuid     4
                   :kr/type         :kr.type/point
                   :kr.geo/latitude 40
                   :kr.geo/latitude 140}}

Notice that the :kr.map/points resolves to a set of entities. Also notice that two of the referenced entities share the same type, latitude, and longitude. A query that pulls the map and all its points might look something like:

(f/reg-pull ::state
  (fn [ref]
    [[:kr/title
      {:kr.map/points [:kr/type
                       :kr.geo/latitude
                       :kr.geo/longitude]}]
     ref]))

However, as specified, this query would not produce a set of unique point results. The two points [:system/uuid 2] and [:system/uuid 4] share all the pulled attributes, and by set logic would unify to produce:

{:kr/title      "Demo Map"
 :kr.map/points #{{:kr/type         :kr.type/point  ; <- Both [:system/uuid 2] and [:system/uuid 4]
                   :kr.geo/latitude 40
                   :kr.geo/latitude 140}
                  {:kr/type         :kr.type/point
                   :kr.geo/latitude 44
                   :kr.geo/latitude 141}}}

This is probably not the desired result, since we would expect three points.

The solution is to ensure the unique identifier is included in the join spec, even if the consuming code does not use it. In reality the unique ids are almost always useful as React keys. But more importantly this guarantees that the results do not unintentionally unify:

(f/reg-pull ::state
  (fn [ref]
    [[:kr/title
      {:kr.map/points [:kr/type
                       :system/uuid     ; <- unique attribute will prevent unification
                       :kr.geo/latitude
                       :kr.geo/longitude]}]
     ref]))
{:kr/title      "Demo Map"
 :kr.map/points #{{:system/uuid     2
                   :kr/type         :kr.type/point
                   :kr.geo/latitude 40
                   :kr.geo/latitude 140}
                  {:system/uuid     3
                   :kr/type         :kr.type/point
                   :kr.geo/latitude 44
                   :kr.geo/latitude 141}
                  {:system/uuid     4
                   :kr/type         :kr.type/point
                   :kr.geo/latitude 40
                   :kr.geo/latitude 140}}}

This same consideration does not apply to vectors or lists, because of the nature of the collection type.

Next: Polymorphic Descriptions

Home: Home

Reflet

Features

Other

Clone this wiki locally