Add persistent implementation of dataset::MutableDataset

[pchampin]

In addition to the dataset::inmem module, it would be nice to have a disk-based persistent implementation of dataset::MutableDataset.

[pchampin]

One way to do it could be to use RockDB. We could even try and use the same layout as used by Oxigraph, making it possible to share the same storage across both crates. @Tpt, what do you think? Is that layout documented somewhere?

[Tpt]

That would be great to make Sophia works with Oxigraph storage! The Oxigraph RockDB layout is not stable yet, I am currently tweaking it to make it a bit more compact and allow efficient range queries. I hope to have time finishing a 0.1 Oxigraph release with a stable RocksDB layout in late December or (more realistically) January.

The basic storage approach should not change: I store in RocksDB keys rotations of quads of EncodedTerm and a string store. RocksDB prefix searches are then used to solve triple patterns. The string store is used as an inverse hash lookup, the strings being hashed inside of EncodedTerm. Hashing strings is very useful for heavy SPARQL query evaluations with a lot of joins, it might not be the best approach for just storing and doing simple triple pattern evaluation.

But, I'm not sure that reimplementing Oxigraph storage in Sophia is the best way to do it. I fear than very quickly you might want also to be able to run SPARQL queries on top of it in Sophia, completely duplicating the work already done in Oxigraph. A better way to go would probably to make Oxigraph usable with Sophia.

[pchampin]

A better way to go would probably to make Oxigraph usable with Sophia.

Yes, implementing Sophia's traits above Oxigraph is also a way to go, and probably the fastest one. I would be concerned, though, that converting from Oxigraph's model to Sophia's would induce some overhead, hence my initial proposal... But definitely worth a try, anyway.

[pchampin]

Not published on crates.io yet, but an adapter for Oxigraph is now available at https://github.com/pchampin/sophia_oxigraph.

[pchampin]

Just pushed a PR on Oxigraph to make it implement the relevant Sophia traits (behind a feature gate). Oxigraph could therefore serve as a reference implementation of a persistent dataset.