-
Notifications
You must be signed in to change notification settings - Fork 1
Current Status 03 03 2015
Hi all, I'm just giving a bit of an info / status dump after group chat today so everyone is on the same page.
https://github.com/HeriotWattMEng2015/ShEx-validator
This is a standalone Node module which parses a ShEx schema and RDF data file and validates the data against the schema. It can be used as a client-side JavaScript library with JavaScript callbacks once Browserifyed, and can also be run directly from the command line. As it uses N3.js to parse the data, it supports data in Turtle, TriG, N-Triples or N-Quads serializations. Already mostly functional, ongoing work on this includes adding simple english explanations of validation errors to the callback response objects, even if those messages are not necessarily used directly by the UI.
https://github.com/HeriotWattMEng2015/ShEx-validator/tree/master/specs
Unit tests for parsing and validation, written in Jasmine. So far there are 4. We probably ought to have 10 times that, given the complexity of both ShEx and the various RDF syntaxes.
The standard ShEx syntax currently supports cardinality (using * and + symbols) and some variants (read: our variant) support negation (using the ! symbol). However, the W3C (http://www.w3.org/wiki/RfcKeywords) actually recommends the keywords defined in IETF RFC 2119 (http://www.ietf.org/rfc/rfc2119.txt) when writing a spec.
These 10 keywords are as defined: "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", "OPTIONAL"
We have decided to modify our variant of ShEx to support storing these requirement levels as a string in the ShEx schema, marked by backticks (e.g. `SHOULD` ). This will allow us to have a single ShEx schema which stores these semantic keywords about shapes without modifying the basic validation functionality. The validator will then be able to display optional warnings or errors if these semantic keywords are not met, on top of the basic cardinality and negation rules in the schema.
https://github.com/HeriotWattMEng2015/HCLSDatasetDescriptions/tree/shex
This task basically encompasses "taking the latest version of the HCLS dataset description from Overview.html and turning it into a working ShEx schema which we can validate hcls.ttl against". So far it has been done with a Perl script and a lot of manual input, but could potentially be simplified by the (as yet non-existent) ShEx Generator front-end tool.
This will be a standalone front-end tool to simplify and standardize the process of authoring an RDF schema in ShEx for both non-technical and technical users. Ideally it should be abstracted away from terms used in ShEx itself to keep it easy to use for users who are familiar with RDF and OWL concepts and terms but not necessarily with Shape Expressions. The basic idea is to allow a user to use simple web UI elements (select dropdowns, drag-drop elements, text fields otherwise, etc.) to create the schema in-memory. Once the model is complete, on button press it will be serialized to ShEx and a download of the ShEx schema will be triggered in the user's browser. Ideally, it should also support uploading / importing an existing ShEx schema, parsing it and generating the model, to allow modification of an existing schema without duplicated effort. This step should be simplified greatly by the work already performed by the ShEx-validator. The power of this tool as a simple ShEx generator and editor will be great, but it could potentially simplify the process even more for users by pulling in properties from specified namespaces using a client-side OWL reasoner. This would then allow the user to easily drag in entities from the various namespaces they are using, seeing the description of each entity as they do so. As this will use the same dependencies as the main ShEx Validata UI, it is part of the same repository and codebase, with primary html file located here: https://github.com/HeriotWattMEng2015/ShExValidata/blob/master/public/generator.html
This will be a standalone front-end tool for a Validata administrator to generate a configuration file for the Validata UI. This configuration file will then be uploaded via FTP (or whatever) to the Validata directory to be ready by it. For now, this just needs to be a simple static web interface to allow a user to upload ShEx schemas and give them a title and description. Once they've uploaded and described all the schemas they want to be available to Validata users, a click of a button should generate a JSON file with the configuration and trigger download. Ideally this tool should also support uploading / importing an existing configuration file, modifying the settings in it and downloading the modified version to avoid needing to duplicate the uploading of every schema to add a new one. Exactly what format the JSON file is in is not important, as long as it is intuitive. It must be able to store a number of schemas with (at least) names and descriptions, but likely other properties in the future. It should also be able to store other Validata settings / options in the future. As the output from this is required by the ShEx Validata UI, it is part of the same repository and codebase, with primary html file located here: https://github.com/HeriotWattMEng2015/ShExValidata/blob/master/public/admin.html
[https://github.com/HeriotWattMEng2015/ShExValidata / http://shextool.eu/](https://github.com/HeriotWattMEng2015/ShExValidata / http://shextool.eu/)
This is the user-facing front end to the ShEx validator, which uses the configuration file generated by the management tool to select schemas, and the ShEx-validator node module to actually validate the data. Currently just using bootstrap and jQuery with a bit of custom JavaScript and CSS, it's just meant to be a shiny front-end to make users feel at ease. This UI task also incorporates the User Experience bits of this project; that is, deciding how best to display errors to the user and displaying them in a non-confusing manner.
As far as I'm aware, the following people should now be working on the following things:
ShEx Validata Management (creating front-end tool)
ShEx-validator (improving callbacks, adding semi-technical, plain english error explanations)
ShEx requirement levels syntax (adding new syntax to shexParser.pegjs and recompiling with PEG to create new ShEx parser, modifying UnaryRule in Erics_RDF.js to pass new value through)
ShEx Schema Generator (creating front-end tool)
Working HCLS ShEx (improving perl script (?), battling HCLS nonsense to improve something.shex)
ShEx Validata UI (actually doing sensible things with validation callbacks to show useful stuff to the user)
ShEx Validata UI (?) (I've sent you a message!)
Please let me know if I've got anything wrong here or I'm missing something!