Add offline module

[ziyuan-linn]

The PR implements an offline version of the ml5.js library.

Webpack will now build ml5-offline.js and ml5-offline.min.js in addition to ml5.js and ml5.min.js. The file ml5-offline.js and ml5-offline.min.js is self-contained and will not make additional network requests. Once a user downloads ml5-offline.min.js, the ml5 library can run entirely without internet.

The entry point for the offline model is src/offline/index.js. It imports the ml5 object, injects a loadOfflineModel function into each ml5 model's prototype, and exports the ml5 object with the modified models. The approach keeps the offline implementation mostly separate from the main source code.

Changes:

• add a script that downloads the pre-trained models from Kaggle.com.
  • The script converts the model JSON and binary files to JS files.
  • model.json is written to a JS file.
  • binary files converted to Uint8Array and then written to JS files.
  • This is done so that Webpack can pack the model files and source code into a single file.
• implement a loadOfflineModel function.
  • This function loads the above data into File objects and creates object URLs for them.
  • The object URLs are then injected into the "model URL" fields (e.g. landmarkModelURL) of the ml5 model.
• For the ml5-offline.js build, add the loadOfflineModel functions into object prototypes of each ml5 model.
• Call the loadOfflineModel function from each ml5 model during initialization if the function is defined.

To test this implementation, find an example sketch and change ml5.js to ml5-offline.js in index.html.

Note: This is WIP, ml5-offline.js only works for HandPose with modelType: "lite".

[shiffman]

This is exciting to see thank you @ziyuan-linn for working on this! One consideration is that running a stand-along node.js script for downloading the models may involves concepts/steps unfamiliar to beginners especially those used to working only in the p5.js web editor. I think it's ok for this to be the primary way to get the models since this is an "edge" use case, however, we should probably include some detailed documentation for total beginners. I could also record a video tutorial on this topic. Alternatively we could consider providing links to some model downloads directly but this might be hard to maintain.

[ziyuan-linn]

@shiffman Thank you for the review! The node.js script for downloading models is meant to be only called by the ml5.js maintainers. I could also set it up as a postinstall script that automatically runs after yarn so we never have to worry about it. The purpose of the script is to download the models so WebPack can bundle the model files into the ml5 library file itself.

As far as the users are concerned, they only need to change ml5.js to ml5-offline.js in the script tag. If they want to go fully offline, they can also download the p5.js and ml5-offline.js scripts and put them in the sketch folder. We can definitely have a quick guide/documentation for this!

[jackbdu]

@ziyuan-linn Thank you for working on this!! I have been looking forward to the offline support! I think automatically downloading the models would be nice.

I built this PR locally with yarn build and it took more than two hours. Was it normal? Activity Monitor showed webpack was using 1.35GB memory and 10% CPU, not sure what the bottleneck was. In any case, HandPose examples worked fine offline with the build.

I am trying to add offline support to FaceMesh by following your HandPose implementation. I modified relevant code in src/offline, src/FaceMesh, and scripts/fetchModelFiles.js. I was able to use fetchModelFiles.js to download the FaceMesh model from "https://www.kaggle.com/api/v1/models/mediapipe/face-landmarks-detection/tfJs/face-mesh/1/download", but the FaceMesh example sketches still seem to be requesting the model remotely.

@ziyuan-linn Can you shed some light on this? Let me know if you need more information.