Sushi Chef script for importing pointb-21csguide content from https://www.pointb.is/21csguide
- Install the system prerequisites
ffmpeg
andimagemagick
by following the prerequisite install instructions. - Install Python 3 if you don't have it already.
- Make sure you also have
pip
installed by running the commandpip --help
in a terminal, and if missing install it. - Create a Python virtual environment for this project:
- Install the virtualenv package:
pip install virtualenv
- The next steps depends if you're using UNIX or Windows:
- For UNIX systems (Linux and Mac):
- Create a virtual env called
venv
in the current directory using the following command:virtualenv -p python3 venv
- Activate the virtualenv called
venv
by running:source venv/bin/activate
. Your command prompt should change and show the prefix(venv)
to indicate you're now working insidevenv
. - Checkpoint: Try running
which python
and confirm the Python in is use the one from the virtual env (e.g.venv/bin/python
) and not the system Python. Check alsowhich pip
is the one from the virtualenv. - You may encounter this error
SSL: CERTIFICATE_VERIFY_FAILED
. to fix it, at the terminal run the commandsudo /Applications/Python (your Python version)/Install Certificates.command
- Create a virtual env called
- For Windows systems:
- Create a virtual env called
venv
in the current directory using the following command:virtualenv venv
. - Activate the virtualenv called
venv
by running.\venv\Scripts\activate
- Checkpoint: Try running
python --version
and confirm the Python version that is running is the same as the one you downloaded in step 2.
- Create a virtual env called
- For UNIX systems (Linux and Mac):
- Install the virtualenv package:
- Run
pip install -r requirements.txt
to install the required python libraries. - Run
pip install -r requirements-dev.txt
to install the development python libraries.
Run the following command to start the jupyter notebook server
jupyter notebook
navigate to the folder notebooks/
and click on the individual notebook to view
and run interactively.
python sushichef.py -v --reset --token=<Kolibri Studio token>
A sushi chef script is responsible for importing content into Kolibri Studio. The ricecooker library provides all helper methods necessary for uploading the content to Kolibri Studio. The ricecooker docs can be found here.
This repo includes two sample chef scripts in examples/openstax_sushichef.py
(json)
and examples/wikipedia_sushichef.py
(html). To find more code examples, search
for sushi-chef-*
on github
to see all the sushi chef scripts. They are all example of how to extract,
transform, and upload content from various sources of openly licensed content.
A sushi chef script has been created for you in sushichef.py
to help you get
started on the import of the content.
- Start by looking at the channel spec that describes the target channel structure, licensing information, and tips about content transformations that might be necessary.
- Add the code necessary to create this channel by modifying the
construct_channel
method of the chef class defined insushichef.py
.
Use the following rubric as a checklist to know when your sushi chef script is done:
- Does the channel correspond to the spec provided?
- Does the content render as expected when viewed in Kolibri?
- Is the channel uploaded to Studio and PUBLISH-ed?
- Is the channel imported to a demo server where it can be previewed?
- Is the information about the channel token, the studio URL, and demo server URL
on notion card up to date? See the Studio Channels table.
If a card for your channel yet doesn't exist yet, you can create one using
the
[+ New]
button at the bottom of the table.
- Do all nodes have appropriate titles?
- Do all nodes have appropriate descriptions (when available in the source)?
- Is the correct language code set on all nodes and files?
- Is the
license
field set to the correct value for all nodes? - Is the
source_id
field set consistently for all content nodes? Use unique identifiers based on the source website or permanent url paths.
- Does the section
Usage
in this README contain all the required information for another developer to run the script? Document and extra credentials, env vars, and options needed to run the script. - Is the Python code readable and succinct?
- Are clarifying comments provided where needed?
Running the sushichef script is only one of the steps in the Kolibri content development workflow:
ricecooker studio kolibri demo server
SPEC-->--CHEF----->-----PUBLISH--->--IMPORT using token and REVIEW
(1) (2) (3) (4) (5)
\______/ /
\______________________________________________________/
It is your responsibility as the chef author to take this channel all the way through this workflow and make sure that the final channel works in Kolibri.
Notes on specific steps:
(1)
: the spec for the channel describes how the channel should be organized(2)
: your main task as a chef author is to implement all the extraction and content transformation described in the spec. If you run into any kind of difficulties with this step post a question in the LE slack channel#sushi-chefs
and someone from the content team will be able assist you. For example, "Hello @here I'm having trouble with the pointb-21csguide chef because X and Y cannot be organized according to the spec because Z." For complicated or very large channels the spec may go through multiple iterations.(3)
: once the channel is on Studio you can preview the structure there and create or update a notion card with the channel information. The next step is to export the channel in the format necessary for use in Kolibri using thePUBLISH
button on Studio. The PUBLISH action exports all the channel metadata to a sqlite3 DB filehttps://studio.learningequality.org/content/databases/{channel_id}.sqlite3
the first time a channel is PUBLISH-ed a secret token is generated that can be used to import the channel in Kolibri. Note down the channel's token.(4)
: the next step is to import your channel into a Kolibri instance. You can use Kolibri installed on your local machine or an online demo server. Admin (devowner
) credentials for the demo server will be provided for you so that you can import and update the channel every time you push a new version. Follow these steps to import your channelDevice
>IMPORT
>ONLINE
>Try adding a token
, add the channel token, select all nodes >IMPORT
.(5)
: You can now go to the Kolibri Learn tab and preview your channel to see it the way learners will see it. Take the time to click around and browse the content to make sure everything works as expected. Update the notion card and leave a comment. For example "First draft of channel uploaded to demo server." This would be a good time to ask a member of the LE content team to review the channel. You can do this using the@Person Name
in your notion comment. Consult the content source notion card to know who the relevant people to tag. For example, you can @-comment theLibrary
person on the card to ask them to review the channel—be sure to specify the channel's "level of readiness" in your comment, e.g., if it's a draft version for initial feedback, or the near-final, spec-compliant version ready for detailed review and QA. For async technical questions tag theSushOps
person on the card or post your question in the#sushi-chefs
channel. For example, "I downloaded this html and js content, but it doesn't render right in Kolibri because of the iframe sandboxing." or "Does anyone have sample code for extracting content X from a shared drive link Y of type Z?".