feat: implement support for handwriting (#20)

* feat: add scaffold for handwriting task

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* feat: latex preview box for handwriting single Qs

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* feat: add KaTeX to markdown so we can render questions

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>

* feat: add modal for handwriting

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>

* style: prepare space for excalidraw

* feat: add default excalidraw into the modal

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>

* feat: wire in live update hook for excalidraw

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>

* style: css override for excalidraw

* feat: disable parts of excalidraw we don't want in JS (TS)

* feat: add clear canvas dialog

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>

* fix: finish integrating mathpix API

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>

* fix: move strokes updates to happen on moue rather than via useEffect

React docs recommend doing update logic on the event,
not using a state variable inbetween.
This was how useLiveUpdates worked previously,
so we switched it to returning a useCallback for updateStrokes rather than a state setter.
This was to prevent an infinite loop from setLatex changing,
and calling itself on change which then caused it to change again.

* style: adjust LaTeX preview box

Co-authored-by: Alexander Biraben-Renard <[email protected]>

* style: add border to canvas

Co-authored-by: Alexander Biraben-Renard <[email protected]>

* test: mock things that don't work in jsdom environment

* fix: fix bug where canvas drawing would be offset from cursor

Co-authored-by: Matthew Alex <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* chore: format canvas.tsx so CI passes

* feat: display section text in canvas dialog

Co-authored-by: Matthew Alex <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* feat: implement view-only handwriting canvas

Co-authored-by: Matthew Alex <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* fix: fix issue where pen was offset from cursor

Co-authored-by: Matthew Alex <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* feat: hide view-only canvas if empty

Co-authored-by: Matthew Alex <[email protected]>
Co-authored-by: nick-bolas <[email protected]>

* chore: do not store escape characters as part of latex answer

* chore: use updated mathpix token route

* refactor: rename MATHS_SINGLE_ANSWER to PROCESSD_HANDWRITING

* refactor: apply suggestions from code review

Fix env vars, minor changes to Canvas.tsx

Co-authored-by: Ivan Procaccini <[email protected]>

* docs: add info about env vars

Co-authored-by: Matthew Alex <[email protected]>

* refactor: extract Canvas props into interface CanvasProps

* refactor: explain why right click disabled

Co-authored-by: Matthew Alex <[email protected]>

* refactor: extract lists of keyboard shortcuts to constants

* style: move excalidraw editor container CSS into index.scss

* refactor: move ConfirmDialog out of Canvas & add tests

* refactor: move handwriting editor props out

* refactor: move handwriting editor styles into css file

* refactor: move viewonlycanvas props into interface

* style: extract excalidraw view container CSS out

Co-authored-by: Matthew Alex <[email protected]>

* style: remove unecessary CSS for latex preview on question page

* style: change edit answer button to standard default

* style: move dialog content CSS to separate file

* refactor: renamed transformStrokesForAPI to serialiseStrokes

* refactor: apply suggestions to reformat live-updates-hook

Co-authored-by: Ivan Procaccini <[email protected]>

* refactor: extract mathpix api to constant

* chore: improve comment explaining timeout for api calls

* chore: add env vars for mathpix to dev.local docker compose

* refactor: remove question text from handwriting modal

---------

Co-authored-by: Alexander Biraben-Renard <[email protected]>
Co-authored-by: nick-bolas <[email protected]>
Co-authored-by: Matthew Alex <[email protected]>
Co-authored-by: Ivan Procaccini <[email protected]>

.env.template

@@ -0,0 +1,3 @@
+# Fill this in with your APP ID and APP KEY from Mathpix, so that the API can be used to convert handwriting to LaTeX.
+MATHPIX_APP_ID=
+MATHPIX_APP_KEY=

.gitignore

@@ -22,3 +22,5 @@
 npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
+
+.env

README.md

@@ -4,6 +4,7 @@ This repo contains the frontend code for the Answerbook project. The frontend is
 ## Setup
 1. Run `nvm use` to ensure you're using the correct Node version.
 2. Run `npm install` to install dependencies (this will also setup commit hooks).
+3. Make a copy of `.env.template` as `.env` and fill it in. This file is used to store environment variables, specifically the API keys needed to use mathpix for handwriting recognition.
 
 ## Running the application
 ### In Docker

dev.docker-compose.yml

@@ -6,6 +6,8 @@ services:
       service: backend
     environment:
       - DB_URL=postgresql://user:pass@db/answerbook
+      - MATHPIX_APP_ID
+      - MATHPIX_APP_KEY
 
   frontend:
     extends:

dev.local.docker-compose.yml

@@ -9,6 +9,8 @@ services:
     command: poetry run uvicorn main:app --reload --host 0.0.0.0 --port 5004
     environment:
       - DB_URL=postgresql://user:[email protected]/answerbook
+      - MATHPIX_APP_ID
+      - MATHPIX_APP_KEY
     volumes:
       - ../answerbook-api:/api
     ports: