Skip to content

Commit

Permalink
commiting to main
Browse files Browse the repository at this point in the history
  • Loading branch information
R. S. Doiel committed Nov 13, 2024
1 parent 8b8cbe3 commit 96ed30e
Show file tree
Hide file tree
Showing 2 changed files with 73 additions and 40 deletions.
68 changes: 44 additions & 24 deletions presentations/presentation1.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
<meta name="author" content="R. S. Doiel, [email protected]" />
<meta name="date" content="2024-11-15" />
<meta name="keywords" content="code4lib, microservice, SQLite3, Deno, TypeScript, Dataset" />
<title>COLD - curating objects with SQLite3, Deno and Dataset</title>
<title>COLD - curating objects with Deno, Dataset and SQLite3</title>
<style type="text/css">
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
Expand All @@ -34,8 +34,8 @@
</head>
<body>
<div class="slide titlepage">
<h1 class="title">COLD - curating objects with SQLite3, Deno and
Dataset</h1>
<h1 class="title">COLD - curating objects with Deno, Dataset and
SQLite3</h1>
<p class="author">
R. S. Doiel, <a href="mailto:[email protected]"
class="email">[email protected]</a>
Expand All @@ -48,17 +48,28 @@ <h1 class="title">COLD - curating objects with SQLite3, Deno and
<div class="slide" id="TOC">

</div>
<div id="cold---curating-objects-with-sqlite3-deno-and-dataset"
<div id="cold---curating-objects-with-deno-dataset-and-sqlite3"
class="slide section level1">
<h1>COLD - curating objects with SQLite3, Deno and Dataset</h1>
<h1>COLD - curating objects with Deno, Dataset and SQLite3</h1>
<p>COLD is an application for curating collections of metadata objects
at Caltech Library</p>
<p>These are used for…</p>
<ul>
<li>crosswalk data</li>
<li>managing vocabularies</li>
<li>aggregate reporting</li>
<li>crosswalking data</li>
<li>aggregated reporting</li>
<li>generated RDM vocabularies</li>
</ul>
</div>
<div id="cold---curating-objects-with-deno-dataset-and-sqlite3-1"
class="slide section level1">
<h1>COLD - curating objects with Deno, Dataset and SQLite3</h1>
<dl>
<dt>COLD</dt>
<dd>
Controlled Object Lists and Datum
</dd>
</dl>
</div>
<div id="before-cold-we-used-spreadsheets" class="slide section level1">
<h1>Before COLD we used spreadsheets</h1>
<ul>
Expand All @@ -67,24 +78,31 @@ <h1>Before COLD we used spreadsheets</h1>
<li>Staff understand spreadsheets</li>
<li>CSV files are easy to process and version control</li>
</ul></li>
<li>Big Problem
<li>Disadvantages
<ul>
<li>Our people list had grown to over 32 columns and 8900 rows</li>
<li>Developers wound up maintaining the spreadsheet</li>
<li>It required developers to manage and maintain the spreadsheet</li>
</ul></li>
</ul>
</div>
<div id="desired-features" class="slide section level1">
<h1>Desired features</h1>
<div id="requirements" class="slide section level1">
<h1>Requirements</h1>
<ul>
<li>Provide a <strong>simple way</strong> to curate metadata object
collections</li>
<li>Provide a data source for <a
href="https://feeds.library.caltech.edu"
class="uri">https://feeds.library.caltech.edu</a></li>
<li>Provide a means of generating RDM vocabularies</li>
<li>Integrate with campus Single Sign On for access control</li>
</ul>
</div>
<div id="deliverables" class="slide section level1">
<h1>Deliverables</h1>
<ul>
<li>Simple curation of metadata object lists</li>
<li>Act as a data source for <a href="https://feeds.library.caltech.edu"
class="uri">https://feeds.library.caltech.edu</a> (feeds is our public
API)</li>
<li>Provide an ability to easily manage vocabularies (work in
progress)</li>
<li>Be easy for other developers to maintain and enhance</li>
<li>Must integrate with campus Single Sign On for access control</li>
<li>A web UI for curating objects</li>
<li>A reports system</li>
<li>Integration points for RDM and Feeds</li>
</ul>
</div>
<div id="project-status" class="slide section level1">
Expand All @@ -100,26 +118,28 @@ <h1>A quick tour of COLD</h1>
href="https://apps.library.caltech.edu/cold/"
class="uri">https://apps.library.caltech.edu/cold/</a></p>
</div>
<div id="development-approach" class="slide section level1">
<h1>Development approach</h1>
<div id="development-history" class="slide section level1">
<h1>Development history</h1>
<ul>
<li>Multiple prototypes with differing implementation choices
<ul>
<li>Python and PostgreSQL via Flask/ORM</li>
<li>Python and PostgreSQL via Flask/ORM (abandoned)</li>
<li>Go plus SQLite3 and browser side JavaScript</li>
<li>Go, PostgreSQL+PostgREST and browser side JavaScript
(abandoned)</li>
<li>Dataset, SQLite3 and TypeScript middleware (three
microservices)</li>
</ul></li>
<li>Currently running a pilot version based on Dataset, SQLite3 and
TypeScript and middleware</li>
</ul>
</div>
<div id="role-of-prototyping" class="slide section level1">
<h1>Role of prototyping</h1>
<ul>
<li>I used the prototypes as a sounding board for requirements and
features</li>
<li>Incrementally worked towards a “production pilot” (current
<li>I incrementally worked towards a “production pilot” (current
stage)</li>
</ul>
</div>
Expand Down
45 changes: 29 additions & 16 deletions presentations/presentation1.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: COLD - curating objects with SQLite3, Deno and Dataset
title: COLD - curating objects with Deno, Dataset and SQLite3
author: "R. S. Doiel, <[email protected]>"
institute: |
Caltech Library,
Expand All @@ -20,30 +20,42 @@ keywords: [ "code4lib", "microservice", "SQLite3", "Deno", "TypeScript", "Datase
url: "https://caltechlibrary.github.io/cold/presentation"
---

# COLD - curating objects with SQLite3, Deno and Dataset
# COLD - curating objects with Deno, Dataset and SQLite3

COLD is an application for curating collections of metadata objects at Caltech Library

- crosswalk data
- managing vocabularies
- aggregate reporting
These are used for...

- crosswalking data
- aggregated reporting
- generated RDM vocabularies

# COLD - curating objects with Deno, Dataset and SQLite3

COLD
: Controlled Object Lists and Datum

# Before COLD we used spreadsheets

- Advantages
- Staff understand spreadsheets
- CSV files are easy to process and version control
- Big Problem
- Disadvantages
- Our people list had grown to over 32 columns and 8900 rows
- Developers wound up maintaining the spreadsheet
- It required developers to manage and maintain the spreadsheet

# Requirements

- Provide a **simple way** to curate metadata object collections
- Provide a data source for <https://feeds.library.caltech.edu>
- Provide a means of generating RDM vocabularies
- Integrate with campus Single Sign On for access control

# Desired features
# Deliverables

- Simple curation of metadata object lists
- Act as a data source for <https://feeds.library.caltech.edu> (feeds is our public API)
- Provide an ability to easily manage vocabularies (work in progress)
- Be easy for other developers to maintain and enhance
- Must integrate with campus Single Sign On for access control
- A web UI for curating objects
- A reports system
- Integration points for RDM and Feeds

# Project status

Expand All @@ -54,18 +66,19 @@ COLD is an application for curating collections of metadata objects at Caltech L

Our COLD pilot deployment, <https://apps.library.caltech.edu/cold/>

# Development approach
# Development history

- Multiple prototypes with differing implementation choices
- Python and PostgreSQL via Flask/ORM
- Python and PostgreSQL via Flask/ORM (abandoned)
- Go plus SQLite3 and browser side JavaScript
- Go, PostgreSQL+PostgREST and browser side JavaScript (abandoned)
- Dataset, SQLite3 and TypeScript middleware (three microservices)
- Currently running a pilot version based on Dataset, SQLite3 and TypeScript and middleware

# Role of prototyping

- I used the prototypes as a sounding board for requirements and features
- Incrementally worked towards a "production pilot" (current stage)
- I incrementally worked towards a "production pilot" (current stage)

# My "Off the shelf" stack

Expand Down

0 comments on commit 96ed30e

Please sign in to comment.