Add rights statements to EAD export (#2)

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: "[BUG]"
+labels: bug
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+_Optional, but include if helpful/relevant_
+**Desktop:**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone/Mobile:**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[FEATURE]"
+labels: enhancement
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/pull_request_template.md

@@ -0,0 +1,23 @@
+## Description
+<!--- Describe your changes.  Why is this required?  What problem does it solve?  What functionality does it extend? -->
+
+## Related GitHub Issue
+<!--- Please link to GitHub Issue here: -->
+
+## Testing
+<!--- Please describe, in detail, how you tested your changes. -->
+
+## Screenshot(s):
+<!--- Optional screenshots of changes if relevant and helpful to reviewers -->
+
+## Checklist
+
+- [ ] ✔️ Have you assigned at least one reviewer?
+- [ ] 🔗 Have you referenced any issues this PR will close?
+- [ ] ⬇️ Have you merged the latest upstream changes into your branch? 
+- [ ] 🧪 Have you added tests to cover these changes?  If not, why:
+
+- [ ] 🤖 Have automated checks (if any) passed?  If not, please explain for the reviewer:
+
+- [ ] 📘 Have you updated/added any relevant readmes/comments in the codebase?
+- [ ] 📚 Have you updated/added any external documentation (e.g. Confluence)?

.github/workflows/backend_plugin_test.yml

@@ -0,0 +1,58 @@
+name: Backend Plugin Testing
+
+on:
+  pull_request:
+    branches:
+      - main
+  push:
+
+jobs:
+  backend_plugins:
+    runs-on: ubuntu-latest
+    env:
+      PROD_ARCHIVESSPACE_VERSION: v3.3.1
+
+    services:
+      db:
+        image: mysql:8
+        env:
+          MYSQL_ROOT_PASSWORD: root
+          MYSQL_DATABASE: archivesspace
+          MYSQL_USER: as
+          MYSQL_PASSWORD: as123
+        ports:
+          - 3307:3306
+        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
+
+    steps:
+    - name: Checkout ArchivesSpace
+      uses: actions/checkout@v4
+      with:
+        ref: ${{ env.PROD_ARCHIVESSPACE_VERSION }}
+        repository: Smithsonian/archivesspace
+
+    - name: Checkout plugin
+      uses: actions/checkout@v4
+      with:
+        path: ${{ github.event.repository.name }}
+
+    - name: Copy plugin to ArchivesSpace and add to config
+      run: |
+        cp -r ${{ github.workspace }}/${{ github.event.repository.name }} ${{ github.workspace }}/plugins
+        cd ./common/config/
+        touch config.rb
+        echo "AppConfig[:plugins] = ['${{ github.event.repository.name }}']" > config.rb
+
+    - uses: Smithsonian/caas-aspace-services/.github/actions/bootstrap@main
+      with:
+        backend: true
+
+    - name: Allow ArchivesSpace functions for app db user
+      env:
+        DB_PORT: "3307"
+      run: |
+        mysql --host 127.0.0.1 --port $DB_PORT -uroot -proot -e "SET GLOBAL log_bin_trust_function_creators = 1;"
+
+    - name: Run Backend plugin tests
+      run: |
+        ./build/run backend:test -Dspec="../../plugins/${{ github.event.repository.name }}"

README.md

@@ -1,2 +1,22 @@
 # jpca_rights_statement
 An ArchivesSpace plugin to support JPCA-style EAD exports.
+
+_Rights Statements_
+
+- Within `<archdesc>`, exports `<userestrict>` holding a `<head>`, `<note>`, and `<list><item><date/></item></list>` matching a resource-level rights statement.  Unpublished notes will be exported with an audience of "internal."  Rights statements are not exported by ASpace by default.
+- Within `<c>`, exports `<userestrict>` holding a `<head>`, `<note>`, and `<list><item><date/></item></list>` matching an archival object-level rights statement.  Unpublished notes will be exported with an audience of "internal."  Rights statements are not exported by ASpace by default.
+
+| Description                                                | ASpace Default (simplified example) | JPCA Override (simplified example)                                      |
+| ---------------------------------------------------------- |------------------------------------ | ----------------------------------------------------------------------- |
+| Resource-level rights statement with a published note.     | not exported                        | `<userestrict id="aspace_[identifier]" type="[rights_type]"><head>Rights Statement</head><note type="[note_type]"><p>[note_content]</p></note><list><item><date normal="[start_date]" type="start" /></item></list></userestrict>`                    |
+| Component-level rights statement with a published note.    | not exported                        | `<userestrict id="aspace_[identifier]" type="[rights_type]"><head>Rights Statement</head><note type="[note_type]"><p>[note_content]</p></note><list><item><date normal="[start_date]" type="start" /></item></list></userestrict>`                    |
+| Resource-level rights statement with an unpublished note.  | not exported                        | `<userestrict id="aspace_[identifier]" type="[rights_type]"><head>Rights Statement</head><note audience="internal" type="[note_type]"><p>[note_content]</p></note><list><item><date normal="[start_date]" type="start" /></item></list></userestrict>` |
+| Component-level rights statement with an unpublished note. | not exported                        | `<userestrict id="aspace_[identifier]" type="[rights_type]"><head>Rights Statement</head><note audience="internal" type="[note_type]"><p>[note_content]</p></note><list><item><date normal="[start_date]" type="start" /></item></list></userestrict>` |
+
+## Tests
+
+Run the backend tests via: 
+
+```
+./build/run backend:test -Dspec="../../plugins/jpca_rights_statement"
+```

backend/lib/jpca_ead_extras_serialize.rb

@@ -0,0 +1,11 @@
+class JPCAEADSerialize < EADSerializer
+
+  def call(data, xml, fragments, context)
+    if context == :archdesc
+      if data.rights_statements
+        serialize_rights(data, xml, fragments)
+      end
+    end
+  end
+
+end

backend/model/jpca_ead_exporter.rb

@@ -0,0 +1,38 @@
+# encoding: utf-8
+require 'nokogiri'
+require 'securerandom'
+
+class EADSerializer < ASpaceExport::Serializer
+  serializer_for :ead
+
+  def serialize_rights(data, xml, fragments)
+    data.rights_statements.each do |rts_stmt|
+      xml.userestrict({ id: "aspace_#{rts_stmt['identifier']}", type: rts_stmt['rights_type'] }) {
+        xml.head('Rights Statement')
+
+        rts_stmt['notes'].each do |note|
+
+          atts = {}
+          atts['type'] = note['type']
+          atts['audience'] = 'internal' if note['publish'] === false
+
+          xml.note(atts) {
+            xml.p {
+              note['content'].each do |c|
+                sanitize_mixed_content(c, xml, fragments)
+              end
+            }
+          }
+        end
+
+        xml.list {
+          xml.item {
+            xml.date({ type: 'start', normal: rts_stmt['start_date'] }) if rts_stmt['start_date']
+            xml.date({ type: 'end', normal: rts_stmt['end_date'] }) if rts_stmt['end_date']
+          }
+        }
+      }
+    end
+  end
+
+end

backend/plugin_init.rb

@@ -0,0 +1,4 @@
+require_relative 'lib/jpca_ead_extras_serialize'
+
+# Register our custom serialize steps.
+EADSerializer.add_serialize_step(JPCAEADSerialize)

backend/spec/export_ead_jpca_overrides_spec.rb

@@ -0,0 +1,163 @@
+# encoding: utf-8
+require 'nokogiri'
+require 'spec_helper'
+require_relative '../../../../backend/spec/export_spec_helper'
+
+# Used to check that the fields EAD needs resolved are being resolved by the indexer.
+require_relative '../../../../indexer/app/lib/indexer_common_config'
+
+describe 'JPCA EAD export mappings' do
+
+  #######################################################################
+  # FIXTURES
+  #######################################################################
+
+  def load_export_fixtures
+    @published_note = build(:json_note_rights_statement, publish: true)
+    @unpublished_note = build(:json_note_rights_statement, publish: false)
+
+    resource = create(:json_resource,
+                      :publish => true,
+                      :rights_statements => [build(:json_rights_statement,
+                                                   notes: [@published_note,
+                                                           @unpublished_note])]
+                      )
+
+    @resource = JSONModel(:resource).find(resource.id)
+
+    @archival_object = create(:json_archival_object,
+                              :resource => {:ref => @resource.uri},
+                              :publish => true,
+                              :rights_statements => [build(:json_rights_statement,
+                                                           notes: [@published_note,
+                                                                   @unpublished_note])]
+                              )
+  end
+
+  def doc_unpublished
+    Nokogiri::XML::Document.parse(@doc_unpublished.to_xml).remove_namespaces!
+  end
+
+  before(:all) do
+    as_test_user('admin') do
+      RSpec::Mocks.with_temporary_scope do
+        # EAD export normally tries the search index first, but for the tests we'll
+        # skip that since Solr isn't running.
+        allow(Search).to receive(:records_for_uris) do |*|
+          {'results' => []}
+        end
+
+        as_test_user("admin", true) do
+          load_export_fixtures
+          @doc_unpublished = get_xml("/repositories/#{$repo_id}/resource_descriptions/#{@resource.id}.xml?include_unpublished=true&include_daos=true")
+
+          raise Sequel::Rollback
+        end
+      end
+      expect(@doc_unpublished.errors.length).to eq(0)
+
+      # if the word Nokogiri appears in the XML file, we'll assume something
+      # has gone wrong
+      expect(@doc_unpublished.to_xml).not_to include("Nokogiri")
+      expect(@doc_unpublished.to_xml).not_to include("#&")
+    end
+  end
+
+  describe 'Within <archdesc>' do
+    context 'when including unpublished' do
+      let(:doc) { doc_unpublished }
+
+      it 'exports rights_statements to <userestrict>' do
+        expect(doc.at_xpath("/ead/archdesc/userestrict/@id").content).
+          to match("aspace_#{@resource.rights_statements.first['identifier']}")
+        expect(doc.at_xpath("/ead/archdesc/userestrict/@type").content).
+          to match(@resource.rights_statements.first['rights_type'])
+        expect(doc.at_xpath("/ead/archdesc/userestrict/head").content).
+          to match('Rights Statement')
+        expect(doc.at_xpath("/ead/archdesc/userestrict/list/item/date/@type").content).
+          to eq('start')
+        expect(doc.at_xpath("/ead/archdesc/userestrict/list/item/date/@normal").content).
+          to match(@resource.rights_statements.first['start_date'])
+      end
+
+      it 'includes published and unpublished notes' do
+        expect(doc.xpath("/ead/archdesc/userestrict/note").count).to eq(2)
+      end
+
+      describe 'the unpublished note' do
+        let(:note) { doc.at_xpath("/ead/archdesc/userestrict/note[@audience='internal']") }
+
+        it 'has an audience of internal' do
+          expect(note.at_xpath("@audience").content).to eq('internal')
+        end
+
+        it 'exports correctly' do
+          expect(note.content).to match(@unpublished_note.content.join(''))
+          expect(note.at_xpath("@type").content).to match(@unpublished_note.type)
+        end
+      end
+
+      describe 'the published note' do
+        let(:note) { doc.at_xpath("/ead/archdesc/userestrict/note[not(@audience='internal')]") }
+
+        it 'has no audience attribute' do
+          expect(note.at_xpath("@audience")).to be(nil)
+        end
+
+        it 'exports correctly' do
+          expect(note.content).to match(@published_note.content.join(''))
+          expect(note.at_xpath("@type").content).to match(@published_note.type)
+        end
+      end
+    end
+  end
+
+  describe 'Within <c>' do
+    context 'when including unpublished' do
+      let(:doc) { doc_unpublished }
+
+      it 'exports rights_statements to <userestrict>' do
+        expect(doc.at_xpath("/ead/archdesc/dsc/c/userestrict/@id").content).
+          to match("aspace_#{@archival_object.rights_statements.first['identifier']}")
+        expect(doc.at_xpath("/ead/archdesc/dsc/c/userestrict/@type").content).
+          to match(@archival_object.rights_statements.first['rights_type'])
+        expect(doc.at_xpath("/ead/archdesc/dsc/c/userestrict/head").content).
+          to match('Rights Statement')
+        expect(doc.at_xpath("/ead/archdesc/dsc/c/userestrict/list/item/date/@type").content).
+          to eq('start')
+        expect(doc.at_xpath("/ead/archdesc/dsc/c/userestrict/list/item/date/@normal").content).
+          to match(@archival_object.rights_statements.first['start_date'])
+      end
+
+      it 'includes published and unpublished notes' do
+        expect(doc.xpath("/ead/archdesc/dsc/c/userestrict/note").count).to eq(2)
+      end
+
+      describe 'the unpublished note' do
+        let(:note) { doc.at_xpath("/ead/archdesc/dsc/c/userestrict/note[@audience='internal']") }
+
+        it 'has an audience of internal' do
+          expect(note.at_xpath("@audience").content).to eq('internal')
+        end
+
+        it 'exports correctly' do
+          expect(note.content).to match(@unpublished_note.content.join(''))
+          expect(note.at_xpath("@type").content).to match(@unpublished_note.type)
+        end
+      end
+
+      describe 'the published note' do
+        let(:note) { doc.at_xpath("/ead/archdesc/dsc/c/userestrict/note[not(@audience='internal')]") }
+
+        it 'has no audience attribute' do
+          expect(note.at_xpath("@audience")).to be(nil)
+        end
+
+        it 'exports correctly' do
+          expect(note.content).to match(@published_note.content.join(''))
+          expect(note.at_xpath("@type").content).to match(@published_note.type)
+        end
+      end
+    end
+  end
+end