Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lod #358

Merged
merged 127 commits into from
Dec 18, 2024
Merged

Lod #358

Show file tree
Hide file tree
Changes from 114 commits
Commits
Show all changes
127 commits
Select commit Hold shift + click to select a range
b23be7c
Create WikidataItem and add to sites
joeroe Mar 24, 2023
a60509b
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
e622d51
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
de6c73b
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
2735aa1
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
d08ca8d
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
b184d36
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
83556db
Add edit button for WikidataLink to site show view
joeroe Mar 30, 2023
81daccb
Add form for Wikidata links
joeroe Apr 19, 2023
839c04e
Limit Wikipedia extracts to first paragraph
joeroe May 3, 2023
2cae6bc
Validate qid (should be an integer)
joeroe May 3, 2023
a9b6a16
Fix simple icons with propshaft
joeroe Oct 31, 2023
782b5aa
Create WikidataItem and add to sites
joeroe Mar 24, 2023
f58cea2
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
c65751e
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
5b1447a
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
bd8a891
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
0b15a9b
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
452ccdd
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
1005449
Add edit button for WikidataLink to site show view
joeroe Mar 30, 2023
864546b
Add form for Wikidata links
joeroe Apr 19, 2023
b114639
Limit Wikipedia extracts to first paragraph
joeroe May 3, 2023
648c63f
Validate qid (should be an integer)
joeroe May 3, 2023
8093121
Fix simple icons with propshaft
joeroe Oct 31, 2023
ffd5747
Merge branch 'lod' of github.com:xronos-ch/xronos.rails into lod
MartinHinz Oct 25, 2024
a0a625f
first working prototype
MartinHinz Oct 27, 2024
ece8973
added link to wikidata item and match
MartinHinz Nov 30, 2024
a8bb098
switched to query wikidata in batches
MartinHinz Nov 30, 2024
8fc58c7
Create WikidataItem and add to sites
joeroe Mar 24, 2023
4d838ad
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
c52e61b
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
00e5b81
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
0f13477
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
37098e7
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
3fe8a0d
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
2620b05
Add edit button for WikidataLink to site show view
joeroe Mar 30, 2023
d7ddcca
Add form for Wikidata links
joeroe Apr 19, 2023
2bc81c4
Limit Wikipedia extracts to first paragraph
joeroe May 3, 2023
05a68dd
Validate qid (should be an integer)
joeroe May 3, 2023
8ca238a
Fix simple icons with propshaft
joeroe Oct 31, 2023
df469b7
Create WikidataItem and add to sites
joeroe Mar 24, 2023
0f4109d
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
30321ae
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
f06183a
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
c1320f4
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
02446ac
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
54aa87e
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
cbc44a6
Add form for Wikidata links
joeroe Apr 19, 2023
4b414f8
Validate qid (should be an integer)
joeroe May 3, 2023
f73ca77
Fix simple icons with propshaft
joeroe Oct 31, 2023
afb16e2
first working prototype
MartinHinz Oct 27, 2024
6502946
added link to wikidata item and match
MartinHinz Nov 30, 2024
79e5e1b
switched to query wikidata in batches
MartinHinz Nov 30, 2024
dcff5e7
bumped defaults to 7.0
MartinHinz Dec 14, 2024
cfb027c
errors/leftovers from merging
MartinHinz Dec 14, 2024
509bd45
Merge branch 'lod' of github.com:xronos-ch/xronos.rails into lod
MartinHinz Dec 14, 2024
64c8460
some layout changes
MartinHinz Dec 14, 2024
8193869
made wikidata request conditional on missing wikidata link
MartinHinz Dec 14, 2024
8bd98b7
fixed lod scope in routes
MartinHinz Dec 14, 2024
f298979
moved wikidata_links to more general lod_links
MartinHinz Dec 15, 2024
5538e3e
added status to lod_links and corrected approval mechanism + warning …
MartinHinz Dec 15, 2024
2e7d6ee
Create WikidataItem and add to sites
joeroe Mar 24, 2023
ec4d382
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
56d52c4
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
be7d7d1
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
dcafc86
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
3faeaff
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
151fb6f
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
f74b459
Add edit button for WikidataLink to site show view
joeroe Mar 30, 2023
bad4a57
Add form for Wikidata links
joeroe Apr 19, 2023
357c1e7
Limit Wikipedia extracts to first paragraph
joeroe May 3, 2023
c7bfd9e
Validate qid (should be an integer)
joeroe May 3, 2023
20e0b63
Fix simple icons with propshaft
joeroe Oct 31, 2023
45b7010
Create WikidataItem and add to sites
joeroe Mar 24, 2023
e50de2d
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
261dee1
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
043a758
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
c385b7a
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
20c8091
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
0ec8cb7
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
3e1a302
Add form for Wikidata links
joeroe Apr 19, 2023
74e5905
Validate qid (should be an integer)
joeroe May 3, 2023
ecf8542
Fix simple icons with propshaft
joeroe Oct 31, 2023
6c92c44
first working prototype
MartinHinz Oct 27, 2024
3535b22
added link to wikidata item and match
MartinHinz Nov 30, 2024
cc0eece
switched to query wikidata in batches
MartinHinz Nov 30, 2024
f57eabd
errors/leftovers from merging
MartinHinz Dec 14, 2024
1e5e6ee
Create WikidataItem and add to sites
joeroe Mar 24, 2023
0354aa2
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
076aa38
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
0e7e26c
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
60ecde3
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
088cae7
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
78c2037
Add Wikipedia extracts to site view
joeroe Mar 28, 2023
f2c1d07
Add edit button for WikidataLink to site show view
joeroe Mar 30, 2023
6babd15
Add form for Wikidata links
joeroe Apr 19, 2023
cbf98d4
Create WikidataItem and add to sites
joeroe Mar 24, 2023
93eec20
Add Wikidata and site links to site show view
joeroe Mar 24, 2023
19d5af1
Use simple icons for Wikimedia logos
joeroe Mar 24, 2023
0a6e33c
Rename WikidataItem to WikidataLink
joeroe Mar 26, 2023
e4742c3
Make Wikidata API requests asynchronous
joeroe Mar 26, 2023
b145731
Switch to turbo frame for async wikidata links
joeroe Mar 27, 2023
a4756ce
Fix simple icons with propshaft
joeroe Oct 31, 2023
efb8940
first working prototype
MartinHinz Oct 27, 2024
59fd245
switched to query wikidata in batches
MartinHinz Nov 30, 2024
49bd1e0
some layout changes
MartinHinz Dec 14, 2024
6093110
made wikidata request conditional on missing wikidata link
MartinHinz Dec 14, 2024
09b208b
fixed lod scope in routes
MartinHinz Dec 14, 2024
5793a0c
moved wikidata_links to more general lod_links
MartinHinz Dec 15, 2024
9e58065
added status to lod_links and corrected approval mechanism + warning …
MartinHinz Dec 15, 2024
f291fb2
rebased
MartinHinz Dec 17, 2024
4c93505
merged master back in
MartinHinz Dec 17, 2024
4fc672d
corrected mess from failed rebase attempt (hopefully)
MartinHinz Dec 17, 2024
621d22e
Fix code scanning alert no. 16: Code injection
MartinHinz Dec 17, 2024
2b162e9
added label for approving link to frontend form for wikidata
MartinHinz Dec 17, 2024
a41bf19
Fixed overeager search-and-replace, fixes #discussion_r1886756310
MartinHinz Dec 17, 2024
3a93c3c
removed simple-icons from package.json, fixes #discussion_r1886761226
MartinHinz Dec 17, 2024
8a2aed2
made request of item conditional on being an wikidata link, fixes #di…
MartinHinz Dec 17, 2024
ad4b5bf
made request of item conditional on being an wikidata link, fixes #di…
MartinHinz Dec 17, 2024
8a5a000
moved as much as possible wikidata related functionality out of lod_l…
MartinHinz Dec 17, 2024
ce4a41c
Fix code scanning alert no. 17: Code injection
MartinHinz Dec 17, 2024
0d93a8d
removed card from show in site
MartinHinz Dec 17, 2024
c7a27d7
changed NEED_WIKIDATA_LINK to MISSING_WIKIDATA_LINK
MartinHinz Dec 17, 2024
1849ba1
added changelog
MartinHinz Dec 17, 2024
43ad678
Bump schema version
joeroe Dec 17, 2024
e7170d9
Remove unused JS import #358
joeroe Dec 17, 2024
3c4d5dc
Fix changelog formatting
joeroe Dec 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ gem 'pg'
# App server
gem 'puma'

# Data cache (required to run Action Cable in production)
# In-memory cache (also required for Turbo broadcasts and ActionCable)
gem 'redis', '~> 4.0'

# Reduce boot times through caching (required in config/boot.rb)
Expand Down Expand Up @@ -94,13 +94,6 @@ gem 'countries'
gem 'country_select'
gem 'i18n_data'

# Geocoding
gem 'geocoder'

# GBIF Backbone Taxonomy
gem 'faraday', '~> 2.5', '>= 2.5.2'
gem 'gbifrb'

# Bibliographic data
gem 'bibtex-ruby'
gem 'citeproc-ruby'
Expand All @@ -110,6 +103,20 @@ gem 'csl-styles'
# lasso
gem 'activerecord-session_store'

# EXTERNAL APIS ---------------------------------------------------------------

gem 'faraday', '>= 2.5.2'

# Geocoding
gem 'geocoder'

# GBIF (Backbone Taxonomy)
gem 'gbifrb'

# Wikidata & Wikipedia
gem 'wikidata-client', require: 'wikidata'
gem 'wikipedia-client', require: 'wikipedia'

# VIEW HELPERS ----------------------------------------------------------------

# Pagination
Expand Down
16 changes: 15 additions & 1 deletion Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ GEM
railties (>= 6.1)
drb (2.2.1)
erubi (1.13.0)
excon (0.112.0)
execjs (2.10.0)
factory_bot (6.5.0)
activesupport (>= 5.0.0)
Expand All @@ -182,6 +183,9 @@ GEM
faraday-net_http (>= 2.0, < 3.5)
json
logger
faraday-excon (2.2.0)
excon (>= 0.109.0)
faraday (>= 2.11.0, < 3)
faraday-net_http (3.4.0)
net-http (>= 0.5.0)
feedjira (3.2.3)
Expand Down Expand Up @@ -211,6 +215,7 @@ GEM
guard (~> 2.1)
guard-compat (~> 1.1)
rspec (>= 2.99.0, < 4.0)
hashie (5.0.0)
httparty (0.22.0)
csv
mini_mime (>= 1.0.0)
Expand Down Expand Up @@ -491,6 +496,13 @@ GEM
websocket-extensions (0.1.5)
whenever (1.0.0)
chronic (>= 0.6.3)
wikidata-client (0.1.0)
excon (~> 0.92)
faraday (~> 2.2)
faraday-excon (~> 2.1)
hashie (~> 5.0)
wikipedia-client (1.17.0)
addressable (~> 2.7)
xpath (3.2.0)
nokogiri (~> 1.8)
zeitwerk (2.7.1)
Expand Down Expand Up @@ -523,7 +535,7 @@ DEPENDENCIES
dotenv-rails
factory_bot_rails
faker
faraday (~> 2.5, >= 2.5.2)
faraday (>= 2.5.2)
feedjira
gbifrb
geocoder
Expand Down Expand Up @@ -568,6 +580,8 @@ DEPENDENCIES
vega
web-console (>= 3.3.0)
whenever
wikidata-client
wikipedia-client

RUBY VERSION
ruby 3.3.6p108
Expand Down
1 change: 1 addition & 0 deletions app/assets/images/simple_icons/wikidata.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions app/assets/images/simple_icons/wikimediacommons.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions app/assets/images/simple_icons/wikipedia.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions app/assets/stylesheets/layout.scss
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,8 @@ span.hero_caption:hover {
background-color: rgba(0, 0, 0, 0.4);
}


.blockquote {
font-size: 1rem;
font-color: "red";
}
17 changes: 17 additions & 0 deletions app/assets/stylesheets/typography.scss
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,20 @@
.dl-inline dd {
display: inline;
}

// Align embedded svg icons with text
// Adapted from https://blog.prototypr.io/align-svg-icons-to-text-and-say-goodbye-to-font-icons-d44b3d7b26b4
.svg-icon {
display: inline-flex;
align-self: center;
}

.svg-icon svg {
height: 1em;
width: 1em;
}

.svg-icon.svg-baseline svg {
top: .125em;
position: relative;
}
75 changes: 75 additions & 0 deletions app/controllers/lod_links_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
class LodLinksController < ApplicationController
load_and_authorize_resource

before_action :set_lod_link, only: [:show, :edit, :update, :destroy]

def show
@wikidata_link.request_item
joeroe marked this conversation as resolved.
Show resolved Hide resolved
if @wikidata_link.item.sitelink_title("enwiki").present?
@wikidata_link.item.request_wikipedia_extract
end

render partial: "lod_link"
end

def new
end

def edit
end

def create
@lod_link = LodLink.new(lod_link_params)

respond_to do |format|
if @lod_link.save
format.html { redirect_back fallback_location: root_path, notice: success_notice }
format.json { render :show, status: :created, location: @lod_link }
else
format.html { render :new, status: :unprocessable_entity }
format.json { render json: @lod_link.errors, status: :unprocessable_entity }
end
end
end

def update
respond_to do |format|
if @lod_link.update(lod_link_params)
format.html { redirect_back fallback_location: root_path, notice: success_notice }
format.json { render :show, status: :ok, location: @lod_link }
else
format.html { render :edit, status: :unprocessable_entity }
format.json { render json: @lod_link.errors, status: :unprocessable_entity }
end
end
end

def destroy
end

private

def set_lod_link
@lod_link = LodLink.find(params[:id])
@wikidata_link = LodLink.where(source: "Wikidata").find(params[:id])
end

def lod_link_params
params.require(:lod_link).permit([
:external_id,
:source,
:linkable_type,
:linkable_id,
:revision_comment,
:status
])
end

def success_notice
@lod_link.linkable_type +
":" +
@lod_link.linkable_id.to_s +
" is now linked to Wikidata item Q" +
@lod_link.external_id.to_s
end
end
46 changes: 46 additions & 0 deletions app/controllers/lods/sites_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
class Lods::SitesController < LodsController
load_and_authorize_resource

# GET /issues/sites/:issue
def index
allowed_methods = lods
if lod_param.present? && allowed_methods.include?(lod_param.to_sym)
@sites = Site.public_send(lod_param)
Fixed Show fixed Hide fixed
else
@sites = Site.all
end

if params.has_key?(:search)
@sites = @sites.search params[:search]
end

if params.has_key?(:sites_order_by)
order = { params[:sites_order_by] => params.fetch(:sites_order, "asc") }
else
order = :id
end
@sites = @sites.reorder(order)

@sites = @sites.with_counts

respond_to do |format|
format.html do
@pagy, @sites = pagy(@sites)

# Fetch and cache Wikidata matches for the current page
@wikidata_matches = fetch_wikidata_matches(@sites)
end
end
end

private

def lods
Site.lods
end

def fetch_wikidata_matches(sites)
# Ensure Wikidata matches are always a hash
Site.wikidata_match_candidates_batch(sites) || {}
end
end
23 changes: 23 additions & 0 deletions app/controllers/lods_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
class LodsController < ApplicationController
include Pagy::Backend

before_action :authenticate_user!

layout "curate"

def index
end

private

def lods
[:issues, :another_allowed_method] # Add all allowed methods here
end

def lod_param
lod = params.fetch(:lod, nil)
return lod if lod.present? and lod.in?(lods.to_s)
end


end
4 changes: 3 additions & 1 deletion app/controllers/sites_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ def destroy
# Use callbacks to share common setup or constraints between actions.
def set_site
@site = Site.find(params[:id])
@wikidata_matches = Site.wikidata_match_candidates_batch([@site]) || {}
end

# Never trust parameters from the scary internet, only allow the white list through.
Expand All @@ -142,7 +143,8 @@ def site_params
{site_type_ids: []},
:country_code,
:revision_comment,
:_destroy
:_destroy,
wikidata_link_attributes: [:qid]
)
end
end
17 changes: 17 additions & 0 deletions app/helpers/application_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -97,5 +97,22 @@ def md(str)
.html_safe
end

def floating_button(path, options = {})
default_classes = "d-block position-absolute top-0 start-100 small"
if options.has_key?(:class)
options[:class] = options[:class] + " " + default_classes
else
options[:class] = default_classes
end

options[:title] = "Edit" unless options.has_key?(:title)

content_tag :div, class: "position-relative" do
link_to path, **options do
bs_icon "pencil"
end
end
end

end

21 changes: 21 additions & 0 deletions app/helpers/lods_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
module LodsHelper
def lod_label(lod)
lod.to_s.upcase
end

def lod_description(lod)
case lod
# Sites
when :missing_wikidata_link
"No Wikidata Link has yet been provided"
else
""
end
end

def lod_badge(lod)
content_tag :span, title: lod_description(lod), class: "badge text-bg-warning" do
lod_label(lod)
end
end
end
29 changes: 29 additions & 0 deletions app/helpers/simple_icons_helper.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
module SimpleIconsHelper

def simple_icon(slug)
content_tag :span, class: "svg-icon svg-baseline" do
embedded_svg "simple_icons/#{slug}.svg", width: 32, height: 32, role: "graphics-symbol", class: "simple-icon"
end
end

# Embed SVGs directly
# adapted from https://blog.cloud66.com/using-svgs-in-a-rails-stack
def embedded_svg(filename, options = {})
assets = Rails.application.assets

asset = assets.load_path.find(filename)

if asset
#file = asset.source.force_encoding("UTF-8")
doc = Nokogiri::HTML::DocumentFragment.parse asset.content
svg = doc.at_css "svg"
options.each {|attr, value| svg[attr.to_s] = value}
#svg["class"] = options[:class] if options[:class].present?
else
doc = "<!-- SVG #{filename} not found -->"
end

raw doc
end

end
4 changes: 4 additions & 0 deletions app/helpers/sites_helper.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
module SitesHelper
include Pagy::Frontend
include DataTableHelper

def wikidata_matches_for(site)
Site.wikidata_match_candidates_batch([site])
end
end
3 changes: 3 additions & 0 deletions app/javascript/application.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,7 @@ import "./controllers"
require("@fontsource/inter")
require("@fontsource/raleway")

//icons
import { siWikipedia, siWikimediacommons, siWikidata } from 'simple-icons'

ActiveStorage.start()
Loading
Loading