Error when building site with jekyll-feed plugin with JEKYLL_ENV=production and parallel localization

[george-gca]

I sometimes get an error when building my site with JEKYLL_ENV=production. Here is part of the stacktrace:

Incremental build: disabled. Enable with --incremental
      Generating...
"Fetching external posts from medium.com:"
"Fetching external posts from medium.com:"
"...fetching https://medium.com/@al-folio/displaying-external-posts-on-your-al-folio-blog-b60a1d241a0a?source=rss-17feae71c3c4------2"
       Jekyll Feed: Generating feed for posts
/home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-imagemagick-1.4.0/lib/jekyll-imagemagick/generator.rb:35:in `mkdir': No such file or directory @ dir_s_mkdir - /home/gca/repos/multi-language-al-folio/_site/pt-br (Errno::ENOENT)
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-imagemagick-1.4.0/lib/jekyll-imagemagick/generator.rb:35:in `generate'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/site.rb:193:in `block in generate'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/site.rb:191:in `each'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/site.rb:191:in `generate'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/site.rb:79:in `process'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:102:in `process_active_language'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:86:in `process_language'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:41:in `block (2 levels) in process'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:40:in `fork'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:40:in `block in process'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:39:in `each'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-polyglot-1.7.0/lib/jekyll/polyglot/patches/jekyll/site.rb:39:in `process'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/command.rb:28:in `process_site'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/commands/build.rb:65:in `build'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/commands/build.rb:36:in `process'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/command.rb:91:in `block in process_with_graceful_fail'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/command.rb:91:in `each'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/command.rb:91:in `process_with_graceful_fail'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/lib/jekyll/commands/serve.rb:86:in `block (2 levels) in init_with_program'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `block in execute'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `each'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `execute'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/mercenary-0.4.0/lib/mercenary/program.rb:44:in `go'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/mercenary-0.4.0/lib/mercenary.rb:21:in `program'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-4.3.2/exe/jekyll:15:in `<top (required)>'
	from /home/gca/.rbenv/versions/3.2.2/bin/jekyll:25:in `load'
	from /home/gca/.rbenv/versions/3.2.2/bin/jekyll:25:in `<top (required)>'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/cli/exec.rb:58:in `load'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/cli/exec.rb:58:in `kernel_load'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/cli/exec.rb:23:in `run'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/cli.rb:492:in `exec'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/cli.rb:34:in `dispatch'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/cli.rb:28:in `start'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.10/libexec/bundle:45:in `block in <top (required)>'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/3.2.0/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	from /home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.10/libexec/bundle:33:in `<top (required)>'
	from /home/gca/.rbenv/versions/3.2.2/bin/bundle:25:in `load'
	from /home/gca/.rbenv/versions/3.2.2/bin/bundle:25:in `<main>'
"...fetching https://medium.com/@al-folio/displaying-external-posts-on-your-al-folio-blog-b60a1d241a0a?source=rss-17feae71c3c4------2"
       Jekyll Feed: Generating feed for posts
       Imagemagick: Searching files in /home/gca/repos/multi-language-al-folio/assets/img/

This only happened sometimes. When trying to build the pt-br localized site before the main one it throws this error. Disabling parallel localization fixes this. It seems that parallel localization doesn't go well with the jekyll-feed plugin.

You can see this behavior in my build and deploy workflow, under Install and Build step.

[untra]

Hi @george-gca ! thank you for the feedback, PRs and issues. I'm glad you're getting success from polyglot!

This only happened sometimes.

Disabling parallel localization fixes this.

these symptoms make sense to me given the context. The parallel localization builds each language site simultaneously, and each built site uses assets that should be present at the time. nondeterminism is a smell of shared + mutated state.

/home/gca/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/jekyll-imagemagick-1.4.0/lib/jekyll-imagemagick/generator.rb:35:in `mkdir': No such file or directory @ dir_s_mkdir - /home/gca/repos/multi-language-al-folio/_site/pt-br (Errno::ENOENT)

ENOENT means the path doesn't exists OR the process doesn't have permission to make the directory. This stack overflow suggests it might be a cause from ownership permissions of the directory for the .rbenv environment. You might want to try this suggestion here: https://stackoverflow.com/a/73514840

I don't know if this stacktrace is actually from jekyll-feed, but the jekyll-imagemagik library seems to be throwing this error. Just personally, imagemagick is one of my least favorite opensource libraries with a variety of hidden traps and memory issues. I have been bitten by this trash software so many times in my professional career.

[george-gca]

Is there an alternative to imagemagick you would recommend?