Uploaded rockspec does not match the source .rockspec

[ColinKennedy]

I've been unable to install my luarocks packages as expected. After looking into it, the issue is that the rockspec that uploads to luarocks does not match what I'd written.

Here is an example.

This is the input .rockspec:

https://github.com/ColinKennedy/mega.vimdoc/blob/90a28a5403b440f7d9c85b7f4ccd04e3dcbdfb58/mega.vimdoc-scm-1.rockspec

rockspec_format = "3.0"
package = "mega.vimdoc"
version = "scm-1"

local user = "ColinKennedy"

description = {
    homepage = "https://github.com/" .. user .. "/" .. package,
    labels = { "neovim", "neovim-plugin" },
    license = "MIT",
    summary = 'A Neovim plugin that converts Lua source code to vimdoc',
}

dependencies = {
    "mini.doc >= 0.14.0, < 1.0",
    "mega.logging >= 1.0.0, < 2.0",
}

test_dependencies = {
    "busted >= 2.0, < 3.0",
    "lua >= 5.1, < 6.0",
    "nlua >= 0.2, < 1.0",
}

-- Reference: https://github.com/luarocks/luarocks/wiki/test#test-types
test = { type = "busted" }

source = {
    url = "git://github.com/" .. user .. "/" .. package,
}

build = {
    type = "builtin",
}

And here what makes it into luarocks, on-upload

https://luarocks.org/manifests/colinkennedy/mega.cmdparse-1.0.3-1.rockspec

local git_ref = 'v1.0.3'
local modrev = '1.0.3'
local specrev = '1'

local repo_url = 'https://github.com/ColinKennedy/mega.vimdoc'

rockspec_format = '3.0'
package = 'mega.vimdoc'
version = modrev ..'-'.. specrev

description = {
  summary = 'A Neovim plugin that converts Lua source code to vimdoc',
  detailed = '',
  labels = { 'neovim', 'neovim-plugin' } ,
  homepage = 'https://github.com/ColinKennedy/mega.vimdoc',
  license = 'MIT'
}

dependencies = { 'lua >= 5.1' } 

test_dependencies = { }

source = {
  url = repo_url .. '/archive/' .. git_ref .. '.zip',
  dir = 'mega.vimdoc-' .. '1.0.3',
}

if modrev == 'scm' or modrev == 'dev' then
  source = {
    url = repo_url:gsub('https', 'git')
  }
end

build = {
  type = 'builtin',
  copy_directories = { 'doc' } ,
}

As you can see, the dependencies section is missing mini.doc and mega.logging. Both of which have uploads in luarocks.

• https://luarocks.org/modules/colinkennedy/mega.logging
• https://luarocks.org/modules/neorocks/mini.doc

The test_dependencies is also missing its contents. Why is this information being stripped out?

I can see in this repo's README that there's apparently a section that says you can optionally define dependencies and test_dependencies (https://github.com/nvim-neorocks/luarocks-tag-release?tab=readme-ov-file#dependencies). I haven't tried if adding things manually to there would solve the problem but the README.md does say the it should be optional. And I would hope that the .rockspec would be taken as the source of truth.

[mrcjkb]

Hey 👋

This is a duplicate of

• [QOL] Ability to use scm rockspec in root directory as a rockspec template #435

For clarification: luarocks-tag-release currently does not use scm rockspecs. It uses a rockspec template, which can have variables that can be populated by inputs.

[ColinKennedy]

So the fix would be to define a template: key, point it to my own .rockspec file, and then replace stuff with '$git_ref' as needed?

Alternatively...delete the .rockspec entirely and use the default template + drive its contents completely by the github workflow?

[mrcjkb]

Yes. Using your own template is usually done if you have a more complex build step than the builtin build type, or if you need to do things that can't be covered with the GitHub workflow's inputs.

[ColinKennedy]

I've encountered an issue with using a custom template. Before running unittests we use luarocks to download all dependencies needed for the unittests to run https://github.com/ColinKennedy/nvim-best-practices-plugin-template/blob/4590508ec1ced872899330b216b75ffc97eca982/.github/workflows/test.yml#L44. e.g. luarocks test foo-scm-1.rockspec --prepare

If I replace the foo-scm-1.rockspec with a more generic template.rockspec - that template is not a real rockspec and cannot be loaded as Lua as-is. And so luarocks cannot read it for dependencies and install them. I don't know of a way to keep this template without causing issues elsewhere.

There is a simple-but-not-ideal solution to the problem which is to replace luarocks test foo-scm-1.rockspec --prepare with just manually installing every dependency. e.g.

luarocks install dependency_a
luarocks install dependency_b
luarocks install dependency_c

luarocks install test_dependency_a
luarocks install test_dependency_b
luarocks install test_dependency_c
luarocks install test_dependency_d

Except that has its own issue - which is that the dependencies listed in a .rockspec will have version ranges but the commands above are just going to install the latest of those packages. And luarocks install doesn't accept version ranges such as luarocks install foo '>=1.0, < 1.0' so we can't just hard-code the version range into that duplicate request.

Right now I can only think of a couple ways out of this. The simplest-but-not-simple initial thought was "what if I expanded the template.rockspec, wrote it to a temporary file, then ran luarocks on that". Sort of like a "do the rockspec expansion but don't actually release it to luarocks (so we can use the expanded file for other purposes)" Which would likely require changes to luarocks-tag-release.

What do you think about all of this?

[mrcjkb]

you can still use an scm rockspec for tests. luarocks-tag-release just ignores it.
It's a bit unfortunate to have two sources of truth, which is why I opened #435.
As of now, we're focusing our priorities on rocks development.
Once rocks has matured, we will replace luarocks with it in this workflow. It will have the ability to generate release rockspecs and also run busted tests from either a project.rockspec or a rocks.toml in the project root. This will make #435 obsolete.

[ColinKennedy]

I understand wanting to do things the right way. Though I would prefer writing a stop-gap replacement tool so I can keep a single source of truth than having to have two files and explaining that to anyone wanting to use my plugin template. If it's easy to do would this repo make a second alt-action that does just the file expansion without any of the upload logic? Alternatively I think I could make my own repo and reuse the Rockspec.generate code to directly write to-disk.

[mrcjkb]

In the Lua ecosystem, it's been common practise to have a rockspecs subdirectory with a separate rockspec for each release.
Having a template has been a huge step up from that.

We're currently focusing all our efforts on getting rocks ready for use.
If you don't want to wait for that, PRs to this repo are welcome 😃

[ColinKennedy]

Right, I forgot that the template is a stopgap in itself. So I'd be asking for a stopgap for a stopgap :)

I just looked on GitHub's documentation https://docs.github.com/en/actions/sharing-automations/creating-actions/publishing-actions-in-github-marketplace - this is the important line "Each repository must contain a single action."

So it looks like adding a multi-action situation isn't going to cut it. I'll see if I can make a separate repository+action and reuse as much as possible from here

[mrcjkb]

you can create composite actions that are composed of multiple existing actions.