Updates for self-hosted runner

[cboettig]

@jzwart I've needed to juggle things for the self-hosted runner to better accommodate loads.

For any tasks that need a self-hosted runner, you will need to request the efi-cirrus runner group as in this example, like so:

    runs-on: efi-cirrus
    container: 
      # IMPORTANT.  Please set a memory limit <= 45 GB.
      image: eco4cast/rocker-neon4cast:latest
      options: --memory="15g"
    steps:

It's really important you include the options: memory="15g" (or whatever up to 45 g) in any workflows on the self-hosted runner. (Unfortunately/astonishingly, GitHub ARC doesn't provide a mechanism to enforce this on runners that use custom docker images other than opting in to limits in the job yaml).

[jzwart]

Thanks @cboettig . The jobs I've requested for efi-cirrus have been queued and never picked up after several hours, e.g., https://github.com/eco4cast/usgsrc4cast-ci/actions/runs/12559914161/job/35016390626 and it seems like there shouldn't be other jobs occupying that runner at the moment. Is the runner setup for running jobs for all repos in eco4cast org?

[cboettig]

@jzwart thanks for the ping! yup you're right, I needed to manually add any public repos to the whitelist for the runner group in https://github.com/organizations/eco4cast/settings/actions/runner-groups/1. done now.

[cboettig]

you should see that you now have a runner listed in this repo's runner list in https://github.com/eco4cast/usgsrc4cast-ci/settings/actions/runners . We probably need to make you an org admin for you to see the previous link for all public repos that can access the runner. @rqthomas should we bump Jake to admin? also should we have a governance process for that?

[cboettig]

@jzwart for https://github.com/eco4cast/usgsrc4cast-ci/actions/runs/12559914161/job/35016390626, can you drop the use of futures here? gdal is already threaded anyway so I don't think it's helping that much.

[jzwart]

Thanks! it works now and I removed futures in the stage 3 update.

I think @rqthomas and I were supposed to work on a governance process for eco4cast org, but we haven't gotten around to doing that yet. This could be motivation to do so