Add customization for fetching SLURM job id

[BramVanroy]

SLURM job IDs are hard to fetch. The current implementation does not flexibly allow for users to retrieve the right ID based on their printed output (which may be different depending on the implementation). As an example, when submitting an sbatch job, the printed string on our system is

Submitted batch job 9914862 on cluster dodrio

The current default implementation simply splits this string on space and selects the last item. In my case that does not make sense, so it greatly depends on how this info is printed.

I therefore suggest to allow the pipeline constructor to have a job_id_retriever Callable argument. Default behavior is the same as it is now. However, if a user provides a callable, they can choose how the process output is returned.

So in my case, I would add

def job_id_retriever(job_id: str) -> str:
    return re.search(r"Submitted batch job (\d+)", job_id).group(1)

SlurmPipelineExecutor(
    ...,
    job_id_retriever=job_id_retriever,
    ...
)

This funcion will be used to postprocess the retrieved submission text.

As said, the default behavior is the same.

[guipenedo]

Hi, would the recently merged job_id_position not be enough in this case?

[BramVanroy]

Hi, would the recently merged job_id_position not be enough in this case?

In my case yes, but I figured - since I had this change done already - I'd do a PR to allow for extended user modification. To catch strings like, e.g. "Submitted job #123" or "Started id-123", or languages that do not use spaces as word delimiters, or whichever odd variation might be out there.

[guipenedo]

partial seems to be missing an import

[BramVanroy]

partial seems to be missing an import

Fixed!