Overhaul software management

[messersc]

Current

One conda env (.yaml), which holds all software necessary to run.

Problems

E.g. #10 with multiple java versions

Solution

Disentangle software used, either by rules, or at least have one environment for R, for python, for java.

Conda

A) build a new conda env for each rule

Snakemake allows to build new conda environments per rule. This might be unwanted, because it takes quite long, e.g. in the case of R. The upside is that the software versions match sea-snap expectations always, which is not guaranteed if one used outside conda envs.

B) have conda environments pre-build (per user)

Snakemake can also use pre-existing conda-envs. So we could have a script that generates e.g.

seasnap-star-v1
seasnap-ipynb-v1
seasnap-

Singularity

If R is too annoying in tandem with conda, we can also go the singularity route and have one image accessible for all users with the required packages. We have some experience with this, currently with irods-icommands and bclconvert.

Drawbacks

With conda option B) and singularity, we would probably need some way to version the envs/images and enforce correct versions in sea-snap. Also, because the latter two options are not self-contained with sea-snap, users would need to set up envs on each machine they work on, i.e. once on cluster, once on workstation and so on.

[mmilek]

In the development branch I just added 2 conda env yaml files for the mapping+sc and DE pipeline that have package versions included.
c01be7f