Merge branch 'release/1.2.0'

.codeclimate.yml

@@ -0,0 +1,29 @@
+engines:
+  duplication:
+    enabled: true
+    config:
+      languages:
+        python:
+          python_version: 3
+          mass_threshold: 50
+  fixme:
+    enabled: true
+  markdownlint:
+    enabled: true
+  pep8:
+    enabled: true
+  radon:
+    enabled: true
+    config:
+      threshold: "C"
+
+ratings:
+  paths:
+    - "**.py"
+    - "**.md"
+
+exclude_paths:
+  - "git-hooks/"
+  - "data/"
+  - "tests/"
+  - "pylintrc"

.gitignore

@@ -4,7 +4,8 @@ __pycache__/
 *.py[cod]
 *$py.class
 .coverage
-cover/
+.cache
+htmlcov/
 /.eggs
 /build
 /dist

.travis.yml

@@ -0,0 +1,29 @@
+notifications:
+  slack: wtsi-cgpit:ptUMR1tkNyZJYd9TpGoss8WR
+  email: false
+
+env:
+  global:
+    - CC_TEST_REPORTER_ID=24af7ff1e88f3b70c8b9a5280ce9604d561dacd8eaa7b1d895128ca2bd724beb
+
+language: python
+
+python:
+  - "3.4"
+  - "3.6"
+
+install:
+  - pip install pytest pytest-cov
+  - pip install progressbar2
+  - pip install xlrd
+
+before_script:
+  - curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
+  - chmod +x ./cc-test-reporter
+  - ./cc-test-reporter before-build
+
+script:
+  - pytest --cov-branch --cov-report term --cov=cgp_seq_input_val
+
+after_script:
+  - ./cc-test-reporter after-build --exit-code $TRAVIS_TEST_RESULT

README.md

@@ -11,7 +11,9 @@ the driver code.
 
 ## Tools
 
-### normalise_manifest.py
+`cgpSeqInputVal` has multiple sub commands, listed with `cgpSeqInputVal --help`.
+
+### cgpSeqInputVal man-norm
 
 Takes input in multiple types and converts to tsv.  If intput is tsv just copied
 the file to the output location (to simplify usage in workflows).  Valid input types
@@ -24,7 +26,7 @@ include:
 
 Absolutely no validation is carried out here.
 
-### validate_manifest.py
+### cgpSeqInputVal man-valid
 
 Takes the `tsv` representation of a manifest and performs validation of the structure
 and data values.  The checks applied are managed by the `cgp_seq_input_val/config/*.json`
@@ -37,9 +39,10 @@ The output is a lightly modified version of the input, adding:
 
 And a `json` version of the file ready for use by downstream systems.
 
-### validate_seq_file.py
+### cgpSeqInputVal seq-valid
 
-Takes an interleaved or a pair of paired-fastq files and produces a simple report of:
+Takes an interleaved or a pair of paired-fastq files and produces a simple report
+of:
 
 ```
 {
@@ -53,77 +56,114 @@ Various exceptions can occur for malformed files.
 
 The primary purpose is to confirm Sanger/Illumina 1.8+ quality scores.
 
-#### Why no BAM/CRAM input?
+#### FASTQ not BAM/CRAM
 
 The flow of the service data will require splitting of any multi-lane BAM/CRAM files
 down to the individual lanes, which we would do to interleaved fastq.  There is no
 current need to parse BAM/CRAM files to check quality encoding directly as the spec
-technically disallows it.  It is possible for BAM files to be incorrectly encoded though.
-
+technically disallows it.  It is possible for BAM files to be incorrectly encoded
+though.
 
-# INSTALL
+## INSTALL
 
-Installation is via `easy_install`.  Simply execute with the path to the compiled 'egg':
+Installation is via `pip`.  Simply execute with the path to the packaged distribution:
 
 ```bash
-easy_install bundles/cgp_seq_input_val-0.1.0-py3.6.egg
+pip install --find-links=~/wheels cgp_seq_input_val
 ```
 
-## Package Dependancies
+### Package Dependancies
 
-`easy_install` will install the relevant dependancies, listed here for convenience:
+`pip` will install the relevant dependancies, listed here for convenience:
 
 * [progressbar2](http://progressbar-2.readthedocs.io/en/latest/)
 * [xlrd](https://github.com/python-excel/xlrd)
 
+## Development environment
 
-# Development environment
-This project uses git pre-commit hooks.  As these will execute on your system it is entirely up to you if you activate them.
+This project uses git pre-commit hooks.  As these will execute on your system it
+is entirely up to you if you activate them.
 
-If you want tests, coverage reports and lint-ing to automatically execute before a commit you can activate them by running:
+If you want tests, coverage reports and lint-ing to automatically execute before
+a commit you can activate them by running:
 
 ```
 git config core.hooksPath git-hooks
 ```
 
-Only a test failure will block a commit, lint-ing is not enforced (but please consider following the guidance).
+Only a test failure will block a commit, lint-ing is not enforced (but please consider
+following the guidance).
 
-You can run the same checks manually without a commit by executing the following in the base of the clone:
+You can run the same checks manually without a commit by executing the following
+in the base of the clone:
 
 ```bash
-./run_tests.py
+./run_tests.sh
 ```
 
-## Development Dependencies
+### Development Dependencies
 
-### Setup VirtualEnv:
+#### Setup VirtualEnv
 
 ```
 cd $PROJECTROOT
 hash virtualenv || pip3 install virtualenv
 virtualenv -p python3 env
-env/bin/pip install progressbar2
-env/bin/pip install xlrd
+source env/bin/activate
+pip install progressbar2
+pip install xlrd
+python setup.py develop # so bin scripts can find module
 ```
 
 For testing/coverage (`./run_tests.sh`)
 
 ```
-env/bin/pip install nose
-env/bin/pip install coverage
-env/bin/pip install pylint
+source env/bin/activate # if not already in env
+pip install pytest
+pip install pytest-cov
+pip install pep8
+pip install radon
+gem install --user-install mdl
 ```
 
-__Also see [Package Dependancies](#package-dependancies)__
+Test that `mdl` is available, if not add the following to your path variable:
+
+```
+export PATH=$HOME/.gem/ruby/X.X.X/bin:$PATH
+```
 
-## Cutting a release
+### Cutting a release
 
-__Make sure the version is incremented in ./setup.py__
+__Make sure the version is incremented__ in `./setup.py`
 
-The release is handled by setuptools:
+The release is handled by wheel:
 
 ```bash
-$ ./setup.py bdist_egg
-# this creates an egg which can be copied to a deployment location, e.g.
-scp dist/cgp_seq_input_val-0.1.0-py3.6.egg user@host:~/
+$ source env/bin/activate # if not already
+$ python setup.py bdist_wheel -d dist
+# this creates an wheel archive which can be copied to a deployment location, e.g.
+$ scp cgp_seq_input_val-1.1.0-py3-none-any.whl user@host:~/wheels
+# on host
+$ pip install --find-links=~/wheels cgp_seq_input_val
 ```
+
+## LICENCE
+
+Copyright (c) 2017 Genome Research Ltd.
+
+Author: CancerIT <[email protected]>
+
+This file is part of cgp_seq_input_val.
+
+cgp_seq_input_val is free software: you can redistribute it and/or modify it under
+the terms of the GNU Affero General Public License as published by the Free
+Software Foundation; either version 3 of the License, or (at your option) any
+later version.
+
+This program is distributed in the hope that it will be useful, but WITHOUT
+ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program. If not, see <http://www.gnu.org/licenses/>.

cgp_seq_input_val/cliutil.py

@@ -1,6 +1,7 @@
 """General command line utility functions"""
 import os
 
+
 def extn_check(parser, choices, fname, readable=False):
     """Checks file extensions fit expected sets
 
@@ -13,7 +14,9 @@ def extn_check(parser, choices, fname, readable=False):
             handle = open(fname, 'r')
             handle.close()
         except FileNotFoundError as error:
+            # can't cover these easily
             parser.error(error)
     if extn not in choices:
+        # can't cover these easily
         parser.error("File doesn't end with {}".format(choices))
     return fname