1) AfterQC is slow 2)Aggregate results from many samples into a single report

[gk-bioin4m8x]

Hi,
I am running 20 paired-end RNA-seq samples since yesterday (more than 24 hours over) and only 7 samples have been completed (others are still running) on 16 GB RAM computer.
Any way to make it faster?

Secondly, I am wondering if there is any possibility to aggregate results from many samples into a single report?As per MultiQC [https://github.com/ewels/MultiQC], AfterQC is not in their list of supported tools.

Any suggestions please!
Thanks!

[alnf]

I second this:) I had to analyze with FastQC after AfterQC, so I can aggregate with MultiQC. I think maybe adding issue to MultiQC repo can motivate authors to implement AfterQC support.

[sfchen]

@gk-bioin4m8x have you installed editdistance module? Are u using pypy or native python?

[alnf]

@gk-bioin4m8x Better to use different issues for different things. I was replying to the aggregation issue only.

[gk-bioin4m8x]

@sfchen
I am using AfterQC inside cygwin (on 64-bit WIndows 8.1). I did as follows inside AfterQC folder:

# editdistance
>make editdistance                        # displayed message of successful installation

# bash script for multiple samples (only showing the syntax)
>python after.py -d input_folder -g good_out -b bad_out

[gk-bioin4m8x]

@alnf
I was editing my issue and saw your reply after I submitted comment. I thought may be because of many samples it may be slow, so combined two issues.
.

[sfchen]

@gk-bioin4m8x How big is your sample data? Did you run them concurrently, or just one by one?

[sfchen]

@gk-bioin4m8x I saw you run with the whold folder, it is correct. But performance may be decreased in cygwin.

If you didn't see any warning information of editdistance when you run AfterQC, then it is installed well.

[gk-bioin4m8x]

@sfchen

1. Each sample is around 10-12 GB (R1 ~ 5-6GB, R2 ~ 5-6GB).
2. I ran them together via bash script.
3. Yes, no warning while installing editdistance.

[gk-bioin4m8x]

@sfchen As integration of AfterQC inside MultiQC is in process, meanwhile do you recommend any other way to integrate AfterQC output from multiple samples?

[sfchen]

gk-bioin4m8x how about a 2-column framed homepage, whose left column contains links to different samples and right column is the QC report of corresponding sample?

[gk-bioin4m8x]

I have AfterQC output from 20 samples, so 20 html files with following features:

1. AfterQC summary (for General Stats table):
   sequencing: .......... pair end
   estimated seq error: ...........%
   total reads: ...........
   filtered out reads: .......... (.................%)
   total bases: ..........................
   filtered out bases: ........................... (.............................%)
   auto trimming front:...., tail:.... (use -f0 -t0 to disable)
2. Good reads and bad reads after filtering (filtering statistics)
3. Sequencing error transform distribution
4. Pair Overlap length histogram
5. Read1 quality curve before filtering
6. Read1 base content distribution before filtering
7. Read1 GC curve before filtering
8. Read1 per base discontinuity before filtering
9. Read1 kmer strand bias before filtering
10. Read1 quality curve after filtering
11. Read1 base content distribution after filtering
12. Read1 GC curve after filtering
13. Read1 per base discontinuity after filtering
14. Read1 kmer strand bias after filtering
15. Read2 quality curve before filtering
16. Read2 base content distribution before filtering
17. Read2 GC curve before filtering
18. Read2 per base discontinuity before filtering
19. Read2 kmer strand bias before filtering
20. Read2 quality curve after filtering
21. Read2 base content distribution after filtering
22. Read2 GC curve after filtering
23. Read2 per base discontinuity after filtering
24. Read2 kmer strand bias after filtering

I am interested for:
(i) a common table with columns from summary section for all samples (good for comparison).
(ii) A combined plot for each feature (2 to 24) from each sample. For e.g. 23 plots, each plot shows aggregated results from all samples.

[sfchen]

Got your ideas. You mean each plot combines results of all samples.

[gk-bioin4m8x]

Yes and it will be very helpful in comparisons. :)

[sfchen]

Seems a good idea, although it may take more effort to implement. I will figure out how to realize that.

[gk-bioin4m8x]

Just curious to know, how much time will it take. :P

[sfchen]

As I mentioned above, aggregating all results is not easy and need more effort. I think it may take a couple of weeks, considering that I am also busy on other projects (e.g MutScan)

[gk-bioin4m8x]

No problem, take your time. :)

[sfchen]

AfterQC should be much faster with peppy now. Please try v0.9.4