Resources

Datasets and online resources

• CAMI community results on Zenodo
• CAMI 2 - Multisample Benchmark Dataset of Mouse Gut, doi:10.4126/FRL01-006421672
• CAMI 2 - Multisample Benchmark Dataset of Human Microbiome Project, doi:10.4126/FRL01-006425518
• CAMI 2 - Challenge Datasets, doi:10.4126/FRL01-006425521
• Benchmark data sets and tool results of the first CAMI challenge
  Belmann, P., Bremges, A., Dahms, E. et al. Benchmark data sets, software results and reference data for the first CAMI challenge. GigaScience Database (2017)
• CAMI datasets and benchmarking portal

Community

• Follow @microbiome_cosi on Twitter
• Follow @CAMI_challenge on Twitter
• Join the CAMI session of the MICROBIOME track at ISMB 2022
• MICROBIOME YouTube channel

Articles, reports, blogs and editorials about CAMI

• Meyer, F., Fritz, A. et al. Critical Assessment of Metagenome Interpretation - the second round of challenges. Nature Methods (2022), doi:10.1038/s41592-022-01431-4
• McHardy, A., Meyer, F. CAMI II: identifying best practices and issues for metagenomics software, Nature Methods, 19 (2022), 412–413
• Meyer, F., Fritz, A., McHardy, A. Critical Assessment of Metagenome Interpretation – the second round of benchmarking challenges provides a comprehensive overview of metagenomic software performance, Behind the Paper, Protocols & Methods Community (2022)
• Bremges, A., McHardy, A.C. Critical Assessment of Metagenome Interpretation enters the second round, mSystems, 3, 4 (2018). pii: e00103-18
• Sczyrba, A., Hofmann, P., Belmann, P. et al. Critical Assessment of Metagenome Interpretation – a benchmark of metagenomics software. Nature Methods, 14, 11 (2017), 1063-1071
• The Critical Assessment of Metagenome Interpretation (CAMI) competition, by Tal Nawy, on Methagora
• Q & A: Alice McHardy on a Planned Initiative to Assess Metagenome Analysis Software, on Genome Web

Tutorials and formats specifications

• Meyer et al. Tutorial: assessing metagenomics software with the CAMI benchmarking toolkit. Nature Protocols (2021), doi:10.1038/s41596-020-00480-3
• CWL-based Bioboxes
• Assembly, (taxonomic) binning, and profiling CAMI file formats

Software

• CAMISIM, software for simulating metagenomes and microbial communities
  https://github.com/CAMI-challenge/CAMISIM/
  Fritz*, Hofmann*, et al. CAMISIM: Simulating metagenomes and microbial communities. Microbiome, 7:17. doi:10.1186/s40168-019-0633-6 (2019)
• AMBER, a package for genome binning assessment implementing metrics used in CAMI I
  https://github.com/CAMI-challenge/AMBER
  Meyer, F. et al. AMBER: Assessment of Metagenome BinnERs, GigaScience, 7, 6, doi:10.1093/gigascience/giy069 (2018)
• OPAL, a package for assessing taxonomic profilers implementing metrics used in CAMI I
  https://github.com/CAMI-challenge/OPAL
  Meyer, F. et al. Assessing taxonomic metagenome profilers with OPAL, Genome Biology, 20:51, doi:10.1186/s13059-019-1646-y (2019)
• Bioboxes, docker containers with standardised interfaces for interchangeable bioinformatics software
  http://bioboxes.org/
  Belmann, P. et al. Bioboxes: standardised containers for interchangeable bioinformatics software. GigaScience, 4, 47 (2015)
• MetaQUAST evaluates and compares metagenome assemblies based on alignments to close references
  http://quast.sourceforge.net/metaquast.html
  Mikheenko, A., Saveliev, V., and Gurevich, A. MetaQUAST: evaluation of metagenome assemblies. Bioinformatics, 32, 7 (2016), 1088-1090

Assembler benchmarking

• Nucleotid.es: continuous, objective and reproducible evaluation of genome assemblers using docker containers