Evaluation workshop for the 2nd round of CAMI challenges

March 12-13, 2020. Braunschweig, Germany


Location: Room 44 at the Braunschweig Integrated Centre of Systems Biology (BRICS), Rebenring 56, 38106 Braunschweig

Organizers: Alice McHardy (Helmholtz Centre for Infection Research), Alexander Sczyrba (Bielefeld University), Fernando Meyer (Helmholtz Centre for Infection Research)


The interpretation of metagenomes relies on sophisticated computational methods for read assembly, binning, and taxonomic classification. To enable meaningful comparisons between methods, identify best practices and use cases, and reduce overhead involved in benchmarking, standards in terms of evaluation data sets, procedures, and metrics are required. In 2014, the Critical Assessment of Metagenome Interpretation (CAMI) initiative was founded to address this need.
Based on the feedback from the first CAMI challenge and suggestions received in previous workshops and conferences, the second round of CAMI challenges was launched in January 2019. In this workshop, the results of the challenges will be analyzed and discussed with participating developer teams. In addition, the focus of future CAMI challenges will be discussed. All talks will be streamed. Interested scientists are encouraged to join.



Please register by sending an email to This email address is being protected from spambots. You need JavaScript enabled to view it.. Requests for travel support, together with a cost estimate, should be sent to Annie as well. Accommodation and reimbursement information is available here.

The registration deadline is March 6, 2020. To guarantee a hotel room, please register by February 24, 2020.


Workshop sessions

  • Presentation of the results of the CAMI II challenges
  • Practical sessions, where workshop participants can work on writing the challenge paper and generating further data visualization plots
  • A third presentation session, where speakers present their thoughts and potential contributions to future CAMI challenges


Timetable (tentative)

Thursday, March 12

Session 1: The results of the 2nd CAMI challenge
9:00 am – 9:30 am Registration
9:30 am – 9:45 am Welcome
Alice McHardy, Helmholtz Centre for Infection Research, Germany
9:45 am – 10:30 am CAMI 2 challenge data & simulation pipeline
10:30 am – 11:00 am Coffee break
11:00 am – 11:45 am An overview of the assembly results
Alexander Sczyrba, University of Bielefeld, Germany
11:45 am – 12:30 pm An overview of the profiling results
David Koslicki, Oregon State University, United States
12:30 pm – 1:30 pm Lunch
1:30 pm – 2:15 pm An overview of the binning results
Fernando Meyer, Helmholtz Centre for Infection Research, Germany
2:15 pm – 2:30 pm TBA
Piotr Wojciech Dabrowski Deng, HTW Berlin, Germany
2:30 pm – 3:00 pm An overview of the results of the clinical pathogen detection challenge
Zhi-Luo Deng, Helmholtz Centre for Infection Research, Germany
3:00 pm – 3:30 pm Coffee break
Session 2: Guest talks + CAMI 2 data analysis and manuscript preparation
3:30 pm – 3:55 pm TBA
Shinichi Sunagawa, ETH Zurich, Switzerland
3:55 pm Definition of members and program for working groups (further data analyses and manuscript section writing teams)
3:55 pm – 6:00 pm Activities within working groups


6:30 pm Dinner at L’Osteria Braunschweig
BraWoPark Shopping Center, Brawo Allee 1, 38102 Braunschweig

Friday, March 13

Session 3: Guest talks + future challenges discussion
9:00 am – 9:25 am Benchmarking challenges in microbial genome assembly
Rayan Chikhi, University of Lille, France
9:25 am – 9:50 am Past and future of (meta)genome assembly evaluation methods
Alexey Gurevich, St. Petersburg State University, Russia
9:50 am – 10:15 am Taxonomic profiling: emerging challenges and shifting paradigms
David Koslicki, Oregon State University, United States
10:15 am – 10:45 am Coffee break
6:00 pm Estimated end of workshop