Evaluation workshop for the 2nd round of CAMI challenges

March 12-13, 2020. Braunschweig, Germany


Location: Room 307/308 at the Braunschweig Integrated Centre of Systems Biology (BRICS), Rebenring 56, 38106 Braunschweig

Organizers: Alice McHardy (Helmholtz Centre for Infection Research), Alexander Sczyrba (Bielefeld University), Fernando Meyer (Helmholtz Centre for Infection Research)

Hotel (for those with a confirmed reservation): IntercityHotel Braunschweig, Willy-Brandt-Platz 3, 38102 Braunschweig. Directions and reimbursement information is available here.



The interpretation of metagenomes relies on sophisticated computational methods for read assembly, binning, and taxonomic classification. To enable meaningful comparisons between methods, identify best practices and use cases, and reduce overhead involved in benchmarking, standards in terms of evaluation data sets, procedures, and metrics are required. In 2014, the Critical Assessment of Metagenome Interpretation (CAMI) initiative was founded to address this need.
Based on the feedback from the first CAMI challenge and suggestions received in previous workshops and conferences, the second round of CAMI challenges was launched in January 2019. In this workshop, the results of the challenges will be analyzed and discussed with participating developer teams. In addition, the focus of future CAMI challenges will be discussed. All talks will be streamed. Interested scientists are encouraged to join.



Please register by sending an email to This email address is being protected from spambots. You need JavaScript enabled to view it.. Requests for travel support, together with a cost estimate, should be sent to Marie-Louise as well.

The registration deadline is March 6, 2020. To guarantee a hotel room, please register by February 24, 2020.


Virtual participation

It will be possible to view the workshop remotely. Contact This email address is being protected from spambots. You need JavaScript enabled to view it. for connection details.

There is also a Gitter chatroom for the workshop at: https://gitter.im/CAMI-Workshop


Workshop sessions

  • Presentation of the results of the CAMI II challenges
  • Practical sessions, where workshop participants can work on writing the challenge paper and generating further data visualization plots
  • A third presentation session, where speakers present their thoughts and potential contributions to future CAMI challenges


Timetable (tentative)

Thursday, March 12

Session 1: The results of the 2nd CAMI challenge
9:00 am – 9:30 am Registration
9:30 am – 9:45 am


Alice McHardy, Helmholtz Centre for Infection Research, Germany

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/ZSHFBNMK5bDWKJS

9:45 am – 10:30 am

CAMI 2 challenge data & simulation pipeline

Adrian Fritz, Helmholtz Centre for Infection Research, Germany

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/FLDrnpPQP8HFZkz

10:30 am – 11:00 am Coffee break
11:00 am – 11:45 am

An overview of the assembly results

Alexander Sczyrba, University of Bielefeld, Germany

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/tp4g2XG9GyGqQrG

11:45 am – 12:30 pm

An overview of the profiling results

David Koslicki, Penn State University, United States

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/2R42BfaTe96Csje

12:30 pm – 1:30 pm Lunch
1:30 pm – 2:15 pm

An overview of the binning results

Fernando Meyer, Helmholtz Centre for Infection Research, Germany

Presentation unavailable

2:15 pm – 2:30 pm

CAMI2: Pathogen Detection

Piotr Wojciech Dabrowski, HTW Berlin, Germany

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/QCgmYWNgS5qroF9

2:30 pm – 3:00 pm

An overview of the results of the clinical pathogen detection challenge

Zhi-Luo Deng, Helmholtz Centre for Infection Research, Germany

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/XnL4Ef96q5LPz3i

3:00 pm – 3:30 pm Coffee break
Session 2: Guest talks + CAMI 2 data analysis and manuscript preparation
3:30 pm – 3:55 pm

Benchmarking taxonomic profiling in metagenomics - some thoughts from a tool developer's perspective

Shinichi Sunagawa, ETH Zurich, Switzerland

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/abroCdjNEcf3KBw

3:55 pm Definition of members and program for working groups (further data analyses and manuscript section writing teams)
3:55 pm – 6:00 pm Activities within working groups


6:30 pm Dinner at L’Osteria Braunschweig
BraWoPark Shopping Center, Brawo Allee 1, 38102 Braunschweig

Friday, March 13

Session 3: Guest talks + future challenges discussion
9:00 am – 9:25 am

Benchmarking challenges in microbial genome assembly

Rayan Chikhi, University of Lille, France

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/ncZrCeDMkPA2Xpo

9:25 am – 9:50 am

Past and future of (meta)genome assembly evaluation methods

Alexey Gurevich, St. Petersburg State University, Russia

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/FiGAY9jDcf8mZq3

9:50 am – 10:15 am

Taxonomic profiling: emerging challenges and shifting paradigms

David Koslicki, Penn State University, United States

Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/TH7nbdg3EyTC9aP

10:15 am – 10:45 am Coffee break
6:00 pm Estimated end of workshop