Evaluation workshop for the 2nd round of CAMI challenges
March 12-13, 2020. Braunschweig, Germany
Location: Room 307/308 at the Braunschweig Integrated Centre of Systems Biology (BRICS), Rebenring 56, 38106 Braunschweig
Organizers: Alice McHardy (Helmholtz Centre for Infection Research), Alexander Sczyrba (Bielefeld University), Fernando Meyer (Helmholtz Centre for Infection Research)
Hotel (for those with a confirmed reservation): IntercityHotel Braunschweig, Willy-Brandt-Platz 3, 38102 Braunschweig. Directions and reimbursement information is available here.
Overview
The interpretation of metagenomes relies on sophisticated computational methods for read assembly, binning, and taxonomic classification. To enable meaningful comparisons between methods, identify best practices and use cases, and reduce overhead involved in benchmarking, standards in terms of evaluation data sets, procedures, and metrics are required. In 2014, the Critical Assessment of Metagenome Interpretation (CAMI) initiative was founded to address this need.
Based on the feedback from the first CAMI challenge and suggestions received in previous workshops and conferences, the second round of CAMI challenges was launched in January 2019. In this workshop, the results of the challenges will be analyzed and discussed with participating developer teams. In addition, the focus of future CAMI challenges will be discussed. All talks will be streamed. Interested scientists are encouraged to join.
Registration
Please register by sending an email to This email address is being protected from spambots. You need JavaScript enabled to view it.. Requests for travel support, together with a cost estimate, should be sent to Marie-Louise as well.
The registration deadline is March 6, 2020. To guarantee a hotel room, please register by February 24, 2020.
Virtual participation
It will be possible to view the workshop remotely. Contact This email address is being protected from spambots. You need JavaScript enabled to view it. for connection details.
There is also a Gitter chatroom for the workshop at: https://gitter.im/CAMI-Workshop
Workshop sessions
- Presentation of the results of the CAMI II challenges
- Practical sessions, where workshop participants can work on writing the challenge paper and generating further data visualization plots
- A third presentation session, where speakers present their thoughts and potential contributions to future CAMI challenges
Timetable (tentative)
Thursday, March 12
| Session 1: The results of the 2nd CAMI challenge | |
| 9:00 am – 9:30 am | Registration |
| 9:30 am – 9:45 am |
Welcome Alice McHardy, Helmholtz Centre for Infection Research, Germany Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/ZSHFBNMK5bDWKJS |
| 9:45 am – 10:30 am |
CAMI 2 challenge data & simulation pipeline Adrian Fritz, Helmholtz Centre for Infection Research, Germany Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/FLDrnpPQP8HFZkz |
| 10:30 am – 11:00 am | Coffee break |
| 11:00 am – 11:45 am |
An overview of the assembly results Alexander Sczyrba, University of Bielefeld, Germany Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/tp4g2XG9GyGqQrG |
| 11:45 am – 12:30 pm |
An overview of the profiling results David Koslicki, Penn State University, United States Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/2R42BfaTe96Csje |
| 12:30 pm – 1:30 pm | Lunch |
| 1:30 pm – 2:15 pm |
An overview of the binning results Fernando Meyer, Helmholtz Centre for Infection Research, Germany Presentation unavailable |
| 2:15 pm – 2:30 pm |
CAMI2: Pathogen Detection Piotr Wojciech Dabrowski, HTW Berlin, Germany Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/QCgmYWNgS5qroF9 |
| 2:30 pm – 3:00 pm |
An overview of the results of the clinical pathogen detection challenge Zhi-Luo Deng, Helmholtz Centre for Infection Research, Germany Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/XnL4Ef96q5LPz3i |
| 3:00 pm – 3:30 pm | Coffee break |
| Session 2: Guest talks + CAMI 2 data analysis and manuscript preparation | |
| 3:30 pm – 3:55 pm |
Benchmarking taxonomic profiling in metagenomics - some thoughts from a tool developer's perspective Shinichi Sunagawa, ETH Zurich, Switzerland Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/abroCdjNEcf3KBw |
| 3:55 pm | Definition of members and program for working groups (further data analyses and manuscript section writing teams) |
| 3:55 pm – 6:00 pm | Activities within working groups |
| 6:30 pm | Dinner at L’Osteria Braunschweig BraWoPark Shopping Center, Brawo Allee 1, 38102 Braunschweig |
Friday, March 13
| Session 3: Guest talks + future challenges discussion | |
| 9:00 am – 9:25 am |
Benchmarking challenges in microbial genome assembly Rayan Chikhi, University of Lille, France Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/ncZrCeDMkPA2Xpo |
| 9:25 am – 9:50 am |
Past and future of (meta)genome assembly evaluation methods Alexey Gurevich, St. Petersburg State University, Russia Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/FiGAY9jDcf8mZq3 |
| 9:50 am – 10:15 am |
Taxonomic profiling: emerging challenges and shifting paradigms David Koslicki, Penn State University, United States Presentation link: https://nextcloud.bifo.helmholtz-hzi.de/s/TH7nbdg3EyTC9aP |
| 10:15 am – 10:45 am | Coffee break |
| 6:00 pm | Estimated end of workshop |