EBPhD Methods seminar – Computationally reproducible research
Session 1 – Kickoff: key properties and definition of computational reproducibility
23 September 2026
Introduction round
- What’s your name?
- What do you expect from this seminar?
- What is one analysis of yours you would not like to rerun from scratch tomorrow; and why?
Today
| Time | What | Who |
|---|---|---|
| 15:00 | Welcome, why this seminar, how it works | Instructors |
| 15:15 | Input 1: Reproducibility in epidemiology (15’ + 15’ discussion) | Group 1 |
| 15:45 | Input 2: Reproducibility in biostatistics & code review (15’ + 15’) | Group 2 |
| 16:15 | Short break | |
| 16:20 | Peer discussion + what we learned today | Everyone |
| 16:35 | Logistics: paper pool, GitLab & Quarto, groups, homework | Instructors |
| 17:00 | End |
Why this seminar
Reproducibility is usually taught as a list of principles
This seminar: the hard way
- Take a published paper.
- Find the numbers that support the main claim of the paper.
- Try to get the same numbers by rerunning the analysis reported in the paper (using all materials available; potentially data and code, often only methods and results sections).
- Understand why things work and most importantly why they don’t.
The output: one joint paper in Replication Research
- Replication Research (R2) is Diamond open access that explicitly welcomes reproductions.
- One combined manuscript: each group’s reproduction is a case.
- We will also preprint the manuscript!
- A draft exists by the end of term; submission happens after the seminar
Everyone is offered authorship.
Volunteers who carry the manuscript to submission after term get lead author positions.
Seven sessions
| # | Date | Session |
|---|---|---|
| 1 | 23 Sept | Kickoff: key properties and definition of computational reproducibility |
| 2 | 7 Oct | Writing a reproduction protocol → final paper selection |
| 3 | 14 Oct | Quarto & GitLab as a reproducibility toolkit → repo setup in class |
| → 5 weeks: finalise, lock and peer review the protocol, start executing | ||
| 4 | 18 Nov | Guest lecture: forensic metascience → status check |
| 5 | 25 Nov | Continued execution, troubleshooting & peer review |
| 6 | 2 Dec | Results and interpretation |
| 7 | 9 Dec | Write-up, submission mechanics & wrap-up |
All sessions take place on Wednesday 15:00–17:00.
Input 1 & 2
Break
5 minutes
Peer discussion
Peer discussion
In pairs (5 min), then plenary (10 min), discuss:
What do I understand by reproducibility?
What do I already do to ensure computational reproducibility of my own work?
Logistics
Paper pool · GitLab & Quarto · Groups · “Homework”
Reproduction targets: why these papers
Search
PLOS ONE + 2025 + epidemiology + meta-analyses
- One journal: all papers published under the same open-science and data availability policy, so our results are comparable (see PLOS Open Science).
- One discipline: all papers were classified as “Epidemiology” by OpenAlex).
- Meta-analyses: well-defined method (effect sizes → pooled estimate + heterogeneity), usually shown in forest plots, easy to do in R
- Data usually in the paper: even without shared raw data, the inputs are often in tables or figures
- I have not pre-screened for code availability.
- Recent: 2025, for feasibility.
Examples in PLOS Medicine
Forest plots in 10.1371/journal.pmed.1004558 and 10.1371/journal.pmed.1003017
How the pool was built
| Step | n |
|---|---|
| Retrieved from OpenAlex (article) | 672 |
| Retrieved from OpenAlex (review) | 44 |
| Total retrieved | 716 |
| Meta-analysis in title | 33 |
| Excluded as protocol | 7 |
| Final candidate pool | 26 |
Retrieved via the OpenAlex R package (
openalexR): PLOS ONE, 2025, primary subfield Epidemiology (and open access)Title screening with two regular expressions: keep “meta-analysis” and drop protocols.
The whole selection is itself reproducible: code on the course website [see build_pool.R].
Tools for collaborative reproductions: GitLab & Quarto
GitLab (UZH)
- One seminar group on gitlab.uzh.ch, one repository per reproduction group (forked from a template repository)
- Protocol “locking” (i.e. a tagged commit)
- Peer review between groups via GitLab issues
- Check after this session: can you log in to gitlab.uzh.ch? (using your edu-ID)
- Only after logging in the first time can I invite you to our group
Quarto
- Shared template for the reproduction protocol and the reproduction report
- Formatting and repo setup will happen together in class (Session 3)
- Not yet familiar with Quarto? A Google Doc version of the template will be available for drafting
Homework: until 7 October (2 weeks)
- [Today: Form reproduction pairs]
- Browse the candidate pool and pick 2 – 3 candidate papers you find interesting
- Read the abstract and methods in detail
- Do not check or rerun the code yet
- Prepare one slide on your candidates and why you chose them [sent to me until October 6]
In Session 2 each pair discusses their candidates and why they were chosen, and we make the final selection together.
Things to check when you shortlist:
- Interest: will you want to spend 8 weeks with these papers?
- Target is more or less clear: which pooled estimates, heterogeneity statistics and forest plots will you reproduce?
- Scope: a standard pairwise meta-analysis of manageable size. Flag network or individual-participant-data meta-analyses before committing – they pose very different problems
- No conflicts: not a paper by you, your group or your supervisor
Questions?
- Course website: methods-seminar-computational-reproducibility (UZH Pages)
- Source: gitlab.uzh.ch/crsuzh/methods-seminar-computational-reproducibility
- Seibold et al. (2021): 10.1371/journal.pone.0251194
- Replication Research: replicationresearch.org
Methods seminar – Computationally reproducible research – Session 1