Every group in this seminar reproduces (at least) one published paper.
All candidates are PLOS ONE research articles from 2025 that OpenAlex classifies under the subfield Epidemiology. Focusing on what journal might make the later results more comparable as all original findings are published under the same open science policies. PLOS journals are particularly interesting since they have a Data Availability Statement requirements.
We are focusing on meta-analysis only, because meta-analyses have a well-defined methodology; specifically, extracted effect sizes are pooled into a meta-analytical effect estimate and a heterogeneity statistic. The results of meta-analyses are most often presented in forest plots. R offers great packages that are easy to use. And most importantly, even if raw data are not shared, meta-analysis should in principle be computationally reproducible since data is usually, at least, presented in Figures and/or Tables in the main paper. This makes the reproduction target selection unambiguous as we will know which numbers / figures to reproduce and it keeps the computational burden very manageable.
We focus on recent papers, published in 2025 for feasibility reasons.
Following Seibold et al. (2021), we have deliberately not pre-screened for whether code exists. Their central finding was that code availability, not data availability, drove reproduction difficulty: only 1 of 11 papers with open data supplied any analysis code. Filtering the pool on code availability would quietly assume away the very thing we want to measure. Some of the papers below will have code. Most, probably, will not. Establishing which is part of the work.
How the pool was built
Retrieval from OpenAlex
Papers were retrieved through the OpenAlex API using the openalexR package. Four filters were applied for the search: source (PLOS ONE), publication year, primary topic subfield, and open-access status. OpenAlex assigns a type to every record, and systematic reviews with meta-analyses land inconsistently under article or review depending on how the metadata reached OpenAlex.
Show the code
## Get them from OpenAlex:##########################plos_epi_2025<-oa_fetch( entity ="works", type ="article", primary_topic.subfield.id ="subfields/2713", # Epidemiology primary_location.source.id ="S202381698", # PLOS ONE publication_year =2025, is_oa =TRUE, verbose =TRUE)# Some might be classified as reviewsplos_epi_2025_revs<-oa_fetch( entity ="works", type ="review", primary_topic.subfield.id ="subfields/2713", # Epidemiology primary_location.source.id ="S202381698", # PLOS ONE publication_year =2025, is_oa =TRUE, verbose =TRUE)raw<-plos_epi_2025%>%bind_rows(plos_epi_2025_revs)
Title screening
Retrieval gives every 2025 PLOS ONE epidemiology paper, not just the meta-analyses. We applied two regular expressions on the titles to exclude records that are not of interest. The first keeps papers that announce a meta-analysis in the title. The second drops protocols. A protocol for a systematic review describes an analysis that has not been run, so there is nothing to reproduce.
Show the code
ma_rx<-regex("meta-?analy|metaanaly", ignore_case =TRUE)protocol_rx<-regex(paste0("study protocol|trial protocol|\\bprotocol\\b for (a|an|the)|",":\\s*(a\\s+)?(study\\s+)?protocol|statistical analysis plan|","protocol for a (systematic review|scoping review|meta-analys)|","systematic review protocol"), ignore_case =TRUE)screened<-raw%>%mutate( mentions_ma =str_detect(display_name, ma_rx), is_protocol =str_detect(display_name, protocol_rx), in_pool =mentions_ma&!is_protocol)pool<-filter(screened, in_pool)
What each step removed
Step
n
Retrieved from OpenAlex (type = article)
672
Retrieved from OpenAlex (type = review)
44
Total retrieved
716
Meta-analysis in title
33
Excluded as protocol
7
Final candidate pool
26
Of 716 PLOS ONE epidemiology papers published in 2025, 33 announce a meta-analysis in the title and 26 survive the protocol exclusion.
The pool
Each row is one candidate. Sort by any column, search the table, and click a title to open the paper.
Two things are worth knowing before you choose. Citation counts here are close to meaningless — a paper published in November 2025 has had a few months to be cited and one published in January has had a year, so the column mostly measures publication date. Do not read it as a quality signal. And the pool has not been screened for whether data or code are actually retrievable; checking that is the first real task of your reproduction, not a property of the pool.
---title: "The candidate paper pool"subtitle: "How the papers you can choose from were selected"format: html: toc: true toc-depth: 3 code-fold: true code-summary: "Show the code" code-tools: trueexecute: warning: false message: falsepage-layout: full---```{r setup, include=FALSE}# library(openalexR)library(dplyr)library(readr)library(here)library(stringr)library(DT)raw_path <- here("data", "plos_epi_2025_raw.csv")pool_path <- here("data", "reproduction_pool.csv")# dir.create(here("data"), showWarnings = FALSE, recursive = TRUE)```## Why these papersEvery group in this seminar reproduces (at least) one published paper. All candidates are PLOS ONE research articles from 2025 that OpenAlex classifies under the subfield *Epidemiology*. Focusing on what journal might make the later results more comparable as all original findings are published under the same open science policies. PLOS journals are particularly interesting since they have a Data Availability Statement requirements.We are focusing on meta-analysis only, because meta-analyses have a well-defined methodology; specifically, extracted effect sizes are pooled into a meta-analytical effect estimate and a heterogeneity statistic. The results of meta-analyses are most often presented in forest plots. `R` offers great packages that are easy to use. And most importantly, even if raw data are not shared, meta-analysis should in principle be computationally reproducible since data is usually, at least, presented in Figures and/or Tables in the main paper. This makes the reproduction target selection unambiguous as we will know which numbers / figures to reproduce and it keeps the computational burden very manageable.We focus on recent papers, published in 2025 for feasibility reasons. Following Seibold et al. ([2021](https://doi.org/10.1371/journal.pone.0251194)), we have deliberately *not* pre-screened for whether code exists. Their central finding was that code availability, not data availability, drove reproduction difficulty: only 1 of 11 papers with open data supplied any analysis code. Filtering the pool on code availability would quietly assume away the very thing we want to measure. Some of the papers below will have code. Most, probably, will not. Establishing which is part of the work.## How the pool was built### Retrieval from OpenAlexPapers were retrieved through the [OpenAlex](https://openalex.org) API using the `openalexR` package. Four filters were applied for the search: source (PLOS ONE), publication year, primary topic subfield, and open-access status. OpenAlex assigns a `type` to every record, and systematic reviews with meta-analyses land inconsistently under `article` or `review` depending on how the metadata reached OpenAlex.```{r retrieval-code, eval=FALSE, echo = TRUE}## Get them from OpenAlex:##########################plos_epi_2025 <- oa_fetch( entity = "works", type = "article", primary_topic.subfield.id = "subfields/2713", # Epidemiology primary_location.source.id = "S202381698", # PLOS ONE publication_year = 2025, is_oa = TRUE, verbose = TRUE)# Some might be classified as reviewsplos_epi_2025_revs <- oa_fetch( entity = "works", type = "review", primary_topic.subfield.id = "subfields/2713", # Epidemiology primary_location.source.id = "S202381698", # PLOS ONE publication_year = 2025, is_oa = TRUE, verbose = TRUE)raw <- plos_epi_2025 %>% bind_rows(plos_epi_2025_revs)``````{r load-raw, echo=FALSE, message=FALSE}raw <- read_csv(here("candidates", "data", "raw_pool.csv"))```### Title screeningRetrieval gives every 2025 PLOS ONE epidemiology paper, not just the meta-analyses. We applied two regular expressions on the titles to exclude records that are not of interest. The first keeps papers that announce a meta-analysis in the title. The second drops protocols. A protocol for a systematic review describes an analysis that has not been run, so there is nothing to reproduce.```{r screening, eval = FALSE, echo=TRUE}ma_rx <- regex( "meta-?analy|metaanaly", ignore_case = TRUE)protocol_rx <- regex( paste0( "study protocol|trial protocol|\\bprotocol\\b for (a|an|the)|", ":\\s*(a\\s+)?(study\\s+)?protocol|statistical analysis plan|", "protocol for a (systematic review|scoping review|meta-analys)|", "systematic review protocol" ), ignore_case = TRUE)screened <- raw %>% mutate( mentions_ma = str_detect(display_name, ma_rx), is_protocol = str_detect(display_name, protocol_rx), in_pool = mentions_ma & !is_protocol )pool <- filter(screened, in_pool)``````{r load-pool, echo=FALSE, message=FALSE}ma_rx <- regex( "meta-?analy|metaanaly", ignore_case = TRUE)protocol_rx <- regex( paste0( "study protocol|trial protocol|\\bprotocol\\b for (a|an|the)|", ":\\s*(a\\s+)?(study\\s+)?protocol|statistical analysis plan|", "protocol for a (systematic review|scoping review|meta-analys)|", "systematic review protocol|meta-analysis protocol" ), ignore_case = TRUE)screened <- raw %>% mutate( mentions_ma = str_detect(title, ma_rx), is_protocol = str_detect(title, protocol_rx), in_pool = mentions_ma & !is_protocol )pool <- read_csv(here("candidates", "data", "reproduction_pool.csv"))```### What each step removed```{r flow, echo = FALSE}tibble( Step = c( "Retrieved from OpenAlex (type = article)", "Retrieved from OpenAlex (type = review)", "Total retrieved", "Meta-analysis in title", "Excluded as protocol", "**Final candidate pool**" ), n = c( sum(raw$type == "article", na.rm = TRUE), sum(raw$type == "review", na.rm = TRUE), nrow(raw), sum(screened$mentions_ma), sum(screened$mentions_ma & screened$is_protocol), nrow(pool) )) |> knitr::kable(align = c("l", "r"))```Of `r nrow(raw)` PLOS ONE epidemiology papers published in 2025, `r sum(screened$mentions_ma)` announce a meta-analysis in the title and `r nrow(pool)` survive the protocol exclusion.## The poolEach row is one candidate. Sort by any column, search the table, and click a title to open the paper.Two things are worth knowing before you choose. Citation counts here are close to meaningless — a paper published in November 2025 has had a few months to be cited and one published in January has had a year, so the column mostly measures publication date. Do not read it as a quality signal. And the pool has not been screened for whether data or code are actually retrievable; checking that is thefirst real task of your reproduction, not a property of the pool. [Download the pool as CSV](`r fs::path_rel(pool_path, here())`)```{r table, echo=FALSE}pool |> arrange(publication_date) |> transmute( Title = sprintf( "<a href='https://doi.org/%s' target='_blank' rel='noopener'>%s</a>", doi, display_name ), `First author` = first_author, `n authors` = str_count(authors, ";") + 1L, Published = as.Date(publication_date), Citations = cited_by_count, DOI = doi ) |> datatable( escape = FALSE, rownames = FALSE, filter = "top", extensions = "Responsive", options = list( pageLength = 25, lengthMenu = c(10, 25, 50, 100), order = list(list(3, "asc")) ), caption = htmltools::tags$caption( style = "caption-side: bottom; text-align: left; padding-top: 0.5em;", sprintf("Candidate papers for computational reproduction (n = %d). Retrieved from OpenAlex; last updated %s.", nrow(pool), format(file.mtime(raw_path), "%d %B %Y")) ) )```