New paper by Michael Höfler: Opening Hypothesis-Testing Flexibilities to Others Before Data Collection. doi.org/10.31234/osf.io/87evq_v1 One of the best metascience papers I read this year on how to integrate exploratory research into more rigorous pipelines. Real progress on this topic
Daniel Lakens
Daniël Lakens (born 1980) is a Dutch experimental psychologist known for his contributions to meta-science, research methods, and applied statistics. He is a faculty member in the Human-Technology Interaction group at Eindhoven University of Technology. In 2023 Lakens was awarded the Ammodo Science Award for fundamental research in the Social Sciences, acknowledging his efforts to enhance research reliability and efficiency. Lakens is known for his work on improving research practices through education. He is best known for developing the online course "Improving Your Statistical Inferences", which has enrolled over 61,000 students. More on Wikipedia
-
Daniël Lakens
@lakens
-
Daniël Lakens
@lakens
Looking forward to talking to data managers at the 4tu researchdata community day www.eventbrite.nl/e/4turesearchdata-community-day-2026-ti… about our open source tool Metacheck. Make your job easier by quickly checking the most common problems in repositories, data, and code. metacheckserve.metacheck.src.surf-hosted.nl
-
Daniël Lakens
@lakens
At the Recognition and reward festival 2026. Together with Sajedeh Rasti we will give a keynote on the importance of improving coordination in science, presenting Sajedeh's research and our collaborative grant acquisition project with the Dutch Metascience Coalition.
-
Daniël Lakens
@lakens
Why are severe tests of claims so important in science? I believe that if we do not severely test claims, claims become less reliable, science will be wrong more often, science is less able to achieve it's potential for society, and trust in science is reduced.
-
Daniël Lakens reposted
Chris Kavanagh
@C_Kavanagh
I made a little visualisation to try and help explain p-values for students. Some people might enjoy: psy104-p-values.netlify.app
-
Daniël Lakens reposted
Sam Bowman
@s8mb
One of my biggest life rules is to never, ever get in an argument with Kelsey Piper.
-
Daniël Lakens
@lakens
There is a lot of vagueness in the literature about when a test is exploratory, and when it is confirmatory. I really like our definition where a test is exploratory when you can't control the Type 1 error rate (even if you tried). See doi.org/10.1080/02640414.2025.2486871
-
Daniël Lakens
@lakens
Big news from the PsyArXiv team! They will be moving to a new host, KnowledgeFutures, which offers more room to develop existing functionalities: blog.psyarxiv.com/2026/11/02/psyarxiv-is-moving-to-a-new-…
I beta tested the platform, and I think it an important step in making PsyArXiv more future-proof! -
Daniël Lakens
@lakens
Getting ready to record a podcast on whether frequentist statistics is related to eugenics (in my view: not at all) and if Fisher is a terrible person (as far as I can tell, no, even though I thought so). The main lesson for me is: read history if you want an informed opinion.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog post, about a really cool new feature in Metacheck: Automatic computational reproducibility checks with Metacheck daniellakens.blogspot.com/2026/10/automatic-computational…
This should be of interest to reviewers, data managers, and anyone sharing data and code. Run the checks on our SURF hosted server! -
Daniël Lakens
@lakens
New blog post, about a really cool new feature in Metacheck: Automatic computational reproducibility checks with Metacheck daniellakens.blogspot.com/2026/10/automatic-computational…
This should be of interest to reviewers, data managers, and anyone sharing data and code. Run the checks on our SURF hosted server! -
Daniël Lakens
@lakens
Updated my blog post on which data repositories to use, as ResearchBox now has an API (researchbox.org/api)! This makes it a FAIR data repository, and one of the best options to share data, code, and materials. See the updates at daniellakens.blogspot.com/2026/08/which-data-repository-s…
-
Daniël Lakens reposted
Meta_Student
@MetaStudentSSRC
One month after opening, Meta-Student has 60 registered users and 10 full project registrations across our pilot studies! That is right on target for what we hoped to see in the first month, and we are delighted with it. Come and join us at meta-student.com
-
Daniël Lakens reposted
Grant Abt
@grantabt
Meta-Student now has 60 registered users and 10 full project registrations!
Currently in a pilot year, Meta-Student is a collaborative meta-synthesis initiative coordinated by the Sports Science Replication Centre.
meta-student.com
@MetaStudentSSRC -
Daniël Lakens
@lakens
If a paper is written by multiple authors, and one is more famous than the rest, it is so insulting to act as if all the ideas in the paper come from the famous researcher.
It is an important tell if you see a scientist do this. -
Daniël Lakens reposted
Joe Warne
@JoeWarne1
One month after opening, Meta-Student has 60 registered users and 10 full project registrations across our pilot studies! That is right on target for what we hoped to see in the first month, and we are delighted with it. Come and join us at meta-student.com
-
Daniël Lakens
@lakens
The Journal of Sports Sciences will require open data from 2027: www.tandfonline.com/doi/full/10.1080/02640414.2026.2750698
All journals will move to this requirement (the higher quality and the better the editor, the sooner).
Make sure you are ready for this change by learning best practices. -
Daniël Lakens reposted
Grant Abt
@grantabt
At @JSportsSci we've now moved to the Level 4 (Open) data sharing policy.
Our editorial below announces the policy change, which takes effect from 1 January 2027.
www.tandfonline.com/doi/full/10.1080/02640414.2026.2750698 -
Daniël Lakens reposted
Dr. Anthony Letai
@NCIDirector
If your Author Accepted Manuscript was supported by @NIH funding, it must be posted publicly to PMC without an embargo. There's no cost and it does not require buying expensive open access plans. Take steps to upload your Author Accepted Manuscript here: www.nihms.nih.gov/login/?next=/submission/
Jay Bhattacharya, MD, PhD
@NIHDirector_Jay
.@JAMAInternalMed recently published a thoughtful piece on public access calling for NIH to assert its position regarding compliance with the NIH Public Access Policy.
NIH has the right to make the Author Accepted Manuscript, which has been supported by taxpayers, public in PubMed Central at time of publication; a right which is independent from any publisher agreement.
This is a free path to compliance for authors, helping us ensure rapid public access to NIH-supported research results. -
Daniël Lakens reposted
Sangeetha Shyam
@sangishyam
Such a pleasure to meet @lakens and S.Rasti @TUeindhoven. A short meet that, reinforcing their university values. Looking forward to understand how we can better do research with social relevance #metascience.
-
Daniël Lakens
@lakens
Today we will for the first time grade master students in our Metascience course based on an oral exam. The completed weekly assignments, and we will quiz them on what they learned. It's our way to AI proof the course. Curious about how it will work out!
-
Daniël Lakens reposted
Rolf Zwaan 🦢
@RolfZwaan
A case of misinformation on academic Bluesky rolfzwaan.substack.com/p/a-case-of-misinformation-on-acad…
-
Daniël Lakens
@lakens
Rolf Zwaan, discussing some low quality criticism on the replication studies he is planning by scientists on bluesky:
"Those who wrote the Bluesky posts produced misinformation. Those who liked those posts fell for the misinformation."
rolfzwaan.substack.com/p/a-case-of-misinformation-on-acad… -
Daniël Lakens
@lakens
At the National Open Science Festival, already very happy to meet so many people working on opening up our science. We will be presenting Metacheck in a hands on workshop later today.
-
Daniël Lakens
@lakens
Episode 85: Duae Culturae nulliusinverba.podbean.com/e/episode-85-duae-culturae/
-
Daniël Lakens
@lakens
New episode of Nullius in Verba! We discuss the 1959 Rede Lecture "The Two Cultures" by C. P. Snow. We reflect on whether there is still a divide between the humanities and the sciences, and how important it is to be educated in both. Enjoy! nulliusinverba.podbean.com/e/episode-85-duae-culturae/
-
Daniël Lakens
@lakens
Researchers often can't increase the size of their sample (e.g., due to research constraints). In my paper on sample size justification I discuss approaches to increase power without increasing the sample size (e.g., within designs, reducing variability) lakens.github.io/statistical_inferences/08-samplesizejust…
-
Daniël Lakens
@lakens
In preparation for a discussion with the lab on AI use tomorrow morning, I thought I would come prepared and write down my thoughts in a blog post: How I use AI in my work
daniellakens.blogspot.com/2026/10/how-i-use-ai-in-my-work… -
Daniël Lakens
@lakens
For a metascience project I am trying to download files from Dryad (a popular data sharing platform in ecology). The platform has such strict API limitations that it is unusable for anything else than a single manual download. Please use Zenodo instead. Dryad is horrible.
-
Daniël Lakens
@lakens
Now Laura Caquelin is talking at the Paul Meehl Graduate School symposium talking about computational reproducibility, with the results of a randomized controlled trial of reproducibility review.
-
Daniël Lakens
@lakens
Now Simon Nolte talking about automated extraction. He is discussing how accuracy, scalability, and transparency should be principles, but difficult to achieve all 3 at the same time.
-
Daniël Lakens
@lakens
Now Joeri Tijdink is talking about wellbeing and research waste in academia. You can jump in the live feed to follow along!
Daniël Lakens
@lakens
We are excited to announce that we are going to broadcast the second day of Meta-Research Symposium 2026 (this Friday on October 9th) for those who were interested but unable to join in person.
JOIN: teams.microsoft.com/l/meetup-join/19%3ameeting_NDg1OTlkOW…
PROGRAM: docs.google.com/document/u/0/d/1kvmTqjEpbBOmCRjl5WF82sTFG… -
Daniël Lakens
@lakens
The Paul Meehl Graduate School Symposium is kicking off with a keynote by the Best Thesis Award winner Charlotte Rulkens on the role of replications in the Humanities, studying the repetition of art attribution.
-
Daniël Lakens
@lakens
The wonderful Gilad Feldman @giladfeldman
is kicking off his workshop in front of a full house at the Paul Meehl Graduate School workshop on how to use AI tools to improve open science practices. -
Daniël Lakens
@lakens
Looking forward to a fun first day of the Paul Meehl graduate school symposium full of workshops and hackathons and social events!
docs.google.com/document/u/0/d/1kvmTqjEpbBOmCRjl5WF82sTFG… -
Daniël Lakens
@lakens
Dear academics, if you want your book to be read, you publish it open access online for free (e.g. a quarto book). If you write a book no one needs to read and that you are writing purely to get tenure, send it to a publishers who sells prestige (without adding any value).
-
Daniël Lakens
@lakens
9 days left to submit an abstract to the 9th edition of the Perspectives on Scientific Error conference that takes place in Copenhagen, Denmark, 27–29 January 2027, errorsin.science. It's my favorite conference combining philosophy of science, metascience, and statistics.
-
Daniël Lakens
@lakens
If you want to listen to a podcast about a team taking a thoughtful approach to replicating a classic study, thinking carefully about translation, replicating multiple versions of the same hypothesis, etc, listen to Rolf Zwaan's "Research as it Happens" shows.acast.com/research-as-it-happens
-
Daniël Lakens reposted
Sangeetha Shyam
@sangishyam
@lakens & his teams work provide a lot of thinking points for Activator
www.nature.com/articles/d41586-026-01815-y -
Daniël Lakens
@lakens
We are excited to announce that we are going to broadcast the second day of Meta-Research Symposium 2026 (this Friday on October 9th) for those who were interested but unable to join in person.
JOIN: teams.microsoft.com/l/meetup-join/19%3ameeting_NDg1OTlkOW…
PROGRAM: docs.google.com/document/u/0/d/1kvmTqjEpbBOmCRjl5WF82sTFG… -
Daniël Lakens
@lakens
Lovely day at the start-up meeting for OpenScienceNL funded projects at Leiden. Great day for networking, and being inspired by what Dutch (meta)scientists are up to.
-
Daniël Lakens
@lakens
You can register for the hybrid event "The work of ethics committees in the social and behavioral sciences in a changing research landscape" at the Center for Advanced Internet Studies in Bochum on 25-9-2026. I will discuss methodological review boards. eveeno.com/320501318
-
Daniël Lakens
@lakens
Episode 84: Codices de Moribus Scientificis - II nulliusinverba.podbean.com/e/scientific-code-of-conduct-ii/
-
Daniël Lakens
@lakens
On the 1st of October I will give a talk on sample size justification for the PCI webinar series. I'll discuss the importance of setting your smallest effect size of interest (SESOI) when planning a study, and how to use sequential designs. peercommunityin.org/pci-webinar-series/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Two weeks left to sign up for the Meta-Research Symposium 2026 organized by the Paul Meehl Graduate School paulmeehlschool.github.io/2026-08-04-sumposium2026/
If you've attended previous years, you know this is a lovely day full of cutting edge meta-science, and a great opportunity to network. -
Daniël Lakens
@lakens
Two weeks left to sign up for the Meta-Research Symposium 2026 organized by the Paul Meehl Graduate School paulmeehlschool.github.io/2026-08-04-sumposium2026/
If you've attended previous years, you know this is a lovely day full of cutting edge meta-science, and a great opportunity to network. -
Daniël Lakens
@lakens
Journals should stop offering to store supplemental information. Everything that is shared with a paper should have it's own DOI, and live in a proper repository. As authors, link to files (e.g., Zenodo), but never add them as supplemental files.
-
Daniël Lakens
@lakens
I think that one reason why psychologists are interested in metascience is that you are trying to improve a system full of scientists who don't want to be told what to do.
-
Daniël Lakens
@lakens
I am afraid @stuartbuck1 is getting multiple comparisons wrong. If you want outcome X AND Y AND Z to be significant (as in the example) you do not correct so the problem never happens. You correct if you want X OR Y OR Z to show an effect.
Stuart Buck
@stuartbuck1
I'm not a fan of multiple comparisons corrections (e.g., Bonferroni, Benjamini-Hochberg, etc.). Just preregister everything and report all results (or else do some kind of multiverse analysis or specification curve), and let the reader decide.
One of the biggest issues for me is the following paradox.
Say I run an RCT testing a drug as to LDL levels, and I'm trying to decide what outcomes to measure: reduction in LDL over 1 year, cardiac outcomes over 3 years, or mortality over 5 years. Suppose I measure all three, and the trial ultimately shows a reduction in all of those outcomes (p=.04 for each).
With Bonferroni etc., the trial would report a "null" effect across the board, even though these outcomes are all consistent and mutually reinforcing, and the evidence for the treatment is way stronger than if I had only collected evidence on cholesterol levels at 1 year.
It makes no sense to me that any single outcome by itself would have been statistically significant, but just because, in the past, I decided to collect more and better evidence, now the treatment has "no effect"? -
Daniël Lakens reposted
I4R
@I4Replication
1/ What is the prevalence of personally identifiable information (PII) in author-collected data publicly shared in the social sciences?
We audited 327 replication packages from 11 leading journals in economics, political science, and psychology
Answer: surprisingly high 🧵 -
Daniël Lakens reposted
Quentin André
@andre_quentin
This is a great initiative @BillAckman! However, some may question your commitment to academic truth given that you are also bankrolling the lawsuit of Francesca Gino, who *check notes* sued three brave scholars for telling the truth?
Steve McGuire
@sfmcguire79
Wow…Roland Fryer and @BillAckman have created a $1 million prize for academic truth-telling, the Carob Trust Prize for Academic Courage.
“The prize awards $1 million each to as many as five social scientists a year who have demonstrated intellectual independence, published findings that were attacked rather than answered, and been validated by the evidence—despite the professional cost. The selection criteria are designed to distinguish courage from contrarianism: Nominees must show a sustained commitment to following logic and evidence regardless of pressure, a willingness to ask questions others avoid, and work that has shifted academic debate, public discourse or policy—often despite being misread, mischaracterized or vilified at the time of publication.” -
Daniël Lakens reposted
Antal Haans
@AntalHaans
All jokes (where they jokes?) aside. Does anyone know a relatively easy way to get a nice overview of all sum of squares, effects as well as errors? Even for mixed ANOVAs.
-
Daniël Lakens
@lakens
Episode 83: Codices de Moribus Scientificis - I nulliusinverba.podbean.com/e/scientific-code-of-conduct-i/
-
Daniël Lakens
@lakens
It was so much fun to work with Ethan Meimoun on this letter with advice to PhD students for when their studies don't work out, and why this is normal, even if it might not feel like it.
It was great to have Ethan around as a lab visitor!
www.researchgate.net/publication/413851913_Why_It_Is_Prob… -
Daniël Lakens reposted
Jakub Tomek
@JakubTomek13
Very glad to see the interest in this work on equivalence testing, and to keep receiving e-mails from readers who find it important and interesting!
A lot of credit goes to @lakens who brought it to many people's attention.
David Eisner
@DavidAEisner
Beyond “non-significant” results: Why and how to test for practical equivalence | PNAS www.pnas.org/doi/10.1073/pnas.2611548123
-
Daniël Lakens
@lakens
From all data repositories available, I wonder why anyone would choose to use Dryad. It has API restrictions on downloads that make it unusable if you want to download data from multiple papers. I can't recommend it, given how it currently works.
-
Daniël Lakens
@lakens
I care about reproducibility, but I find it deeply problematic that the British ecological society is trying to extract free labour from academics (oh no, you get free membership for a year!) to work for them for 3 years on reproducibility checks.
apply.workable.com/britishecologicalsociety/j/AF12AB53FF
Daniël Lakens
@lakens
They publish 7 journals with Wiley - that provides more than enough income to compensate people fairly. Wiley can use some of the millions in profit it makes to *actually pay for added value*. Also, we can automate some of these checks now www.scienceverse.org/metacheck_book/chapters/mod-reproduc…
-
Daniël Lakens reposted
Daniël Lakens
@lakens
If your journal performs manual reproducibility checks, and you are interested in comparing the performance of manual checks with the automated Reproducibility_Check module that I am developing for Metacheck (www.scienceverse.org/metacheck_book/chapters/mod-reproduc…, does not use any AI) do reach out!
-
Daniël Lakens
@lakens
If your journal performs manual reproducibility checks, and you are interested in comparing the performance of manual checks with the automated Reproducibility_Check module that I am developing for Metacheck (www.scienceverse.org/metacheck_book/chapters/mod-reproduc…, does not use any AI) do reach out!
-
Daniël Lakens reposted
Pitooooorra
@jesdaza
La version del manual "mejorarando tus inferencias estadísticas" de Daniel Läkens en castellano. Es un complemento ideal para cualquier curso básico de estadística en ciencias sociales.
Daniël Lakens
@lakens
A Spanish translation of my textbook Improving Your Statistical Inferences, made by Juan G. Fernández Fernández, is now available here: juangff85.github.io/inferencias-estadisticas-ESP/
I am very grateful Juan worked so hard to made the contents available to Spanish speakers! -
Daniël Lakens
@lakens
A Spanish translation of my textbook Improving Your Statistical Inferences, made by Juan G. Fernández Fernández, is now available here: juangff85.github.io/inferencias-estadisticas-ESP/
I am very grateful Juan worked so hard to made the contents available to Spanish speakers! -
Daniël Lakens reposted
Daniël Lakens
@lakens
New Blog post: Which Data Repository Should you Use?
In light of OSF closing down, I compare Zenodo, Dataverse, ResearchBox, PsychArchive, and local repositories on six important dimensions. If you want to know which to pick: It depends!
daniellakens.blogspot.com/2026/08/which-data-repository-s… -
Daniël Lakens reposted
Chetan Chawla
@ChetanChawla
Data repos are an incredibly important part of the scientific commons, and critical for reproducible/ credible research. Great overview of the post-OSF landscape 👇🏽
Daniël Lakens
@lakens
New Blog post: Which Data Repository Should you Use?
In light of OSF closing down, I compare Zenodo, Dataverse, ResearchBox, PsychArchive, and local repositories on six important dimensions. If you want to know which to pick: It depends!
daniellakens.blogspot.com/2026/08/which-data-repository-s… -
Daniël Lakens
@lakens
New Blog post: Which Data Repository Should you Use?
In light of OSF closing down, I compare Zenodo, Dataverse, ResearchBox, PsychArchive, and local repositories on six important dimensions. If you want to know which to pick: It depends!
daniellakens.blogspot.com/2026/08/which-data-repository-s… -
Daniël Lakens
@lakens
Two stipendiary Research Fellowship positions on the theme of Replication at King's College, Cambridge, UK Shoykd be interesting for metascientists who completed a PhD less than 2 years ago: www.kings.cam.ac.uk/research/current-research-opportunities
-
Daniël Lakens
@lakens
New episode of Nullius in Verba! nulliusinverba.podbean.com/e/episode-82-scientia-nova-aet… We reflect on how governments should support science, and discuss what we like a lot and what we like less about the Trump administration's 'Science: A New Golden Age' policy document.
-
Daniël Lakens
@lakens
Episode 82: Scientia: Nova Aetas Aurea nulliusinverba.podbean.com/e/episode-82-scientia-nova-aet…
-
Daniël Lakens
@lakens
The Metacheck Shiny app to download all private OSF Repositories is done. The manual is updated, everything is in the dev branch on GitHub. We explain how you can run repo_check, data_check, code_check, and codebook_check on your repository with 1 line of code. Launch next week.
-
Daniël Lakens
@lakens
My favorite Metascience conference will land in Copenhagen next year. The 9th (!) edition of the Perspectives on Science Error conference will be from January 27 to 29. Block your calendar, and see you there! errorsin.science
-
Daniël Lakens reposted
Daniël Lakens
@lakens
The Metacheck team is busy building an improved version of the osf_download function (www.scienceverse.org/metacheck_book/chapters/local-files.… ). You will soon be able to use metacheck to download all files, wikis, and meta-data. You will also be able to upload everything to Zenodo.
-
Daniël Lakens reposted
Koenfucius 🔍
@koenfucius
How good are you at inferring the correlation of a set of data points?
Check out Daniel @lakens’ game (part of his “Improving your statistical inferences” book) and test (and hone!) your skill:
lakens.github.io/statistical_inferences/06-effectsize.html -
Daniël Lakens
@lakens
The Metacheck team is busy building an improved version of the osf_download function (www.scienceverse.org/metacheck_book/chapters/local-files.… ). You will soon be able to use metacheck to download all files, wikis, and meta-data. You will also be able to upload everything to Zenodo.
-
Daniël Lakens
@lakens
Postdoctoral Associate in Meta-Research at Carnegie Mellon University
cmu.wd5.myworkdayjobs.com/en-US/CMU/job/Pittsburgh-PA/Pos… -
Daniël Lakens reposted
Gregory R. Hancock
@GregoryRHancock
Here’s a fun paper led by my amazing colleague Dayeon Lee, in press in the Journal of Educational and Behavioral Statistics. Could be that last beach read of the summer…
arxiv.org/abs/2608.02535 -
Daniël Lakens
@lakens
Just edited our next podcast episode which will be released August 14th. The title of episode 82 is: Scientia: Nova Aetas Aurea.
Any ideas what we will discuss?
Daniël Lakens
@lakens
In preparation, I read Vannevar Bush's autobiography 'Pieces of Action' which gives a good insight into the life of a person who substantially shaped science policy:
Bush, V. (1970). Pieces of the action. New York, Morrow. archive.org/details/piecesofaction00bush -
Daniël Lakens reposted
Daniël Lakens
@lakens
In the last 2 years I (and our lovely collaborators) have been rather obsessively working on Metacheck - a tool to automatically check best practices in papers. In this RIOTS summerschool talk, I walk you through what the tool can do, and future plans www.youtube.com/watch?v=OWd2zEwhI-4
-
Daniël Lakens
@lakens
In the last 2 years I (and our lovely collaborators) have been rather obsessively working on Metacheck - a tool to automatically check best practices in papers. In this RIOTS summerschool talk, I walk you through what the tool can do, and future plans www.youtube.com/watch?v=OWd2zEwhI-4
Daniël Lakens
@lakens
I am currently working on a major addition to help researchers share data and code in a FAIR and reproducible manner. It's an incredibly exciting project that I strongly believe has immense potential.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Analysis Multiversi is out now: nulliusinverba.podbean.com/e/episode-81-analysis-multiversi/
What is the epistemic goal of multiverse analysis? Should we incorporate the uncertainty in many analyst approaches into our statistical inferences? (😬)? Is the variability mainly due to low effort of non-experts? -
Daniël Lakens
@lakens
Analysis Multiversi is out now: nulliusinverba.podbean.com/e/episode-81-analysis-multiversi/
What is the epistemic goal of multiverse analysis? Should we incorporate the uncertainty in many analyst approaches into our statistical inferences? (😬)? Is the variability mainly due to low effort of non-experts? -
Daniël Lakens
@lakens
Episode 81: Analysis Multiversi nulliusinverba.podbean.com/e/episode-81-analysis-multiversi/
-
Daniël Lakens
@lakens
I still see people who try to defend preregistration for secondary data analysis when the data is publicly accessible. It is a nonsensical thing to do. Fire up a lab notebook and log your decisions, by all means, be transparent. But you can't preregister. www.tandfonline.com/doi/full/10.1080/2833373X.2024.2376046
-
Daniël Lakens
@lakens
I have a renewed appreciation for the statistical software packages Jamovi and JASP. Use them! In terms of reproducibility, they can't be beat. Archiving the data, code, and output in a single file is a genius move. Achieving the same reproducibility in R is very hard work.
-
Daniël Lakens
@lakens
Nice podcast episode with Nature editor Magdalena Skipper on why she believes Metascience is an important young discipline that can help us to understand how to improve our science, and the lessons learned from the Replication Crisis.
thehpspodcast.buzzsprout.com/2180146/episodes/19536659-s6… -
Daniël Lakens
@lakens
Yesterday we recorded a Nullius in Verba podcast episode in which we critically reflect on multiverse analysis, it's misuse, and it's very limited uses in science. I bring up some points of our preprint 'There is only one correct analysis' (osf.io/preprints/psyarxiv/hvjxf_v1). Out Friday!
-
Daniël Lakens reposted
Anne Scheel
@annemscheel
New paper w/ @LeonidTiokhin & @lakens!
Registered Reports (RRs) are great for science b/c publication is guaranteed before results are known, reducing publication bias & QRPs. This supposedly also makes them great for *scientists*. But is that really true?
www.biorxiv.org/content/10.64898/2026.07.17.739248v1 -
Daniël Lakens
@lakens
New episode of @nulliusverbapod out now!
Daniël Lakens
@lakens
Episode 80: Intelligentia Artificialis nulliusinverba.podbean.com/e/episode-80-intelligentia-art…
-
Daniël Lakens reposted
Rafael Izbicki
@rizbicki
Delighted to share our new paper, “On focusing statistical power for searches and measurements in particle physics,” in EPJ C!
FTS focuses power on relevant parameter regions while preserving frequentist coverage—an alternative to generalized likelihood-ratio tests.
+ -
Daniël Lakens
@lakens
Episode 80: Intelligentia Artificialis nulliusinverba.podbean.com/e/episode-80-intelligentia-art…
-
Daniël Lakens
@lakens
The taverne amendment in Dutch copyright law allows all Dutch scientists to share their work - irrespective of the copyright publishers try to impose - after 6 months.
Time for another amendment:
After 6 months publishers are required to make all publications open access. -
Daniël Lakens
@lakens
After a little break due to busy schedules Nullius in Verba will be back with a new podcast episode later today!
We will discuss when and how artificial intelligence and automation can improve the way we do science. Keep an eye on your podcast players! -
Daniël Lakens reposted
Daniël Lakens
@lakens
We are hiring an assistant professor in Metascience! Join our lovely department of Human-Technology Interaction, with a very active Metascience group, at Eindhoven University of Technology. Apply here before August 14, or share the vacancy in your network! www.tue.nl/en/working-at-tue/vacancy-overview/assistant-p…
-
Daniël Lakens
@lakens
I am working on a new function in Metacheck's upcoming data-validate module. I am interested in detecting and labeling scales and measures. It is working pretty well, but in case I am reinventing the wheel - do you know of tools that can find scales in datasets in repositories?
-
Daniël Lakens
@lakens
We are hiring an assistant professor in Metascience! Join our lovely department of Human-Technology Interaction, with a very active Metascience group, at Eindhoven University of Technology. Apply here before August 14, or share the vacancy in your network! www.tue.nl/en/working-at-tue/vacancy-overview/assistant-p…
-
Daniël Lakens reposted
Nicholas Coles, PhD
@coles_nicholas_
Interested in scientific collaboration, collective intelligence/ignorance, and an epic intellectual journey? 🤔
Consider a PhD with the Social Science Collaboratory! 🌐
We provide five years of funding, excellent mentorship, and an extremely deep supply of toy dinosaurs. 🦕 -
Daniël Lakens reposted
Daniël Lakens
@lakens
In our recent paper criticizing the idea of Type S and M errors, we discuss the idea that 'the null is never true' and explain why that idea is silly (why not doubt that an effect of 1 × 10−32 can be true?) and scientifically unfalsifiable. journals.sagepub.com/doi/full/10.1177/25152459261432530
-
Daniël Lakens
@lakens
In our recent paper criticizing the idea of Type S and M errors, we discuss the idea that 'the null is never true' and explain why that idea is silly (why not doubt that an effect of 1 × 10−32 can be true?) and scientifically unfalsifiable. journals.sagepub.com/doi/full/10.1177/25152459261432530
-
Daniël Lakens reposted
Data Colada
@DataColada
The Cover-Up in Gino v Harvard.
datacolada.org/136 -
Daniël Lakens
@lakens
I grind my coffee beans by hand, listen to music on vinyl, do stats on my fingers, read the literature on paper, I have my meetings in person in a cafe smoking a pipe, I teach with the blackboard, and write down ideas in Waterman Serinity Blue.
I produce 100% artisanal science.
Daniël Lakens
@lakens
I even make artisanal typos in words like serenity and wear them like a badge of honour.
-
Daniël Lakens
@lakens
Reproducibility is not a goal but a means.
I always thought this was too trivial to have to point it out, but maybe I was wrong. -
Daniël Lakens
@lakens
I'm working on a new Metacheck module: Datacheck. Repocheck already lists all files on Github, Zenodo, OSF, Researchbox. Datacheck will download them and.... which automated checks would you like for datafiles? Can be general, or file specific (e.g., excel files). Let me know 👇
-
Daniël Lakens
@lakens
Our metacheck package now has an online manual! In 29 short chapters we explain how to set up metacheck (running all modules 100% locally if you want), and explain the 18 available modules. www.scienceverse.org/metacheck_book/
We are still in beta, and would love to get feedback! -
Daniël Lakens reposted
Quentin André
@andre_quentin
HBS' latest filing in the Gino lawsuit contains a few new details:
1. A committee of 26 professors, admins, and fellows reviewed the allegations, and *unanimously concluded* that there was *clear and convincing evidence* of research misconduct. -
Daniël Lakens
@lakens
In Norway, staying in this lovely farm (the elderly home of my friend and collaborator Peder Isager) for a week with the lab and friends, to talk about research waste, Meehl's corroboration index, agent based models, and Metacheck. Good food, great weather, and amazing people ❤️
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New preprint: There is only one correct analysis. osf.io/preprints/psyarxiv/hvjxf_v1
We take a critical look at recent proposals for multiverse or many analyst procedures, and strongly argue that epistemic uncertainty should be reduced, not “embraced”. > -
Daniël Lakens
@lakens
New preprint: There is only one correct analysis. osf.io/preprints/psyarxiv/hvjxf_v1
We take a critical look at recent proposals for multiverse or many analyst procedures, and strongly argue that epistemic uncertainty should be reduced, not “embraced”. >Post not available here, view on X
Daniël Lakens
@lakens
Uncertainty about auxiliary hypotheses should not be integrated in our confidence intervals. This mixed up epistemic uncertainty with aleatoric uncertainty. We link to Wimsatt's work on robustness, and argue multiverse analyses provide 'pseudo-robustness'.
>
Daniël Lakens
@lakens
Integrative experiments - where effects are tested across auxiliary assumptions - are a more useful tool than multiverse analysis and many analyst studies. If you like to ponder the philosophical basis of statistical procedures, you might enjoy this paper! osf.io/preprints/psyarxiv/hvjxf_v1
-
Daniël Lakens
@lakens
This is an excellent idea. If you have never watched these lectures, they deserve a place high up in your priority list! And so fun to watch it in a group!
Grace Hart (she/her)
@francesghart
Raisin’ ‘em right! (Paul Meehl lecture series with @CCSLabYale summer interns!)
meehl.umn.edu/video -
Daniël Lakens reposted
Daniël Lakens
@lakens
Our new paper, Rethinking Type S and M Errors, is out. We argue that these statistics do not facilitate the proper design of studies or the meaningful interpretation of results.
doi.org/10.1177/25152459261432530 -
Daniël Lakens reposted
Daniël Lakens
@lakens
This paper started with lovely discussions with Gianmarco Altoe over a spritz in Padova, trying to resolve our disagreements about when and where Type S and M errors are useful.
It is interesting how all alternatives to a p value seem to have greater downsides. -
Daniël Lakens
@lakens
Our new paper, Rethinking Type S and M Errors, is out. We argue that these statistics do not facilitate the proper design of studies or the meaningful interpretation of results.
doi.org/10.1177/25152459261432530
Daniël Lakens
@lakens
This paper started with lovely discussions with Gianmarco Altoe over a spritz in Padova, trying to resolve our disagreements about when and where Type S and M errors are useful.
It is interesting how all alternatives to a p value seem to have greater downsides. -
Daniël Lakens
@lakens
If you like Metascience, come by June 25th in Oslo! We are in Norway for a lab retreat, and will present work on effect size interpretation and Metacheck. See you there!
ONH Metascience Symposium 2026 | Oslo Nye Høyskole oslonyehoyskole.no/Arrangement/ONH-Metascience-Symposium-… -
Daniël Lakens
@lakens
Campbell, 1988, explaining why he disliked working in Chicago and almost considered leaving academia (which would have been a huge loss) due to the highly competitive environment and the idea you should write only brilliant papers.
-
Daniël Lakens
@lakens
I have updated the layout (but the old one is a slider away), and added projects. If you are building open source tools that assist scientists in adopting best practices, let me know and I will add you.
Daniël Lakens
@lakens
I built a website that tracks open source tools that provide automated checks for scientific manuscripts: lakens.github.io/automated_review_daily_build/
It provides a summary through RSS feed that you can subscribe to. It tracks 13 repositories - if you want your repo to be included, let me know! -
Daniël Lakens
@lakens
I built a website that tracks open source tools that provide automated checks for scientific manuscripts: lakens.github.io/automated_review_daily_build/
It provides a summary through RSS feed that you can subscribe to. It tracks 13 repositories - if you want your repo to be included, let me know!
Daniël Lakens
@lakens
Our Metacheck team would love to keep track of what our peers are working on, and to facilitate collaboration. Making these tools is hard work, so we should help each other out where we can!
-
Daniël Lakens reposted
♕Deborah Mayo♕
@learnfromerror
In my new blogpost I pull back the cover regarding claims that low powered tests "exaggerate" population effects: "low power and an all too standarderror: continuation of 'don't turn power on its head'. errorstatistics.com/2026/06/07/low-power-and-an-all-too-s…
-
Daniël Lakens reposted
♕Deborah Mayo♕
@learnfromerror
New CFP at Synthese! I invite you to contribute: Topical Collection: severity and learning from error
-
Daniël Lakens
@lakens
After hearing the enthusiastic description of the importance of the work of Popper at the start of this episode, I am almost tempted to buy the 200k trading card....
quantitudethepodcast
@quantitudepod
S7E23 Quantitude Philosopher of Science Trading Cards
In today’s episode we talk about creating collectible trading cards for some of our favorite philosophers of science and the impact each has had on how we think about the practice of research. -
Daniël Lakens
@lakens
My PhD Sajedeh Rasti has published a World View piece in Nature today: "Don’t compete, collaborate: why collective funding applications are the future" www.nature.com/articles/d41586-026-01815-y She discusses how she brought together a team of meta-scientists to collaboratively apply for funding.
Daniël Lakens
@lakens
If you prefer to read open access texts, then read our more extensive overview (submitted to Meta-Psychology) here: osf.io/preprints/psyarxiv/fmnc5_v2/
I'm extremely proud of the excellent work Sajedeh is doing - she is an incredibly talented young scholar. -
Daniël Lakens
@lakens
With Deborah Mayo, Kent Staley, and Wendy Parker we are editing a special issue of Synthese on "Severity and Learning from Error": link.springer.com/collections/ebjdhfadcd Deadline for submissions is the 15th of December, 2026. If this is a topic that you are interested in, consider contributing!
-
Daniël Lakens
@lakens
We launched our beta version 0.0.1 of Metacheck today! After our second year of work on the tool, we now have validated modules, hugely expanded functionality, optional (or local) LLM use, checks inside code files, and much more! Try it out in R now: www.scienceverse.org/metacheck/
-
Daniël Lakens
@lakens
Vlasta Sikimić is opening the conference on AI and Peer Review at Eindhoven University of Technology. It will be a very interesting day, no doubt full of spirited discussions!
-
Daniël Lakens
@lakens
If you are sad we haven't released a podcast for a while, and you'd like to hear me talk about coordination in science (and why it is such an important solution to many of the challenges we have been facing for decades), see my recent talk here: youtu.be/g8LFwyQsML8
-
Daniël Lakens reposted
Koenfucius 🔍
@koenfucius
Is power posing a Thing after all?
The author of the original research offers new evidence.
Daniel @lakens first ignored, but thankfully later took a critical look 👏 —finding little to convince, and much to be sceptical.
Salutary reading for fans:
daniellakens.blogspot.com/2026/05/evaluating-dr-cuddys-cl… -
Daniël Lakens reposted
Florian Kühlwein
@FloKuehlwein
Good news! Registered Reports are fun, they have many benefits (see below) and they give you many chances to review your paper ;) so far, all my studies are designed as RRs! Way to go 💪🏽
Daniël Lakens
@lakens
Excellent news! Nature is expanding Registered Reports to all the fields in which they publish! A great result by all those proposing this strong solution to publication bias and selective reporting (and getting expert feedback before data collection)! www.nature.com/articles/d41586-026-01629-y
-
Daniël Lakens
@lakens
Excellent news! Nature is expanding Registered Reports to all the fields in which they publish! A great result by all those proposing this strong solution to publication bias and selective reporting (and getting expert feedback before data collection)! www.nature.com/articles/d41586-026-01629-y
-
Daniël Lakens reposted
Jesper N. Wulff
@jesper_wulff
"When Cuddy says ‘The evidence is there’, this is not correct for the Andolfi and Antonietti article, which does not seems to exist in the scholarly record."
Lord, give me strength to read the rest and keep an open mind. -
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog post: Evaluating Dr. Cuddy’s Claim that the Debunking of Power Posing is a Myth.
daniellakens.blogspot.com/2026/05/evaluating-dr-cuddys-cl…
On an AI generated description of a non-existent study, incorrectly citing findings from studies, and the importance of scientific criticism. -
Daniël Lakens reposted
Richard Wiseman
@RichardWiseman
We have just published an article on preregistration in parapsychology. Thanks to @lakens for great ref comments peerj.com/articles/21360/ @thePeerJ @UHertsResearch @KPUNews
-
Daniël Lakens reposted
Florian Kühlwein
@FloKuehlwein
Great blog post! Highly recommended
Daniël Lakens
@lakens
New blog post: Evaluating Dr. Cuddy’s Claim that the Debunking of Power Posing is a Myth.
daniellakens.blogspot.com/2026/05/evaluating-dr-cuddys-cl…
On an AI generated description of a non-existent study, incorrectly citing findings from studies, and the importance of scientific criticism. -
Daniël Lakens
@lakens
New blog post: Evaluating Dr. Cuddy’s Claim that the Debunking of Power Posing is a Myth.
daniellakens.blogspot.com/2026/05/evaluating-dr-cuddys-cl…
On an AI generated description of a non-existent study, incorrectly citing findings from studies, and the importance of scientific criticism. -
Daniël Lakens
@lakens
If a scientist uses AI to make strong claims in public about the scientific literature, and some references do not exist, and the generated text does not correcfly reflect the findings, should we inform them privately to fix the text or should we share this information publicly?
-
Daniël Lakens
@lakens
At the Vrije Universiteit Amsterdam to join today's session on the national dialogue on academic freedom.
-
Daniël Lakens
@lakens
The choice is not between p-values or effect sizes. They go hand in hand. The p-value is there to keep you from seeing effects where there are none. If that bar is passed, you can treat effects as non-zero, and start to reason about what they mean in practice or for a theory.
-
Daniël Lakens
@lakens
Reviewing a paper, and it is so nice that the Metacheck R package we made allows me to download all OSF Files by just typing: metacheck::osf_file_download("2w123") (not the actual repo). All files are downloaded, maintaining the folder structure, without any hassle.
-
Daniël Lakens
@lakens
A master student, Mink Veltman, is helping us to develop a miscitation detection module for Metacheck. If you want to help us to reduce miscitations, it would be great if you could take 5 minutes to fill out this survey: forms.cloud.microsoft/e/J7ugQw8mic
Thanks in advance! -
Daniël Lakens
@lakens
New interactive simulation in my online textbook illustrating publication bias in meta-analyses, and some techniques to model bias-adjusted effect sizes. None of these work perfectly. We need an unbiased literature to draw valid inferences. lakens.github.io/statistical_inferences/12-bias.html
-
Daniël Lakens
@lakens
This year I have had a paper which was reviewed by 5 people, and today another paper reviewed by 6 people. And here I was thinking we were in a peer reviewer crisis where nobody wanted to review anymore. I value criticism more than most, but... This seems a bit much?
-
Daniël Lakens
@lakens
Can strongly recommend Richard Hamming's book The Art of Doing Science and Engineering: Learning to Learn. A bit old, but still great insights.
It is an expanded version of his famous talk You and Your Research, which is also worth watching: youtu.be/a1zDuOPkMSw -
Daniël Lakens
@lakens
Klaus Fiedler passed away two days ago. joachimfunke.de/2026/05/13/nachruf-auf-klaus-fiedler-1951…
In 2017 we did a fun interview for In Mind on the replication crisis. www.in-mind.org/blog/post/we-are-ready-to-move-an-intervi…
We didn't agree on everything, but I greatly respect he took on the role of Devil's Advocate as a scientific principle. -
Daniël Lakens
@lakens
Are you interested in Shaping an Open Science Culture? Apply for this 2 year post-doc position at UMC Amsterdam. Joeri Tijdink is the PI, but you will work in a large interdisciplinary team (including me).
werkenbij.amsterdamumc.org/en/vacatures/research/postdoc-… -
Daniël Lakens reposted
Daniël Lakens
@lakens
I also created a version of the 'guess the correlation' game. Students always enjoyed it, but the original website (with fun retro design) started to require a google account login. So I created my own version. Play it here: lakens.github.io/statistical_inferences/06-effectsize.html
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Updated my textbook with a cleaner lay-out, better accessibility, and eight new interactive visualizations made completely in html (such as this illustration of the positive predictive value). lakens.github.io/statistical_inferences/02-errorcontrol.html
-
Daniël Lakens
@lakens
I also created a version of the 'guess the correlation' game. Students always enjoyed it, but the original website (with fun retro design) started to require a google account login. So I created my own version. Play it here: lakens.github.io/statistical_inferences/06-effectsize.html
-
Daniël Lakens
@lakens
Updated my textbook with a cleaner lay-out, better accessibility, and eight new interactive visualizations made completely in html (such as this illustration of the positive predictive value). lakens.github.io/statistical_inferences/02-errorcontrol.html
-
Daniël Lakens reposted
Statistical Horizons
@StatHorizons
Are you a researcher burdened by costly and time-consuming data collection? Join @lakens for "Sample Size Justification" on June 9-12 to learn how to justify your sample size, make the most of your resources, and efficiently test your hypothesis. statisticalhorizons.com/seminars/sample-size-justification/
-
Daniël Lakens
@lakens
Many Labs 5 was such a cool project. Authors who argued that a study from the Reproducibility Project: Psychology was 'too different', leading to a null result, could propose their own replication. Mostly null results again. Blaming auxiliaries is easy. Replication is hard.
-
Daniël Lakens reposted
John V. Kane
@UptonOrwell
🎉🎉🎉NOW AVAILABLE for Pre-Order! 🥳
So much thought, time, energy and love went into writing this. For those kind enough to order it, I genuinely hope you enjoy and find value in it. I tried my best to write for those who want to learn how to do quant social science research. -
Daniël Lakens
@lakens
New blog post: There are 13 ways to analyse a replication study, but only one of them is coherent. Where I explain (based on our own recent preprint) why the SCORE project was right to focus on "p < .05 + similar effect size" when reporting the results. daniellakens.blogspot.com/2026/05/there-are-13-ways-to-an…
-
Daniël Lakens reposted
Ash Paul
@pash22
How To Analyze A Replication Study osf.io/preprints/psyarxiv/7ydgu_v1 via @lakens et al
-
Daniël Lakens
@lakens
New preprint: How to analyze a replication study. osf.io/preprints/psyarxiv/7ydgu_v1/
Following methodological falsificationism, claims in replications should follow the original test (so typically p<0.05). We also discuss some consequences a coherent philosophical approach to statistics. > -
Daniël Lakens
@lakens
New article shows something I expect will be a crucial topic for metascience moving forward: the adoption of best practices has reached a plateau, and P-hacking is still a problem. We have reached the limit of bottom-up adoption and it is not sufficient.
link.springer.com/article/10.1057/s41267-026-00859-6 -
Daniël Lakens reposted
Daniël Lakens
@lakens
New (free) workshop by the Paul Meehl Graduate School on agent-based modeling: paulmeehlschool.github.io/2026-04-24-agent-based-models/ The workshop will be exclusively in-person on May 27, 2026, in Eindhoven. Register before May 18th!
-
Daniël Lakens
@lakens
New (free) workshop by the Paul Meehl Graduate School on agent-based modeling: paulmeehlschool.github.io/2026-04-24-agent-based-models/ The workshop will be exclusively in-person on May 27, 2026, in Eindhoven. Register before May 18th!
-
Daniël Lakens
@lakens
Eye-opening documentary on how consulting companies worked their way into Australian universities, and made very questionable management decisions, leading to serious problems. Seems like a clear warning to not rely on these companies. www.youtube.com/watch?v=quc24Ot9Qwg
-
Daniël Lakens reposted
Jason Kerwin
@jt_kerwin
What is going on at PNAS
This doesn’t make it past the desk at an econ journal
Jay Van Bavel, PhD
@jayvanbavel
Each extra year of schooling increases earnings by ~8%
This is more than the cost of education, which implies that additional educatioin pays off financially: www.pnas.org/doi/10.1073/pnas.2537049123 -
Daniël Lakens
@lakens
Rolf Zwaan is starting a new podcast 'As it Happens', following a large-scale psychology research project, where labs around the world collaborate to study human memory. (I am one of the collaborators). The first episode is out! shows.acast.com/research-as-it-happens
-
Daniël Lakens
@lakens
It is mainly strange though.
Dr. Ashley T. Rubin
@ashleytrubin
It's a weird mixture of strange and funny when you learn someone hates you with a passion, but you've never heard of them.
-
Daniël Lakens
@lakens
This week marks the moment where I attended a meeting, took a seat at a table, and one of the other 2 men (both at least 20 years older than I am) said 'ah, so this is the old men's table'. I am 45. AITJ for being slightly insulted or should I just accept the label applies now?
-
Daniël Lakens reposted
Yusaku Horiuchi
@YusakuHoriuchi
I have reviewed 14 manuscripts so far this year and have 3 to review soon. I receive at least 1, sometimes 2-3, requests a week. I believe the number of submitted manuscripts to many journals has dramatically increased. Should I review 40-60 manuscripts a year?
-
Daniël Lakens
@lakens
When we started to promote replication studies (e.g. Koole & Lakens, 2012) we had a scientific culture where publishing a direct replication with a null result was practically impossible, and most didn't even consider trying. That culture has changed - although still not enough.
-
Daniël Lakens
@lakens
The second part of our podcast episode on disagreement is out. nulliusinverba.podbean.com/e/disagreement-ii/
For a while, we will record and release episodes on an irregular schedule. We are both too busy with other fun things to maintain a 2 week schedule. -
Daniël Lakens
@lakens
It is pretty cool that you can now easily write your own software. I made a pdf book to spoken text pipeline, where I play the audiobook in a custom player on Android, where I can press a button that will store the text of the passage playing as a note in Obsidian. So useful!
-
Daniël Lakens
@lakens
Lovely to see everyone coding away during the Paul Meehl Graduate School workshop on Cognitive Computational models paulmeehlschool.github.io/2026-03-10-cognitive-model/, with visitors from Poland, Czech Republic, and Germany. Thanks to Leendert van Maanen and Dominik Bachmann for teaching it!
-
Daniël Lakens
@lakens
Episode 79: Dissensio - II nulliusinverba.podbean.com/e/disagreement-ii/
-
Daniël Lakens
@lakens
Standing in line for Record Store Day in Rotterdam with some good coffee. The wait time (1.5 hours!) was well spent giving feedback on a paper that I am very excited about.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
We are inviting applications for a two-year postdoctoral position in a collaborative meta-science project on the effectiveness of data and code sharing policies in research-performing organizations. www.tue.nl/en/working-at-tue/vacancy-overview/postdoc-in-…
-
Daniël Lakens
@lakens
The first time a serious journal asks me to review a paper that is completely written by AI.
How do we deal with this? Is it fair to write 'this is either written by AI, or mirrors it's lack of original content'?
How do I deal with the 1% probability a human wrote it anyway? -
Daniël Lakens
@lakens
We are inviting applications for a two-year postdoctoral position in a collaborative meta-science project on the effectiveness of data and code sharing policies in research-performing organizations. www.tue.nl/en/working-at-tue/vacancy-overview/postdoc-in-…
Daniël Lakens
@lakens
This project is part of the collaborative grant application Sajedeh described here: bsky.app/profile/sajedehra.bsky.social/post/3mig37ozkhc2i. You will be working with my team - do reach out to any current or past lab members to learn why this is a great environment to do metascientific research.
-
Daniël Lakens
@lakens
This author did not understand the difference between a hypothesis test and an estimate in 2007. Nor did he knew about correction methods for sequential designs. You should probably not read articles by people like this.
Gordon H. Guyatt
@GuyattGH
This article in @AnnalsofIM highlights ethical problems in stopping trials early for benefit: seriously inflated overestimates of treatment effect violates ethical requirements of scientific validity and social value and leads to misguided patient choices.
pubmed.ncbi.nlm.nih.gov/17577007/ -
Daniël Lakens
@lakens
The problem with scientific organizations is that politically, you want them to be large to have influence, but scientifically, you have nothing in common with many of the members. You have to pick influence or shared scientific values. (I am not a member of any organization).
-
Daniël Lakens
@lakens
"comprehensive study pre-registrations or Registered Reports, which ensure peer reviewing of the study protocol and acceptance in principle before data collection and analysis, are a readily implementable measure that should be considered essential."
www.frontiersin.org/journals/neuroscience/articles/10.338… -
Daniël Lakens
@lakens
Statisticians are trained in how to compute statistics, but not why to compute statistics. At least 10% of their training should be in philosophy of statistics. They should also spend 10 credits on their own empirical research project to understand how data is acually collected.
-
Daniël Lakens
@lakens
Re-reading Hasok Chang's call for pluralism in Science in his 2012 'Is water H20?'. It is so badly argued for, his discussion is so one-sided, and his rejection to engage with criticisms is so annoying, I find it hard to complete. Will discuss it in an upcoming podcast episode.
-
Daniël Lakens
@lakens
Hope to see you at the Conference on Transparency, Technology and AI in Peer Review on June 5, 2026, which is organized in collaboration with the Center for Humans and Technology at Eindhoven University of Technology.
Register at: meta-eindhoven.github.io/events/ -
Daniël Lakens reposted
Daniël Lakens
@lakens
First of a 2 part episode on 'Disagreement' in science of Nullius in Verba is out: www.podbean.com/eas/pb-3vw2d-1a8de28 . Why is disagreement and criticism so important in science? Do we need to interact with critics? Is criticism, as Popper wrote, how 'our mind grows and transcends itself'?
-
Daniël Lakens
@lakens
First of a 2 part episode on 'Disagreement' in science of Nullius in Verba is out: www.podbean.com/eas/pb-3vw2d-1a8de28 . Why is disagreement and criticism so important in science? Do we need to interact with critics? Is criticism, as Popper wrote, how 'our mind grows and transcends itself'?
-
Daniël Lakens
@lakens
Episode 78: Dissensio nulliusinverba.podbean.com/e/disagreement/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
A year ago @sajedeh_rasti and I reached out to 20 Dutch metascientists and asked: Instead of competing for this upcoming grant, how about we collaborate, determine priorities, and combine our expertise? Both grants were funded. We discuss our experience in a new preprint. ->
-
Daniël Lakens
@lakens
A new set of papers, sharing the long-awaited result of several reproducibility and replicability projects, including commentaries, is published today. I look forward to reading the studies, and re-using the data generated! www.nature.com/collections/idajfifcfg
-
Daniël Lakens
@lakens
A year ago @sajedeh_rasti and I reached out to 20 Dutch metascientists and asked: Instead of competing for this upcoming grant, how about we collaborate, determine priorities, and combine our expertise? Both grants were funded. We discuss our experience in a new preprint. ->
Daniël Lakens
@lakens
The preprint is available on Psyarxiv: osf.io/preprints/psyarxiv/fmnc5_v2
We had not heard of people trying collaborative grant writing like this before. If you have experiences with this format you would like to share, let us know! -
Daniël Lakens
@lakens
Interesting interview with Anthony Edwards on his experiences working with Ronald Fisher, diving into the topic of Eugenics, with a much more nuanced take than you'll nowadays find on social media. Worth watching: www.youtube.com/watch?v=kqLB5a8op5c
-
Daniël Lakens
@lakens
FAIR data and code is both the most boring part of Open Science, and the part that will most fundamentally change how science is done.
This gap is regrettable, but understandable. Most scientists can't imagine what their science would be like if research outputs were FAIR. -
Daniël Lakens reposted
The Curious
@Thecurioustr
Psikoloji, bağımsız bir bilim olarak kurulduğu günden bu yana üç büyük kriz yaşamıştır. Krizlerin her birinde tartışılan konular farklı olsa da hepsinin aynı soru etrafinda döndüğü söylenebilir: Psikoloji olgun bilimsel disiplinlerdeki gibi genellenebilir/evrensel bilgi üretebilir mi?
Bu konuşmada söz konusu üç krizi meta-metodolojik bir perspektiften ele alacak ve her krizin psikolojiye nasıl bir metodolojik atılım kazandırdığını tartışacağız.
Katılım için aşağıdaki formu doldurmanız yeterli. Ya da yorum atın, mesaj kutunuza zoom linkini iletelim!
forms.gle/UYuv7ajhHvrsa8666
31 Mart Salı 19.00 -
Daniël Lakens
@lakens
"20% of the studies presented causal claims that were not supported by their designs, opening the risk of their results being misinterpreted and their effect sizes being exaggerated [..] exacerbated by a lack of method transparency and construct validity" journals.sagepub.com/doi/abs/10.1177/01492063261424849
-
Daniël Lakens
@lakens
My Talk 'What's the benefit of preregistration for big team science' is online, and you can watch it here: www.youtube.com/watch?v=31lZG6iN0gQ
-
Daniël Lakens
@lakens
It probably took way too long, but I have becomes such a fan of Obsidian over the last weeks! The open .md format makes it so versatile! Combined with python scripts, you can automate workflows, build dashboards, track to-do's, and keep all info organized. obsidian.md
-
Daniël Lakens reposted
BITSS
@UCBITSS
Thrilled to announce Devon Crawford, Program Director of the Office of Research Quality at NIH NINDS, as the Keynote Speaker for the BITSS AM! Join us Thurs, April 16 to hear from Devon and experts using LLMs to advance research transparency.
Register: www.bitss.org/events/bitss-annual-meeting-2026/ -
Daniël Lakens reposted
Lauri Nummenmaa
@LNummenmaa
Excellent point - we also need to remember that a substantial proportion of research findings are significant yet meaningfully null, and even larger portion are *practically* null (i.e. completely useless in any practical sense)
Daniël Lakens
@lakens
Some people think that 'the null is never true' is a killer argument against p values. It isn't. Just test against a meaningful null - e.g., r > 0.1, instead of r > 0. Moving to range predictions is a big improvement - also if you like p values. doi.org/10.1177/1745691620958012
-
Daniël Lakens reposted
Daniël Lakens
@lakens
It is very funny to see that authors who in 2019 argued that "Preregistration is redundant, at best", are now co-authoring papers where their collaborators want to preregister all studies. I guess they weren't even able to even convince their direct co-authors!
-
Daniël Lakens
@lakens
It is very funny to see that authors who in 2019 argued that "Preregistration is redundant, at best", are now co-authoring papers where their collaborators want to preregister all studies. I guess they weren't even able to even convince their direct co-authors!
-
Daniël Lakens
@lakens
Some people think that 'the null is never true' is a killer argument against p values. It isn't. Just test against a meaningful null - e.g., r > 0.1, instead of r > 0. Moving to range predictions is a big improvement - also if you like p values. doi.org/10.1177/1745691620958012
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New Paul Meehl Graduate School workshop announced: Computational Cognitive Models
paulmeehlschool.github.io/2026-03-10-cognitive-model/
Please note that the workshop will be exclusively in-person on April 20, 2026. -
Daniël Lakens
@lakens
Almost in Paris for a talk on 'Which Research is Worth Doing Well' at the University of Paris Nanterre. It is so convenient to be able to walk to the train station, get into a train, and end up 2 countries away - all while working on the revisions of a paper.
-
Daniël Lakens
@lakens
New Nullius In Verba episode on miscitations:
nulliusinverba.podbean.com/e/ep-77-miscitatio/
How often are citations to the scientific literature outright misleading? Do we really need to spell out that people are supposed to read what they cite? And should we review citations as they do in law? -
Daniël Lakens
@lakens
Episode 77: Miscitatio nulliusinverba.podbean.com/e/ep-77-miscitatio/
-
Daniël Lakens
@lakens
New Paul Meehl Graduate School workshop announced: Computational Cognitive Models
paulmeehlschool.github.io/2026-03-10-cognitive-model/
Please note that the workshop will be exclusively in-person on April 20, 2026. -
Daniël Lakens reposted
Noah van Dongen
@Noah_van_Dongen
Theory matters in psychological science — but where does theory in psychology currently stand?
We’re inviting psychological researchers for our survey to share their views on the state of theory.
Overview: doi.org/10.31234/osf.io/2fjx4_v2
Survey: forms.gle/Ct4qq4a4raun9LSs5 -
Daniël Lakens reposted
Maciej Behnke
@BehnkeMaciej
Emotional stimuli in psychology just got a high-tech upgrade. 🚀In our new paper in AMPPS, we introduce and validate the Library of AI-Generated Affective Images (LAI-GAI). We combined generative AI with a rigorous validation pipeline to modernize affect induction procedures.👇
-
Daniël Lakens
@lakens
I remember the widespread pushback when the Peer Reviewer Openness Initiative started in 2017, where we would only review papers that share data and code (or explain why that is not possible) www.opennessinitiative.org/the-initiative/
Now, funders and journals are requiring it.
Daniël Lakens
@lakens
The lesson for me was: academics don't like being told what to do by peers - not even if they are told to do the right thing.
But they do not resists change on any principle. If the people with money or in control of journals say the same thing, they will do it.
Daniël Lakens
@lakens
A combination of a lack of principles, and not wanting to be told what to do, is an extremely unattractive property of many academics.
As an acedemic, you have responsibilities to society. It would be good if these responsibilities were more salient (and the own ego a bit less). -
Daniël Lakens reposted
♕Deborah Mayo♕
@learnfromerror
New blogpost: Continuing the blizzard of 26 power howlers. Check to see that you can shovel through this easy peasy power quiz. Then see the bottom notes to ponder why many critics of tests get this backwards. errorstatistics.com/2026/03/03/continuing-the-blizzard-of…
-
Daniël Lakens reposted
Hudson Golino
@GolinoHudson
🛠️ transforEmotion just crossed 25K CRAN downloads!! Paper documenting the latest version is now out in Computational Communication Research led by Aleksandar Tomašević, Alex Christensen, and myself.
www.aup-online.com/content/journals/10.5117/CCR2026.2.2.TOMA
The paper walks through a complete multimodal workflow on the MAFW dataset and the package is already being used for work ranging from psychotherapy transcripts to political video analysis to corporate communications.
The new version adds some things I'm particularly excited about:
❤️ We switched to uv for Python dependency management
🔧 Local RAG: Full Retrieval-Augmented Generation running on your laptop with small open-source LLMs (TinyLLAMA, Gemma 3, Qwen3, Ministral). No API keys, no cloud costs. Gemma 3 gives structured table/JSON outputs.
🔧 VAD scoring: Direct valence-arousal-dominance prediction from text, images, or video.
🔧 Vision model registry: Swap between CLIP, BLIP, ALIGN, or your own fine-tuned models with a single argument.
🔧 Built-in benchmarking: evaluate_emotions() and validate_rag_predictions() for standardized evaluation. We demonstrate the workflow on FindingEmo in the paper.
Everything runs locally on CPU and a standard laptop. This matters if you work with sensitive data, if you care about reproducibility, or if you simply don't want to pay OpenAI to classify emotions.
📦 CRAN: cran.r-project.org/web/packages/transforEmotion/index.html
💻 GitHub: github.com/atomashevic/transforEmotion
www.aup-online.com/content/journals/10.5117/CCR2026.2.2.TOMA -
Daniël Lakens
@lakens
It is a joy to see people use and appreciate our paper on exploratory research, how it is a crucial step in the research process, how to be transparant about it, and how to do it well. It has 'sport science' in the title, but is relevant for many fields! www.tandfonline.com/doi/10.1080/02640414.2025.2486871?url…
-
Daniël Lakens reposted
Daniël Lakens
@lakens
We are pleased to announce that the Paul Meehl Graduate School is launching the PMGS Dissertation Award, recognizing outstanding PhD dissertations that advance meta-research.
See the eligibility criteria and apply before June 1st, 2026 at:
paulmeehlschool.github.io/award/ -
Daniël Lakens
@lakens
If you really want people to read what you wrote, you will give it away for free, like a Jehova Witness that is handing out bibles.
-
Daniël Lakens
@lakens
We are pleased to announce that the Paul Meehl Graduate School is launching the PMGS Dissertation Award, recognizing outstanding PhD dissertations that advance meta-research.
See the eligibility criteria and apply before June 1st, 2026 at:
paulmeehlschool.github.io/award/ -
Daniël Lakens
@lakens
Boulder route difficulty ratings deserve more attention from the measurement community. The certainty with which my friends can agree a route is a 6B, but not a 6B+, is a marvel of validity and reliability in measurement of which I do not understand how it has been achieved.
-
Daniël Lakens
@lakens
Magnitude based inference has been severely discredited. Bayesian statistics are more widely misused by non-experts than p-values.
Learn best practices in hypothesis testing, instead of switching to techniques you understand even less: lakens.github.io/statistical_inferences/01-pvalue.html
Brad Schoenfeld, PhD
@BradSchoenfeld
I'd note my lab stopped using null hypothesis testing several years ago, opting instead for Bayesian stats and magnitude-based inferences. IMHO, it should be the default in applied sciences
www.nature.com/articles/d41586-019-00857-9
Daniël Lakens
@lakens
(also, in general, don't take statistics advice from non-experts. I will never give you advice on how to look good naked and build muscles. But I know more about how to learn from data than most people on this platform).
-
Daniël Lakens
@lakens
My thoughts are with the people of Iran, and their continued suffering under their own regime, and the regime of others. I hope they will soon see more peaceful times.
-
Daniël Lakens
@lakens
Episode 76: Incitamenta - II nulliusinverba.podbean.com/e/incentives-ii/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New Paul Meehl Graduate School course: Advanced Basic Statistics Workshop: Deepening Your Understanding of Standard Statistical Analyses
March 27, 2026
Sign up:
paulmeehlschool.github.io/2026-02-23-analysis/ -
Daniël Lakens
@lakens
New Paul Meehl Graduate School course: Advanced Basic Statistics Workshop: Deepening Your Understanding of Standard Statistical Analyses
March 27, 2026
Sign up:
paulmeehlschool.github.io/2026-02-23-analysis/ -
Daniël Lakens reposted
Daniël Lakens
@lakens
If you think concerns about replicability started in the 2010's, and you don't know this was just a repetition of exactly the same concerns from the 1960's, you are failing to learn from history of science. Dig in: rips-irsp.com/articles/10.5334/irsp.1038 and rips-irsp.com/articles/10.5334/irsp.1038
-
Daniël Lakens
@lakens
If you think concerns about replicability started in the 2010's, and you don't know this was just a repetition of exactly the same concerns from the 1960's, you are failing to learn from history of science. Dig in: rips-irsp.com/articles/10.5334/irsp.1038 and rips-irsp.com/articles/10.5334/irsp.1038
-
Daniël Lakens
@lakens
#thenullisnevertrueexceptwhenitistrue
Oliver Scott Curry
@Oliver_S_Curry
A meta-analysis of 121 studies (N = 91,462) finds that the relationship between social networking site use and self-esteem is a minuscule r = −0.079 #smma doi.org/10.1016/j.paid.2019.109639
-
Daniël Lakens
@lakens
198 effect sizes in ego depletion resesrch showed an effect size of d=0.62. Preregistered large replications (including some by original authors) yielded an effect size of 0. No one has been able to offer any other explanation for this huge research waste than massive p-hacking.
-
Daniël Lakens reposted
Anna Brown
@KiaNeu6
I always treat “data was available upon request” papers as “data not available.”
Also, as a reviewer, I have never approved any article with such disclaimer for publication.
John B. Holbein
@JohnHolbein1
“Among articles stating that data was available upon request, only 17% shared data upon request.”
-
Daniël Lakens
@lakens
The lack of uptake of sequential analysis shows how irrational scientists are, and how their methods are driven by norms. Sequential analyses give you more flexibility and are more efficient than a single hypothesis test, and yet, they are still very rare. lakens.github.io/statistical_inferences/10-sequential.html
-
Daniël Lakens
@lakens
A great new preprint on the importance of pilot studies for the validity of studies that are performed. Such an important tooic, that is discussed too little. I especially liked the section on the need for transparent reporting. osf.io/t968e_v1 By @YashvinSeetahul et al.
-
Daniël Lakens reposted
Igor Radun (Traffic psychology is my second name)
@Liikennepsykol1
I don’t get it. Why do some active travel researchers get away with openly stating their agenda, and why does no one seem to care? 🤨😫
The examples below are not cherry-picked, they come from well-known and influential figures in the field.
Does anyone care?
#cycling -
Daniël Lakens
@lakens
Currently a lot of attention is going to get people to preregister, to check preregistrations, and noting deviations. But the most important step will be evaluating the impact of the deviations! There is almost no work on this. Good topic for a young metascientist!
-
Daniël Lakens
@lakens
An interesting Metascientific question is how we deal with deviations from preregistrations. Can we evaluate their impact on the severity of tests? If not, should studies with substantial deviations be reported as non-preregistered?
-
Daniël Lakens
@lakens
If you set out to test a hypothesis, you should preregister it. If you deviate from a preregistration, report a table with all deviations, and evaluate the consequences for the validity and severity of the test. As a reviewer, ask for such a table!
online.ucpress.edu/collabra/article/10/1/117094/200749/Wh… -
Daniël Lakens
@lakens
If you have never read the classic 'cargo cult science' speech by Feynman, you are missing out! You can listen to it, as Smriti reads it in full in the prologue of the 8th Nullius In Verba episode (almost 2 years ago!) nulliusinverba.podbean.com/e/cargo-cult-science/
-
Daniël Lakens
@lakens
Revised version of our preprint on Rethinking Type S and M errors. Reviewers thought our criticism was reasonable and well-supported, but we have removed a figure, less attention to approaches to correct for bias, and strengthened our criticism. osf.io/preprints/psyarxiv/2phzb_v2
-
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
Our new episode on incentives in science is out now! We talk about the different kinds of incentives (prestige, awards, money, tenure, etc.)--when they help and how they might hurt science
nulliusinverba.podbean.com/e/incentives-i/ -
Daniël Lakens
@lakens
Episode 75: Incitamenta - I nulliusinverba.podbean.com/e/incentives-i/
-
Daniël Lakens
@lakens
Approximately 40% of all studies preregistered on the Open Science Framework are never shared publicly. That’s a lot. Researchers give several reason, and null results and bad planning (or lack of priority) are the main reasons.
journals.sagepub.com/doi/10.1177/25152459241296031 -
Daniël Lakens
@lakens
In standard scientific reports in psych 96% of first mentioned main hypotheses are supported. In Registered Reports, this is 46%. The 96% is clearly biased (given true H1 rate and power). Lack of transparency means we do not know the true baserate.
journals.sagepub.com/doi/10.1177/25152459211007467 -
Daniël Lakens
@lakens
Babbage, 1830, discussing the problem that scientists selectively report findings that they want to be true.
Confirmation bias is a strong human tendency. This is why we need to design science in a way that prevents conformation bias from leading us away from the truth.
Daniël Lakens
@lakens
Note that this is a centrury before p-values become popular. It is a century before the first use of 'publish or perish'. Changing statistics won't fix it. Creating journals for null results won't fix it. You need to prevent people from ignoring what proves them wrong.
-
Daniël Lakens reposted
Eva Vivalt
@evavivalt
I've built a new tool!
You can upload your pre-analysis plan or registered report, pre-submission to a registry or journal, and it will screen it for completeness, clarity, and consistency. Plans that clear all the checks can share a public page documenting that it passed them, which could be provided to funders or referees as a third-party signal. 1/🧵 -
Daniël Lakens reposted
Christian Bokhove
@cbokhove
Have to distinguish bias types. Publication bias likely agnostic. A qualitative researcher would say that incorporating their views is by design. Recently this by Daniel Lakens interesting.
Daniël Lakens
@lakens
New blog, inspired by the excellent qualitative paper by Makel and colleagues: On the reliability and reproducibility of qualitative research.
I reflect on realist ontologies in qualitative research, and how I will incorporate these in my own research.
daniellakens.blogspot.com/2026/02/on-reliability-and-repr… -
Daniël Lakens
@lakens
The Metacheck team (www.scienceverse.org/metacheck/) and some interested collaborators are hackatonning away on how to create and validate new metacheck modules to automatically detect information in scientific papers. #pse8
-
Daniël Lakens reposted
Dr Cody Porter
@Cody_N_Porter
Insights from the #pse8 conference
Day 2: I was particularly struck by a presentation from Prof Leif Nelson on the detection of problematic data.
I wonder how many of us have came across problematic papers - or datasets and don’t what to do!?
#researchintegrity #openscience -
Daniël Lakens
@lakens
We continue with the field of medical imaging with deep learning, in a talk by Attila Simkó from Umeå University, talking about computationally reproducing code.
-
Daniël Lakens
@lakens
Leif Nelson pointing out that lawsuits are not great for science, as we need to be able to criticize others without the threat of having to pay huge legal fees, and I vest a ton of time, to defend ourselves.
-
Daniël Lakens
@lakens
When we want to distinguish erroneous results from true effects, we need to perform replication studies where we systematically vary factors that aim to isolate the source of the error in repeated conceptual and direct replications. We need to "probe" for sources of error!
-
Daniël Lakens
@lakens
First talk of the Perspectives on Scientific Error conference under way! KaitlynHarper from LMU, Munich, talking about how we should balance novelty and replication.
-
Daniël Lakens
@lakens
Everything is ready for the Perspectives on Scientific Error conference that starts tomorrow in Leiden! I look forward to hanging out with the mix of metascientists, philosophers of science, and statisticians! So many old friends will be there (and hopefully some new ones)! #pse8
-
Daniël Lakens
@lakens
My colleague Krist Vaessen wrote a new book: “Neomania: How our obsession with innovation is failing science, and how to restore trust”. It's a great analysis how the drive for novelty hinders reliable scientific progress. Open Access, so read it here: books.openbookpublishers.com/10.11647/obp.0507/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog, inspired by the excellent qualitative paper by Makel and colleagues: On the reliability and reproducibility of qualitative research.
I reflect on realist ontologies in qualitative research, and how I will incorporate these in my own research.
daniellakens.blogspot.com/2026/02/on-reliability-and-repr… -
Daniël Lakens
@lakens
New blog, inspired by the excellent qualitative paper by Makel and colleagues: On the reliability and reproducibility of qualitative research.
I reflect on realist ontologies in qualitative research, and how I will incorporate these in my own research.
daniellakens.blogspot.com/2026/02/on-reliability-and-repr… -
Daniël Lakens
@lakens
Indeed, it is perhaps surprising this has not been pointed out, but as far as I know, qualitative research is among the most replicable research areas in science. Or at least, all the replication studies I have seen are basically successful replications.
Daniël Lakens
@lakens
Note that this in itself is a well-replicated finding. At least a dozen studies show that when researchers replicate qualitative research, or re-analyze it, they come to basically identical themes. There is nothing special about qualitative research with respect to replicability.
Daniël Lakens
@lakens
This is a great qualitative research paper. It sheds light on questionable research practices. But more importantly it demonstrates direct replications in qualitative research work as you would expect: independent teams reach basically the same conclusions royalsocietypublishing.org/rsos/article/13/1/251888/47981…
-
Daniël Lakens
@lakens
Note that this in itself is a well-replicated finding. At least a dozen studies show that when researchers replicate qualitative research, or re-analyze it, they come to basically identical themes. There is nothing special about qualitative research with respect to replicability.
Daniël Lakens
@lakens
This is a great qualitative research paper. It sheds light on questionable research practices. But more importantly it demonstrates direct replications in qualitative research work as you would expect: independent teams reach basically the same conclusions royalsocietypublishing.org/rsos/article/13/1/251888/47981…
-
Daniël Lakens
@lakens
This is a great qualitative research paper. It sheds light on questionable research practices. But more importantly it demonstrates direct replications in qualitative research work as you would expect: independent teams reach basically the same conclusions royalsocietypublishing.org/rsos/article/13/1/251888/47981…
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
You still have time to sign up for the upcoming workshop of PMGS.
Olmo van den Akker will guide you through evaluating and writing high quality preregistration.
See more and sign up here:
paulmeehlschool.github.io/workshops/preregistration/ -
Daniël Lakens
@lakens
New episode of Nullius In Verba! We discuss the jingle-jangle fallacy, the problem of vague concepts, how the incentive structures promote vagueness, why people who prefer more rigour have to be called the validity 'police', and much more!
nulliusinverba.podbean.com/e/episode-74-notiones-vague/ -
Daniël Lakens
@lakens
Episode 74: Notiones Vague nulliusinverba.podbean.com/e/episode-74-notiones-vague/
-
Daniël Lakens
@lakens
René with some good suggestions during his workshop on peer review and scientific criticism on when and how to ignore suggestions by reviewers or editors, such as when they ask to drop non-significant results, or add hypotheses.
-
Daniël Lakens
@lakens
Full room for the Paul Meehl Graduate School workshop on Scientific Criticism and Peer Review taught by René Bekkers. In the introduction round, it is clear there is a lot of need for education on how to review the work of others!
-
Daniël Lakens reposted
Timothy Bates
@timothycbates
Building on their foundation of aspredicted.org and researchbox.org @uri_sohn and colleagues now fill the missing link with ascollected.org - you can document who collected, who cleaned, who has copies of the raw data in the project. Nice!
-
Daniël Lakens
@lakens
This was a really great discussion with people pushing the boundaries of scientific resesrch in their ManyMany projects. Really the cutting edge of scientific knowledge generation!
Daniël Lakens
@lakens
Tomorrow I will give a webinar on preregistration for the ManyMany's: What’s the use of preregistration for Big Team Science? You can join for free for the talk, and discussion afterwards.
For more info, see: manymanys.github.io/events/ -
Daniël Lakens
@lakens
Tomorrow I will give a webinar on preregistration for the ManyMany's: What’s the use of preregistration for Big Team Science? You can join for free for the talk, and discussion afterwards.
For more info, see: manymanys.github.io/events/ -
Daniël Lakens reposted
Laurent Prétôt 🐠 🐒 👶
@LaurentPretot
Join us tomorrow at 11 AM (ET) for an exciting event on pre-registration in big team science with @lakens. Free and open to all. Registration info ⬇️⬇️⬇️
ManyManys
@TheManyManys
📢 Join us for an exciting webinar on #preregistration in #bigteamscience with @lakens! When and where? Jan. 27 at 11 AM on Zoom ✨💡Free and open to all, both new and experienced in #openscience! 🌍 There is still time to register here manymanys.github.io/events/
-
Daniël Lakens
@lakens
Enhancing Transparency and Replicability in Entrepreneurship Research with Preregistrations, Registered Reports, and Registered Revisions: A Call for Papers
journals.sagepub.com/doi/full/10.1177/10422587251401175 -
Daniël Lakens
@lakens
In the age of 20 minute AI papers, I realize our model of honest signaling in academic publishing will become increasingly relevant:
journals.plos.org/plosone/article?id=10.1371/journal.pone…
More journals might adopt submission fees. If submission is costly, it will reduce dishonest paper submissions. -
Daniël Lakens
@lakens
I am intrigued by people who complain a lot, but do nothing to improve things. Some people complain about the same things for years. It seems to me that if they would spend that energy into something productive, they would be happier, and the world would be a bit better.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
The Perspectives on Scientific Error conference program is live! perspectivesonscientificerror2026.wordpress.com Keynotes: Leif Nelson about catching liars, Vlasta Sikimić about Science in the Age of AI, František Bartoš on Robustness Reports, and Femke Truijens on error in methods.
-
Daniël Lakens
@lakens
If in the year 2026 you think it is a good idea to edit a special issue for a Frontiers journal, you need to seriously rethink your life decisions.
-
Daniël Lakens
@lakens
One of my new year resolutions is to update my online MOOC. The first has been online for a decade, and around 7000 people enroll each year. That is a lot, and it is worth updating. I will combine both MOOCs into one, update everything, and align it with my free textbook.
-
Daniël Lakens
@lakens
The Perspectives on Scientific Error conference program is live! perspectivesonscientificerror2026.wordpress.com Keynotes: Leif Nelson about catching liars, Vlasta Sikimić about Science in the Age of AI, František Bartoš on Robustness Reports, and Femke Truijens on error in methods.
-
Daniël Lakens
@lakens
A hypothesis developed based on the data is often more likely to be true, than if you had not used the data.
The problem is not whether the hypothesis is true.
The problem is the hypothesis was not severely tested. You can't *claim* it is true until you test it on new data. -
Daniël Lakens
@lakens
Often a single Registered Report is more informative than a meta-analysis. The meta-analysis will show a non-zero estimate and we will not know if it is due to bias. Heterogeneity is huge, so the main recommendation of a meta-analysis is that future research is needed anyway.
-
Daniël Lakens
@lakens
At the Dutch Reproducibility meeting. We start with Uli Dirnagl who is discussing how to change behavior through the work of the Quest centre in Berlin. How do we generate more impact for patients through metascience?
-
Daniël Lakens
@lakens
Trump introduces a tariff for goods from The Netherlands. Great. Bring it on. I am happy to suffer any damage to our economy, and subsequent reduction to my wealth, for doing the right thing (being willing to protect Greenland).
-
Daniël Lakens
@lakens
Part 2 of our discussion on Scientism at Nullius in Verba is out:
nulliusinverba.podbean.com/e/episode-73-scientismus-ii/
We talk about 6 problems with scientism that have been raised by Susan Haack, and if we should feel bad about having some sympathy for scientism. -
Daniël Lakens
@lakens
Episode 73: Scientismus - II nulliusinverba.podbean.com/e/episode-73-scientismus-ii/
-
Daniël Lakens
@lakens
I know registrations opened only 4 days ago, but if you still want to sign up, do so quickly! There are only around 20 spots left!
Daniël Lakens
@lakens
The program for the Perspectives on Scientific Error conference Februari 11 to 13 in Leiden is online!
Registration is open! perspectivesonscientificerror2026.wordpress.com
This is one of the most inspiring and fun metascience meetings around - lots of time to interact, great talks - so do come! -
Daniël Lakens reposted
Alexandra Elbakyan
@ringo_ring
um, it seems that Sci-Hub .se domain has been finally blocked.
P.S. You can access Sci-Hub by alternative addresses such as sci-hub.ru (yes, this world needs alternative DNS and independent legal systems to ensure freedom - not a single world government) -
Daniël Lakens
@lakens
New study shows Portugese researchers engage in questionable research practices (sometimes frequently), even though most admit these are problematic. QRPs studied are mainly related to integrity, but includes hypothesizing after results are known.
journals.plos.org/plosone/article?id=10.1371/journal.pone… -
Daniël Lakens reposted
John D. Cook
@JohnDCook
Checklists are highly underrated.
See Atul Gawande's book Checklist Manifesto for examples. -
Daniël Lakens reposted
ReproducibiliTea
@ReproducibiliT
📽️"Taboo topics we can no longer avoid: negligence, tampering, and fraud" / with Ian Hussey @ianhussey
youtu.be/sdRRafephyM
The event was organised by ReproducibiliTea UniBasel @RTeaUnibasel -
Daniël Lakens
@lakens
The social constructivist movement in science has provided the fuel for the rise of anti-vaxxers and climate change denial. What is the best paper that provides a historical analysis of these negative consequences of social constructivism?
-
Daniël Lakens
@lakens
I really dislike opinionated teaching. I don't think teachers should tell students 'but this view is wrong' - you should teach students to evaluate the arguments for and against, and expect (and even want!) some students to disagree with you after they reach their own conclusion.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
The program for the Perspectives on Scientific Error conference Februari 11 to 13 in Leiden is online!
Registration is open! perspectivesonscientificerror2026.wordpress.com
This is one of the most inspiring and fun metascience meetings around - lots of time to interact, great talks - so do come! -
Daniël Lakens
@lakens
The program for the Perspectives on Scientific Error conference Februari 11 to 13 in Leiden is online!
Registration is open! perspectivesonscientificerror2026.wordpress.com
This is one of the most inspiring and fun metascience meetings around - lots of time to interact, great talks - so do come! -
Daniël Lakens
@lakens
The uprising in Iran is incredibly brave. I hope this time it will be successful!
-
Daniël Lakens
@lakens
You can sign up until January 23 for the free Paul Meehl Graduate School workshop on Scientific Criticism and Peer review taught by René Bekkers. It promises to be an extremely interesting day, so do join us in Eindhoven on January 30th! paulmeehlschool.github.io/workshops/criticism/
-
Daniël Lakens reposted
Colin Camerer
@CFCamerer
NEW we are hiring 2-3 Postdocs in Social & Decision Neuroscience (SDN) at Caltech. The posting is here www.hss.caltech.edu/about/job-op...Applications are due 15 February 2026. Our core group in SDN is RAdolphs, me, DMobbs, JO'Doherty, and ARangel. Our track record of postdoc success is strong
-
Daniël Lakens
@lakens
I don't know who Bill Ackman is, but if he had been more famous this would have been a great teaching example for my metascience course on how people are fooled by confirmation bias. He wants this to be true so much, he has convinced himself it must be so.
Bill Ackman
@BillAckman
For those interested in the @francescagino @HarvardHBS case, please read this transcript. It explains in great detail the sources/causes of the data anomalies in her papers that were the basis for Harvard taking away her tenure.
Those that have been convinced that Francesca is guilty of academic fraud generally focus all of their attention on the @DataColada analysis.
While Data Colada deserves credit for identifying the data anomalies, it did not provide any proof or evidence as to the source of the anomalies, whether they came from Gino, her research associates or otherwise.
This transcript will open your eyes to the truth. Please give it a careful read. Thank you.
theginocase.info/wp-content/uploads/2025/11/The-Law-S4E4.pdf
Bill Ackman
@BillAckman
Almost nothing makes my blood boil more than when a large powerful institution unfairly destroys someone’s reputation, and its principal reason for doing so is to minimize bad publicity in an effort to protect its own ‘reputation.’
Sadly, I have seen this occur in many academic institutions when a faculty member or student’s reputation, career, and often life are destroyed for a crime they did not commit. In the good cases, it is often many years later where all of the facts emerge and the individual is exonerated, but unfortunately justice delayed is often justice denied. Sadly, many such examples lead the accused to depression and occasionally even suicide.
In the case I am writing about, the facts are complicated, and despite my predilection for long-winded posts (and this one is not going to be short), I am not going to have room to provide many of the details of the case here, but instead I will refer those that are interested in learning more to a superb, four-part podcast that goes into the gory details (you can comfortably listen at 1.5 times speed) which, with transcripts, can be found here:
theginocase.info?page_id=19
This is an important story about @Harvard, academic institutions, due process, and someone whose life is at stake, so I thought to share the high-level facts in an effort for this story to be heard, perhaps someday eventually as widely as the accusations and the conviction have already been disseminated.
By way of background, I have long had an interest in behavioral science/economics. To that end, the Pershing Square Foundation made grants of approximately $40 million to Harvard University to enable it to recruit some of the best behavioral scientists/economists in the world to create a center of excellence (The Foundations of Human Behavior) for research into better understanding human behavior.
My interest in Harvard and behavioral economics made me particularly interested in a story that first publicly appeared in June of 2023 when a Harvard Business School professor named Francesca Gino whose research focused on “honesty and ethical behavior” was (ironically) accused of falsifying her research. In June 2023, she was put on administrative leave, stripped of her titles, and removed as head of the Unit of Negotiation, Organizations and Markets at HBS. Nearly two years later, after a lengthy investigation, in May 2025, Gino's tenure was revoked and she was fired.
The Gino story caught my attention for a number of reasons. I am HBS alum. I have been one of the largest funders of behavioral economics at Harvard, the department of which Gino was a member, and since October 7th, 2023, I had developed a much more skeptical view of the University. Also, my father was an early and large funder of ethics research at HBS so his ethics endowment likely has been the source of funding for some of Gino’s research. So I had many reasons to be interested in this case.
The story itself was also one made for the headlines. The idea that a tenured Harvard professor whose research was focused on honesty and ethical behavior committed research fraud was guaranteed to go super viral.
I was also open to the other side of the story. Early in my career, I was falsely accused of market manipulation for a research report I released on MBIA and a short position that I took in the company. I shortly thereafter found myself under investigation (catalyzed by MBIA’s power and influence as the largest guarantor of NY State and City bonds) by then NY Attorney General Eliot Spitzer and the SEC which followed suit.
My reputation was quickly destroyed by the resulting publicity, and I spent nearly a year in depositions defending myself. I was fortunate in being able to afford good counsel and in having family, friends, and business partners who supported me through this extremely challenging time.
The NYAG and the SEC eventually went away, but they did not formally withdraw the investigations until years later. Fortunately, by then, I was totally vindicated, but it was more than five years between being accused, before the facts – MBIA’s and the other bond insurers implosion during the GFC – exonerated me.
While I had always believed that one is innocent until proven guilty, going through such an experience made me particularly sensitive to this highly important feature of our legal system and our democracy, which brings me back to Francesca Gino.
Francesca reached out to me in late 2023 (I first met her years before when she had presented her work to the HBS Board of Dean’s Advisors of which I was then a member), a time when I was highly publicly critical of then-Harvard president Claudine Gay. Gay was found to have plagiarized large portions of her limited research oeuvre (50 or so unattributed word-for-word insertions in 8 of 17 of her papers).
The resulting publicity combined with her mishandling of an explosion of anti-Semitism on campus led to her stepping down from the presidency. But despite the vast amount of unattributed work in Gay’s published research, she remains at Harvard to this day as a $1 million per annum or so member of the faculty, apparently in good standing.
While I noted the disparity between how Gino’s case compared with how Gay’s was handled by the University, at that point, I did not know the details about Gino’s situation.
In May 2024, Francesca reached out to me for help. She had spent nearly all of her resources funding her defense and was running out of money. I expressed an interest in learning more and she organized a Zoom with Laurence Lessig, a highly respected and distinguished law school professor at Harvard and the former director of the Edmund J. Safra Center for Ethics at Harvard who was helping her pro bono. David Klafter, a partner of mine and my most trusted advisor on these kinds of matters, joined me on the Zoom. Lessig made a very powerful case for Gino’s innocence and offered to provide us with whatever information we needed to understand her side of the story.
To make a long story short, David and I have carefully examined the evidence and we strongly believe that Gino is entirely innocent. Furthermore, by no means has Harvard met its “clear and convincing” standard for finding academic misconduct. Lastly, it appears that Harvard may have also violated its own statute of limitations provisions in terminating Gino’s tenure as the papers in questions are more than a decade old, and Harvard’s statute of limitations is six years. While the facts are complicated and take time to understand, a clear picture emerges once you do the work.
In short, Gino appears to be a victim of some unintentional data errors made by some of the research associates that assisted her in her research as well as scammer(s) who earned fees for answering various surveys that led to corrupted data sets for some of her papers (behavioral science researcher often use sites like Mechanical Turk (mturk.com) where random members of the public can earn fees for filling out surveys which are used as sources of data).
You might ask why if we were able to get to this conclusion based on impartial analysis would Harvard come to a different conclusion, a conclusion (finding Gino guilty) which one would assume is highly damaging to Harvard’s reputation. The answer, in short, is that we approached our analysis without any bias or desire to get to a particular conclusion, and we have the benefit of some distance and a lack of emotion about the situation.
For context, HBS played an important role in my education and success, and I continue to teach classes and speak to students there every year. I have a high regard for Srikant Datar, the Dean, on whose advisory board I used to serve. While I care enormously about the institution, I care more about the truth. Once you come to understand how big institutions and their leaders respond to the fear of negative publicity and the incentives of the attention economy, you can better understand why an innocent person has been found guilty here. In some ways, this is the more interesting part of the story.
The story begins with Harvard being contacted in June 2021 by datacolada.org (“DC”), an entity that seeks to identify fraud and replication issues in academic research. DC’s policy is to bring data and/or replication issues to the attention of the author so that the author can either defend or correct their work. In this case, DC had found data anomalies in four of Gino’s 140 papers. Interestingly, DC did not follow its own policy of approaching the author, and instead approached Harvard and shared its analysis, while not disclosing the issues it had identified to Gino.
Shortly after being approached by DC about anomalies it had identified in Gino’s work, Harvard discarded its pre-existing, two-page research integrity policy – one which had been created with the input of Harvard’s senior faculty and had been in place for many years – and replaced it with a new 16-page policy, and did so without seeking the consent or even informing the Harvard faculty that it was doing so.
The new 16-page policy had a number of notable additions. Of significance, the new policy had a confidentiality provision which made it a termination offense if the subject of an investigation publicly disclosed any of the facts about the investigation, including whether an investigation was underway. Second, it limited the subject under investigation to work with only two individuals as their advisors and restricted the subject’s communication with the administration to one person, a newly created position of Research Integrity Officer.
Why did Harvard revise its policy under the cover of darkness and introduce these new provisions in connection with the Gino case? The answer I believe is that Harvard feared the potential reputational damage of having a professor who studied ethics and integrity being found guilty of committing research fraud.
Harvard wanted to limit the risk of being embarrassed and to carefully control the release of any information about the case so it gagged Gino, severely limited those she could work with in formulating her defense, and restricted her access to the faculty and members of the administration leading the investigation.
I believe that Harvard thought the accusations were so reputationally damaging to the institution that it was hoping to resolve the matter as privately as possible, and if disclosure was required, to do so in a way that minimized the reputational damage to the University. While these are reasonable goals, the impact on Gino was to eliminate her due process and fundamentally take away her ability to defend herself until after the University had already come to a false conclusion about her culpability, the train had left the station, and the damage was done.
Harvard was also under pressure to complete its investigation. Data Colada was clearly excited about its findings of data anomalies in works by a Harvard ethics professor and likely viewed the scoop as a great story for a future media onslaught that would ‘advance’ Data Colada's brand and image in the mind of the general public.
We live in an attention economy where scoops like this one could be transformative in putting DC on the front page of the NY Times and cause it to go viral on social media, which in fact it later did. This is the only credible explanation I can surmise as to why DC violated its policy of sharing their findings with the author first. This was clearly a breakout public relations opportunity for DC and it did not want it to go to waste.
DC also put significant pressure on Harvard to complete its investigation in a timely fashion. In order to keep Harvard’s goal of confidentiality, the Dean of HBS, Srikant Datar, made a deal with DC where it agreed not to publicly release its findings if Harvard promptly launched its own investigation and while the investigation was under way. And DC continually pressured Harvard to get its work done quickly.
When a Harvard professor is accused of research misconduct, the University's policy is to bring these accusations to the professor within a week of learning of them. But in this case, Harvard waited more than three months to inform Gino that she was under investigation. And when Harvard finally told Gino she was under investigation, it did not share DC’s analysis or the details of the investigation with her until months later.
The Gino gag order limited her to sharing info about the investigation to only two individuals as her advisors. She chose a faculty colleague as one of her advisors. The other advisor was a lawyer recommended by the Research Integrity Officer (RIO), a Harvard administrator named Alain Bonacossa, who represented the University and the Hearing Committee in its investigation.
The Gino gag order remained in place until the University concluded its investigation so it would be nearly two years from the time DC approached Harvard before she could begin defending herself publicly. Harvard on the other hand hired Ropes & Gray and an army of other lawyers and public relations advisors to assist it in its investigation and prosecution of Gino.
After the inquiry stage of the investigation was completed, which took several months, Gino was informed that a formal investigation would now begin, and that the University would be hiring a forensic research firm, Maidstone, to assist it in its investigation, among other things, to review the terabytes of data under question.
When Gino asked RIO Bonacosso whether she could hire her own forensic research firm (Gino’s advisors strongly recommended that she do so), he told her that she could not as she had already identified her two advisors and was not permitted to hire a third.
[Later, in trial and under oath, Bonacosso denied that he told Gino that she could not hire a forensic firm. His statement under oath strains credulity in that of course Gino would have hired a forensic firm if she was permitted to do so, particularly as her advisors strongly suggested that she do so.]
When Gino was first notified that she was under investigation, she was required to immediately turn in all of her University electronic devices. She was told that the University would make a forensic image of her devices so that all metadata and other files would be preserved.
The forensic image would have records of websites she visited and include system logs which could reveal when and how files were created, including remnants of deleted files. A forensic image of this information was critical as it is not stored forever as system logs have routines to erase these data and files after a period of time.
Gino later learned that the University did not make a forensic image of her devices and only copied a subset of the files. As a result, important data that could have exonerated her more easily have been permanently destroyed. The loss of this information also impaired the work of Maidstone in doing its forensic analysis and made Gino’s defense more challenging.
Gino was not permitted to speak to or interview any of her research assistants that worked on her papers as part of her defense. She was initially comforted by the fact that she was told that the University would do so as part of their investigation. In fact, the University interviewed only two of the 12 RAs that worked on the subject papers, and to only a limited extent.
Clearly, minimizing public disclosure rather than getting to the truth was Harvard’s driving principle in this investigation for what good reason would there be to limit an investigation to only a tiny subset of the RAs who led the research for the papers in question.
After the completion of the Maidstone and other investigative reports, Gino received copies of the drafts to review, but was given only two or so weeks to complete her review before she had her first and only opportunity to speak to the Investigative Committee members on a Zoom call to respond to their questions arising from the Maidstone forensic analysis, without the benefit of having her own forensic consultant to review and challenge the report. When Gino requested more time to analyze the Maidstone report in preparation for her interview, that request was denied by the Committee.
On the Zoom call with the Committee, Gino was asked to explain how the anomalies in some of her datasets had occurred. Without the benefit of access to the underlying metadata and without her own forensic data expert, she was not able to answer the Committee’s questions to their satisfaction.
While Maidstone did not determine who was responsible for the data anomalies, the Committee ultimately concluded that Gino was responsible. Bear in mind that the Investigative Committee came to this conclusion without the benefit of interviewing 10 of the 12 research assistants who worked on the papers with Gino and without hearing from a forensic expert who represented Gino who could challenge Maidstone's analysis.
The Committee ultimately accepted the conclusions of the Maidstone report. On June 13th, 2023, Gino was encouraged to resign with Harvard proposing that ‘things can go quietly’ or words to that effect if she were to do so. Gino refused to resign, proclaiming her innocence. Four days later, on June 17th, Harvard went public with the accusations and the University thereafter put Gino on administrative leave.
Once Harvard went public, Data Colada released its analysis to the public and the press and social media had a field day. At this point, Gino was now no longer gagged. She was then able to hire her own forensic expert and begin a careful review and detailed analysis of the Maidstone report. She finally had access to her own data and the resources to evaluate the facts, but without the data which were destroyed by the University’s failure to make a forensic image of her devices.
Gino and her advisors spend the next six months analyzing her data and the Maidstone report and find serious flaws with their analysis. In January 2024, Gino’s lawyers approach Harvard and ask for the opportunity to make a presentation about the flaws they have identified which discredited the Maidstone report.
Gino's lawyers’ request is denied and Harvard thereafter asks what Gino wants to resolve the matter. She of course wants her job back as she believes that she can now prove that there has been no wrongdoing. Harvard says that returning to work at the University is not possible, but offers the possibility of a financial settlement, an offer Gino promptly rejects.
Having rejected the opportunity to go quietly, Gino shortly thereafter receives a letter from President Gay informing her that the ‘Third Statute’ tenure review process is now underway, proceedings to revoke tenure which apparently had not previously been invoked in the nearly 400 years since Harvard’s founding.
Gino is told that the basis for the launch of the tenure revocation review was a ‘painstaking and comprehensive process’ which relied on the conclusions of the Maidstone report which provides ‘clear and convincing evidence’ of academic misconduct.
So before Gino has a proper opportunity to respond to the Maidstone report with the benefit of expert advice, Harvard has already accepted the report as fact, a report which becomes the basis for the University seeking to revoke her tenure.
Fourteen months later, Gino and her forensic expert respond to the Maidstone report with a detailed 93-page analysis identifying serious flaws in the report which they deliver to the Hearing Committee on August 1st, 2024, with the University required to respond under the Third Statute rules by August 16, 2024.
On August 16th, 2024, Harvard effectively withdraws the flawed Maidstone report, and over Gino’s lawyers strong objections, submits a new 230-page report from Stanford Sociology professor Jeremy Freese who has new theories of research misconduct in Gino’s work.
The submission of the Freese report is the first time Gino and her counsel are informed that the University is no longer relying on the Maidstone report for their tenure revocation process, and the first they learn of Professor Freese’s involvement.
The rules of the Third Statute proceedings provide that discovery is now closed impairing Gino’s and her expert’s ability to respond to these new allegations, but the rules somehow allow the new claims and evidence in the Freese report to be submitted by the University.
Bear in mind that Gino and her forensic expert had spent the previous 14 months preparing its response to the Maidstone report at a cost of $2 million. Now that work has effectively been tossed aside, and she is now asked to respond to a new expert who brings in new evidence with new theories of wrongdoing and new hypothetical scenarios of academic fraud, and she is not given access to needed additional discovery.
Gino is given one month to rebut the Freese report. She asks for more time and for additional discovery, and both requests are denied.
To make a very long story short, the Hearing Committee thereafter concludes that Gino is guilty as charged and the University revokes her tenure on May 20, 2025.
Now imagine you were accused of a crime by a company you work for, but you are not allowed to know what the crime is. You are limited in your defense to hiring two people – not two law firms – but two people.
The company has $50 billion of financial resources. You are a relatively senior employee but with limited resources. The company hires a forensic expert and armies of lawyers that work through terabytes of data and concludes that you are a crook.
Exonerating evidence in the possession of the company is destroyed by the company’s failure to preserve data. You are not allowed to hire your own financial expert and lawyers until after the company concludes that you are guilty. At this point, the ship has sailed and the Titanic does not turn on a dime. The press has already convicted you in the mind of the public.
After your reputation is in tatters, you are able to begin to marshal a defense. You are able to dispute each of the various conclusions of the company when you finally can hire your own expert, but the company ignores your defense and at the eleventh hour, hires a new expert and comes up with new theories of wrongdoing.
The company gives you 30 days to respond to the new report, but it is not feasible for you to adequately respond because the discovery period is deemed ended by the company and 30 days is not nearly sufficient time to investigate and respond to the new allegations.
You have been fired, the first employee in the company’s nearly 400-year history to be fired by a special Third Statute process, and you can’t get a new job because your reputation has been destroyed. You have four small children to support and your financial resources are gone.
What would you do?
You would sue the company in a real court of law where the Constitution and legal precedent protects you. And that it what Gino has done. Discovery is supposed to be completed by the end of this month with a trial scheduled this coming December.
When Francesca reached out to me in June of 2024 for help, I was sufficiently compelled to do so. We have been funding her legal and expert costs since that time, and we will provide whatever resources she needs to clear her name.
The problem with big institutions is that they are massive bureaucracies which occasionally put aside the core principles on which they are founded because they are embarrassed, because they are busy, and/or it is easier to ignore the harm they may cause to the little guy in the interest of protecting their reputation and/or avoiding further distraction to leadership. They also often assume that the little guy will eventually give up and/or run out of resources to keep fighting.
I am going to make sure that doesn’t happen here.
I learned a lot when I went to Harvard. I am applying what I learned at Harvard to protect Harvard from itself and to help someone who needs help because Veritas matters.
If Harvard would like to resolve this outside of a court of law, I am here to help. Sadly, my guess is that Harvard won’t call as they are too proud and pot committed.
Daniël Lakens
@lakens
It is important to realize academics have so little oversight, and need to log so little of how they do science, you can always blame a rogue research assistant for your fraud. The question is if we want let people get away so easily.
www.the100.ci/2024/12/18/rogue-ra/. -
Daniël Lakens reposted
Daniël Lakens
@lakens
Starting the new year with a new podcast episode of Nullius In Verba on 'Scientism' nulliusinverba.podbean.com/e/episode-72-scientismus-i/
In the first of two parts, we ask if it is ok to think that science is the best way to generate knowledge, or if this is too dismissing of 'other ways of knowing'. -
Daniël Lakens
@lakens
It makes me happy to see that a scientific book that looks interesting is available for free, open access. It also raises my expectation that it will be a good book, as the authors really wanted it to be read (unlike authors who merely want to have a book published).
-
Daniël Lakens
@lakens
What the USA has done is absolutely disgraceful. Time for a boycot of US products, especially tech, and for the population in the USA to instigate a change in leadership in their own country.
-
Daniël Lakens reposted
Quentin André
@andre_quentin
Bill Ackman, apparently running out of things to be confidently wrong about, has decided to defend (and bankroll!) Francesca Gino.
I have a paper on "south-pointing compasses" : Advisors that are so often wrong that it makes them very diagnostic of the truth. Bill is like that.
Bill Ackman
@BillAckman
Almost nothing makes my blood boil more than when a large powerful institution unfairly destroys someone’s reputation, and its principal reason for doing so is to minimize bad publicity in an effort to protect its own ‘reputation.’
Sadly, I have seen this occur in many academic institutions when a faculty member or student’s reputation, career, and often life are destroyed for a crime they did not commit. In the good cases, it is often many years later where all of the facts emerge and the individual is exonerated, but unfortunately justice delayed is often justice denied. Sadly, many such examples lead the accused to depression and occasionally even suicide.
In the case I am writing about, the facts are complicated, and despite my predilection for long-winded posts (and this one is not going to be short), I am not going to have room to provide many of the details of the case here, but instead I will refer those that are interested in learning more to a superb, four-part podcast that goes into the gory details (you can comfortably listen at 1.5 times speed) which, with transcripts, can be found here:
theginocase.info?page_id=19
This is an important story about @Harvard, academic institutions, due process, and someone whose life is at stake, so I thought to share the high-level facts in an effort for this story to be heard, perhaps someday eventually as widely as the accusations and the conviction have already been disseminated.
By way of background, I have long had an interest in behavioral science/economics. To that end, the Pershing Square Foundation made grants of approximately $40 million to Harvard University to enable it to recruit some of the best behavioral scientists/economists in the world to create a center of excellence (The Foundations of Human Behavior) for research into better understanding human behavior.
My interest in Harvard and behavioral economics made me particularly interested in a story that first publicly appeared in June of 2023 when a Harvard Business School professor named Francesca Gino whose research focused on “honesty and ethical behavior” was (ironically) accused of falsifying her research. In June 2023, she was put on administrative leave, stripped of her titles, and removed as head of the Unit of Negotiation, Organizations and Markets at HBS. Nearly two years later, after a lengthy investigation, in May 2025, Gino's tenure was revoked and she was fired.
The Gino story caught my attention for a number of reasons. I am HBS alum. I have been one of the largest funders of behavioral economics at Harvard, the department of which Gino was a member, and since October 7th, 2023, I had developed a much more skeptical view of the University. Also, my father was an early and large funder of ethics research at HBS so his ethics endowment likely has been the source of funding for some of Gino’s research. So I had many reasons to be interested in this case.
The story itself was also one made for the headlines. The idea that a tenured Harvard professor whose research was focused on honesty and ethical behavior committed research fraud was guaranteed to go super viral.
I was also open to the other side of the story. Early in my career, I was falsely accused of market manipulation for a research report I released on MBIA and a short position that I took in the company. I shortly thereafter found myself under investigation (catalyzed by MBIA’s power and influence as the largest guarantor of NY State and City bonds) by then NY Attorney General Eliot Spitzer and the SEC which followed suit.
My reputation was quickly destroyed by the resulting publicity, and I spent nearly a year in depositions defending myself. I was fortunate in being able to afford good counsel and in having family, friends, and business partners who supported me through this extremely challenging time.
The NYAG and the SEC eventually went away, but they did not formally withdraw the investigations until years later. Fortunately, by then, I was totally vindicated, but it was more than five years between being accused, before the facts – MBIA’s and the other bond insurers implosion during the GFC – exonerated me.
While I had always believed that one is innocent until proven guilty, going through such an experience made me particularly sensitive to this highly important feature of our legal system and our democracy, which brings me back to Francesca Gino.
Francesca reached out to me in late 2023 (I first met her years before when she had presented her work to the HBS Board of Dean’s Advisors of which I was then a member), a time when I was highly publicly critical of then-Harvard president Claudine Gay. Gay was found to have plagiarized large portions of her limited research oeuvre (50 or so unattributed word-for-word insertions in 8 of 17 of her papers).
The resulting publicity combined with her mishandling of an explosion of anti-Semitism on campus led to her stepping down from the presidency. But despite the vast amount of unattributed work in Gay’s published research, she remains at Harvard to this day as a $1 million per annum or so member of the faculty, apparently in good standing.
While I noted the disparity between how Gino’s case compared with how Gay’s was handled by the University, at that point, I did not know the details about Gino’s situation.
In May 2024, Francesca reached out to me for help. She had spent nearly all of her resources funding her defense and was running out of money. I expressed an interest in learning more and she organized a Zoom with Laurence Lessig, a highly respected and distinguished law school professor at Harvard and the former director of the Edmund J. Safra Center for Ethics at Harvard who was helping her pro bono. David Klafter, a partner of mine and my most trusted advisor on these kinds of matters, joined me on the Zoom. Lessig made a very powerful case for Gino’s innocence and offered to provide us with whatever information we needed to understand her side of the story.
To make a long story short, David and I have carefully examined the evidence and we strongly believe that Gino is entirely innocent. Furthermore, by no means has Harvard met its “clear and convincing” standard for finding academic misconduct. Lastly, it appears that Harvard may have also violated its own statute of limitations provisions in terminating Gino’s tenure as the papers in questions are more than a decade old, and Harvard’s statute of limitations is six years. While the facts are complicated and take time to understand, a clear picture emerges once you do the work.
In short, Gino appears to be a victim of some unintentional data errors made by some of the research associates that assisted her in her research as well as scammer(s) who earned fees for answering various surveys that led to corrupted data sets for some of her papers (behavioral science researcher often use sites like Mechanical Turk (mturk.com) where random members of the public can earn fees for filling out surveys which are used as sources of data).
You might ask why if we were able to get to this conclusion based on impartial analysis would Harvard come to a different conclusion, a conclusion (finding Gino guilty) which one would assume is highly damaging to Harvard’s reputation. The answer, in short, is that we approached our analysis without any bias or desire to get to a particular conclusion, and we have the benefit of some distance and a lack of emotion about the situation.
For context, HBS played an important role in my education and success, and I continue to teach classes and speak to students there every year. I have a high regard for Srikant Datar, the Dean, on whose advisory board I used to serve. While I care enormously about the institution, I care more about the truth. Once you come to understand how big institutions and their leaders respond to the fear of negative publicity and the incentives of the attention economy, you can better understand why an innocent person has been found guilty here. In some ways, this is the more interesting part of the story.
The story begins with Harvard being contacted in June 2021 by datacolada.org (“DC”), an entity that seeks to identify fraud and replication issues in academic research. DC’s policy is to bring data and/or replication issues to the attention of the author so that the author can either defend or correct their work. In this case, DC had found data anomalies in four of Gino’s 140 papers. Interestingly, DC did not follow its own policy of approaching the author, and instead approached Harvard and shared its analysis, while not disclosing the issues it had identified to Gino.
Shortly after being approached by DC about anomalies it had identified in Gino’s work, Harvard discarded its pre-existing, two-page research integrity policy – one which had been created with the input of Harvard’s senior faculty and had been in place for many years – and replaced it with a new 16-page policy, and did so without seeking the consent or even informing the Harvard faculty that it was doing so.
The new 16-page policy had a number of notable additions. Of significance, the new policy had a confidentiality provision which made it a termination offense if the subject of an investigation publicly disclosed any of the facts about the investigation, including whether an investigation was underway. Second, it limited the subject under investigation to work with only two individuals as their advisors and restricted the subject’s communication with the administration to one person, a newly created position of Research Integrity Officer.
Why did Harvard revise its policy under the cover of darkness and introduce these new provisions in connection with the Gino case? The answer I believe is that Harvard feared the potential reputational damage of having a professor who studied ethics and integrity being found guilty of committing research fraud.
Harvard wanted to limit the risk of being embarrassed and to carefully control the release of any information about the case so it gagged Gino, severely limited those she could work with in formulating her defense, and restricted her access to the faculty and members of the administration leading the investigation.
I believe that Harvard thought the accusations were so reputationally damaging to the institution that it was hoping to resolve the matter as privately as possible, and if disclosure was required, to do so in a way that minimized the reputational damage to the University. While these are reasonable goals, the impact on Gino was to eliminate her due process and fundamentally take away her ability to defend herself until after the University had already come to a false conclusion about her culpability, the train had left the station, and the damage was done.
Harvard was also under pressure to complete its investigation. Data Colada was clearly excited about its findings of data anomalies in works by a Harvard ethics professor and likely viewed the scoop as a great story for a future media onslaught that would ‘advance’ Data Colada's brand and image in the mind of the general public.
We live in an attention economy where scoops like this one could be transformative in putting DC on the front page of the NY Times and cause it to go viral on social media, which in fact it later did. This is the only credible explanation I can surmise as to why DC violated its policy of sharing their findings with the author first. This was clearly a breakout public relations opportunity for DC and it did not want it to go to waste.
DC also put significant pressure on Harvard to complete its investigation in a timely fashion. In order to keep Harvard’s goal of confidentiality, the Dean of HBS, Srikant Datar, made a deal with DC where it agreed not to publicly release its findings if Harvard promptly launched its own investigation and while the investigation was under way. And DC continually pressured Harvard to get its work done quickly.
When a Harvard professor is accused of research misconduct, the University's policy is to bring these accusations to the professor within a week of learning of them. But in this case, Harvard waited more than three months to inform Gino that she was under investigation. And when Harvard finally told Gino she was under investigation, it did not share DC’s analysis or the details of the investigation with her until months later.
The Gino gag order limited her to sharing info about the investigation to only two individuals as her advisors. She chose a faculty colleague as one of her advisors. The other advisor was a lawyer recommended by the Research Integrity Officer (RIO), a Harvard administrator named Alain Bonacossa, who represented the University and the Hearing Committee in its investigation.
The Gino gag order remained in place until the University concluded its investigation so it would be nearly two years from the time DC approached Harvard before she could begin defending herself publicly. Harvard on the other hand hired Ropes & Gray and an army of other lawyers and public relations advisors to assist it in its investigation and prosecution of Gino.
After the inquiry stage of the investigation was completed, which took several months, Gino was informed that a formal investigation would now begin, and that the University would be hiring a forensic research firm, Maidstone, to assist it in its investigation, among other things, to review the terabytes of data under question.
When Gino asked RIO Bonacosso whether she could hire her own forensic research firm (Gino’s advisors strongly recommended that she do so), he told her that she could not as she had already identified her two advisors and was not permitted to hire a third.
[Later, in trial and under oath, Bonacosso denied that he told Gino that she could not hire a forensic firm. His statement under oath strains credulity in that of course Gino would have hired a forensic firm if she was permitted to do so, particularly as her advisors strongly suggested that she do so.]
When Gino was first notified that she was under investigation, she was required to immediately turn in all of her University electronic devices. She was told that the University would make a forensic image of her devices so that all metadata and other files would be preserved.
The forensic image would have records of websites she visited and include system logs which could reveal when and how files were created, including remnants of deleted files. A forensic image of this information was critical as it is not stored forever as system logs have routines to erase these data and files after a period of time.
Gino later learned that the University did not make a forensic image of her devices and only copied a subset of the files. As a result, important data that could have exonerated her more easily have been permanently destroyed. The loss of this information also impaired the work of Maidstone in doing its forensic analysis and made Gino’s defense more challenging.
Gino was not permitted to speak to or interview any of her research assistants that worked on her papers as part of her defense. She was initially comforted by the fact that she was told that the University would do so as part of their investigation. In fact, the University interviewed only two of the 12 RAs that worked on the subject papers, and to only a limited extent.
Clearly, minimizing public disclosure rather than getting to the truth was Harvard’s driving principle in this investigation for what good reason would there be to limit an investigation to only a tiny subset of the RAs who led the research for the papers in question.
After the completion of the Maidstone and other investigative reports, Gino received copies of the drafts to review, but was given only two or so weeks to complete her review before she had her first and only opportunity to speak to the Investigative Committee members on a Zoom call to respond to their questions arising from the Maidstone forensic analysis, without the benefit of having her own forensic consultant to review and challenge the report. When Gino requested more time to analyze the Maidstone report in preparation for her interview, that request was denied by the Committee.
On the Zoom call with the Committee, Gino was asked to explain how the anomalies in some of her datasets had occurred. Without the benefit of access to the underlying metadata and without her own forensic data expert, she was not able to answer the Committee’s questions to their satisfaction.
While Maidstone did not determine who was responsible for the data anomalies, the Committee ultimately concluded that Gino was responsible. Bear in mind that the Investigative Committee came to this conclusion without the benefit of interviewing 10 of the 12 research assistants who worked on the papers with Gino and without hearing from a forensic expert who represented Gino who could challenge Maidstone's analysis.
The Committee ultimately accepted the conclusions of the Maidstone report. On June 13th, 2023, Gino was encouraged to resign with Harvard proposing that ‘things can go quietly’ or words to that effect if she were to do so. Gino refused to resign, proclaiming her innocence. Four days later, on June 17th, Harvard went public with the accusations and the University thereafter put Gino on administrative leave.
Once Harvard went public, Data Colada released its analysis to the public and the press and social media had a field day. At this point, Gino was now no longer gagged. She was then able to hire her own forensic expert and begin a careful review and detailed analysis of the Maidstone report. She finally had access to her own data and the resources to evaluate the facts, but without the data which were destroyed by the University’s failure to make a forensic image of her devices.
Gino and her advisors spend the next six months analyzing her data and the Maidstone report and find serious flaws with their analysis. In January 2024, Gino’s lawyers approach Harvard and ask for the opportunity to make a presentation about the flaws they have identified which discredited the Maidstone report.
Gino's lawyers’ request is denied and Harvard thereafter asks what Gino wants to resolve the matter. She of course wants her job back as she believes that she can now prove that there has been no wrongdoing. Harvard says that returning to work at the University is not possible, but offers the possibility of a financial settlement, an offer Gino promptly rejects.
Having rejected the opportunity to go quietly, Gino shortly thereafter receives a letter from President Gay informing her that the ‘Third Statute’ tenure review process is now underway, proceedings to revoke tenure which apparently had not previously been invoked in the nearly 400 years since Harvard’s founding.
Gino is told that the basis for the launch of the tenure revocation review was a ‘painstaking and comprehensive process’ which relied on the conclusions of the Maidstone report which provides ‘clear and convincing evidence’ of academic misconduct.
So before Gino has a proper opportunity to respond to the Maidstone report with the benefit of expert advice, Harvard has already accepted the report as fact, a report which becomes the basis for the University seeking to revoke her tenure.
Fourteen months later, Gino and her forensic expert respond to the Maidstone report with a detailed 93-page analysis identifying serious flaws in the report which they deliver to the Hearing Committee on August 1st, 2024, with the University required to respond under the Third Statute rules by August 16, 2024.
On August 16th, 2024, Harvard effectively withdraws the flawed Maidstone report, and over Gino’s lawyers strong objections, submits a new 230-page report from Stanford Sociology professor Jeremy Freese who has new theories of research misconduct in Gino’s work.
The submission of the Freese report is the first time Gino and her counsel are informed that the University is no longer relying on the Maidstone report for their tenure revocation process, and the first they learn of Professor Freese’s involvement.
The rules of the Third Statute proceedings provide that discovery is now closed impairing Gino’s and her expert’s ability to respond to these new allegations, but the rules somehow allow the new claims and evidence in the Freese report to be submitted by the University.
Bear in mind that Gino and her forensic expert had spent the previous 14 months preparing its response to the Maidstone report at a cost of $2 million. Now that work has effectively been tossed aside, and she is now asked to respond to a new expert who brings in new evidence with new theories of wrongdoing and new hypothetical scenarios of academic fraud, and she is not given access to needed additional discovery.
Gino is given one month to rebut the Freese report. She asks for more time and for additional discovery, and both requests are denied.
To make a very long story short, the Hearing Committee thereafter concludes that Gino is guilty as charged and the University revokes her tenure on May 20, 2025.
Now imagine you were accused of a crime by a company you work for, but you are not allowed to know what the crime is. You are limited in your defense to hiring two people – not two law firms – but two people.
The company has $50 billion of financial resources. You are a relatively senior employee but with limited resources. The company hires a forensic expert and armies of lawyers that work through terabytes of data and concludes that you are a crook.
Exonerating evidence in the possession of the company is destroyed by the company’s failure to preserve data. You are not allowed to hire your own financial expert and lawyers until after the company concludes that you are guilty. At this point, the ship has sailed and the Titanic does not turn on a dime. The press has already convicted you in the mind of the public.
After your reputation is in tatters, you are able to begin to marshal a defense. You are able to dispute each of the various conclusions of the company when you finally can hire your own expert, but the company ignores your defense and at the eleventh hour, hires a new expert and comes up with new theories of wrongdoing.
The company gives you 30 days to respond to the new report, but it is not feasible for you to adequately respond because the discovery period is deemed ended by the company and 30 days is not nearly sufficient time to investigate and respond to the new allegations.
You have been fired, the first employee in the company’s nearly 400-year history to be fired by a special Third Statute process, and you can’t get a new job because your reputation has been destroyed. You have four small children to support and your financial resources are gone.
What would you do?
You would sue the company in a real court of law where the Constitution and legal precedent protects you. And that it what Gino has done. Discovery is supposed to be completed by the end of this month with a trial scheduled this coming December.
When Francesca reached out to me in June of 2024 for help, I was sufficiently compelled to do so. We have been funding her legal and expert costs since that time, and we will provide whatever resources she needs to clear her name.
The problem with big institutions is that they are massive bureaucracies which occasionally put aside the core principles on which they are founded because they are embarrassed, because they are busy, and/or it is easier to ignore the harm they may cause to the little guy in the interest of protecting their reputation and/or avoiding further distraction to leadership. They also often assume that the little guy will eventually give up and/or run out of resources to keep fighting.
I am going to make sure that doesn’t happen here.
I learned a lot when I went to Harvard. I am applying what I learned at Harvard to protect Harvard from itself and to help someone who needs help because Veritas matters.
If Harvard would like to resolve this outside of a court of law, I am here to help. Sadly, my guess is that Harvard won’t call as they are too proud and pot committed. -
Daniël Lakens
@lakens
Starting the new year with a new podcast episode of Nullius In Verba on 'Scientism' nulliusinverba.podbean.com/e/episode-72-scientismus-i/
In the first of two parts, we ask if it is ok to think that science is the best way to generate knowledge, or if this is too dismissing of 'other ways of knowing'. -
Daniël Lakens
@lakens
Episode 72: Scientismus - I nulliusinverba.podbean.com/e/episode-72-scientismus-i/
-
Daniël Lakens
@lakens
A highlight of my year has breaking through the 6b+ barrier in bouldering, managing some of my first 6c routes. Meeting up with a lovely crowd of friends once or twice a week is such a pleasant way to stay in shape.
-
Daniël Lakens
@lakens
Our Spotify Wrapped tells us that collectively you have listened a total of 181 days to our podcast - and that's just for those of you using Spotify to listen to @nulliusverbapod
Thank you all so much for listening! -
Daniël Lakens
@lakens
There are many dubious papers on correcting for multiple comparisons. And the issue is so simple:
1) You need to control error rates of claims. If multiple tests inflate error rates, you must lower alpha.
2) you adjust on the level of a claim.
lakens.github.io/statistical_inferences/02-errorcontrol.html. -
Daniël Lakens
@lakens
"The replication crisis in Psychology has undermined both intra-scientific and public trust. This review shows–using Wilholt's notion of epistemic trust–that replicability is fundamental for epistemic reliability" #metascience
www.sciencedirect.com/science/article/pii/S2352250X25002635 -
Daniël Lakens
@lakens
It reminds me of the question 'who will replicate the replicators?' that people used to ask during the replication crisis. Turns out the answer is: the replicators themselves, after they have gained a decade of expertise on a topic :)
Daniël Lakens
@lakens
Very happy to have received a replication grant from NWO! We will be reproducing power analyses of large scale replication projects, to learn how to improve power analyses, and increase quality control in large team Science projects! www.openscience.nl/en/news/27-studies-re-examined
-
Daniël Lakens
@lakens
Very happy to have received a replication grant from NWO! We will be reproducing power analyses of large scale replication projects, to learn how to improve power analyses, and increase quality control in large team Science projects! www.openscience.nl/en/news/27-studies-re-examined
-
Daniël Lakens
@lakens
I'm very happy to continue my work at the Human-Technology Interaction group as a Full Professor of Metascience.
Working at this department has been a great source of joy over the last 15 years. -
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog post: Dogmatic Bayesianism Disorder
daniellakens.blogspot.com/2025/12/dogmatic-bayesianism-di… -
Daniël Lakens reposted
Ash Paul
@pash22
Dogmatic Bayesianism Disorder: This humorously intended sarcastic blog post by @lakens directly mimics classifications of psychological disorders in the Diagnostic and Statistical Manual of the American Psychological Association, but Dogmatic Bayesianism is not in the DSM-5 as an actual psychological disorder yet @awaisaftab
daniellakens.blogspot.com/2025/12/dogmatic-bayesianism-di… -
Daniël Lakens
@lakens
In my paper Sample Size Justification I discuss 6 approaches to justifying sample sizes. The focus of the paper is to teach people the tools they need to be honest when they explain where their sample size comes from. I know, so radical ;) online.ucpress.edu/collabra/article/8/1/33267/120491/Samp…
-
Daniël Lakens reposted
David Bessis
@davidbessis
This viral post by @paulg was based on a visual by @cremieuxrecueil. Taken at face value, it seemed to prove that IQ is almost entirely determined by genes. But when you look at the actual sources, you realize that this conclusion is fundamentally wrong.⤵️
-
Daniël Lakens reposted
David Bessis
@davidbessis
The first crazy thing: the yellow dataset (Burt) is known to be a fabrication. I have no clue why @cremieuxrecueil decided to include it, as the Burt fraud is one of the biggest and most documented scandal in the history of science.
en.wikipedia.org/wiki/Cyril_Burt -
Daniël Lakens
@lakens
New blog post: Dogmatic Bayesianism Disorder
daniellakens.blogspot.com/2025/12/dogmatic-bayesianism-di… -
Daniël Lakens
@lakens
In 2011 Bayesians argued science would improve if we switch from frequentist to Bayesian statistics.
In 2025 we have realized science improves if we have scientists with smarter ideas about improving science than proposing to switch from Bayesian to frequentist statistics. -
Daniël Lakens
@lakens
3 Bayesian lies, from a preprint abstract: "Unlike p-values, Bayes factors allow for quantifying support both for and against the existence of an effect, facilitate ongoing evidence monitoring, and maintain coherent long-run behavior as additional studies are incorporated." 1/2
Daniël Lakens
@lakens
P-values do each of these things *better* than Bayes factors, through equivalence testing, sequential analysis, and meta-analysis. Better, because you get error control, so you do not make false claims too often. 2/2
-
Daniël Lakens reposted
Thomas Basbøll
@Inframethod
PS. This also resonates nicely with my approach. A conviction is merely our incentive to find out if we are wrong.
www.weizmann.ac.il/mcb/alon/sites/mcb.UriAlon/files/uploa… -
Daniël Lakens reposted
Thomas Basbøll
@Inframethod
-
Daniël Lakens reposted
Thomas Basbøll
@Inframethod
Thanks @lakens and @smreeteemehta! I had not read Medawar's paper. Great stuff. I have to say I teach business school students to treat their theories as systems of expectation that frame their hypotheses. (Link in next post.)
Nullius In Verba
@nulliusverbapod
New episode on P. Medawar's paper "Is Scientific Paper a Fraud?" is out now!
nulliusinverba.podbean.com/e/scientific-paper-fraud/ -
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
New episode on P. Medawar's paper "Is Scientific Paper a Fraud?" is out now!
nulliusinverba.podbean.com/e/scientific-paper-fraud/ -
Daniël Lakens
@lakens
Metacheck can now automatically 1) download a preprint from PsyArXiv, 2) create a report of the checks (stats, reporting guidelines, references, code, etc), 3) retrieve the authors email, and 4) send them the report.
Would you as an author appreciate this? When yes, when no? -
Daniël Lakens reposted
Daniël Lakens
@lakens
New episode of Nullius In Verba! We discuss Peter Medawar's piece 'Is the scientific paper a fraud?' nulliusinverba.podbean.com/e/scientific-paper-fraud/
-
Daniël Lakens
@lakens
New episode of Nullius In Verba! We discuss Peter Medawar's piece 'Is the scientific paper a fraud?' nulliusinverba.podbean.com/e/scientific-paper-fraud/
-
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
When it comes to reliable means of identifying what’s true, science is…
-
Daniël Lakens
@lakens
Episode 71: Commentarius Scientificus: Fraus? www.podbean.com/site/error?msg=user-site-page-not-found-r…
-
Daniël Lakens
@lakens
We had a very interesting visit and talk from Anna-Marie Ortloff at the lab yesterday, talking about how researchers in HCI and at CHI interpret effect sizes. Long story short: A lot of room and need for improvement. You can find her papers on this at: scholar.google.com/citations?user=J-pyJyIAAAAJ&hl=en&inst…
-
Daniël Lakens
@lakens
Many Gen AI tools are free now. If you do not train yourself (e.g. you never learn how to program without them) you will be dependent on them. In the future they will increase their prices drastically, and you will have to pay, because you can't do without them anymore.
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Last chance to register! 🚨 In just a few days, registration for our Quarto workshop closes. Learn how to write reproducible manuscripts more efficiently and professionally with @ambra_prg
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for our Quarto workshop at Paul Meehl Graduate School!
Join us on December 12 in Eindhoven to learn how to use Quarto for reproducible research and communication.
More details and registration here:
paulmeehlschool.github.io/2025-11-03-quarto/ -
Daniël Lakens
@lakens
For many social dilemma's in Science (e.g. the slow uptake of diamond open access journals) stronger top down management is necessary. It won't just happen. If scientists will not create this management themselves, someone is going to create it for us.
-
Daniël Lakens
@lakens
New prologue episode of Nullius In Verba. Smriti reads Medewar's 'Is the scientific paper a fraud?'. On induction, deduction, and where hypotheses come from.
nulliusinverba.podbean.com/e/ep71-prologue/
We will discuss the paper (originally a talk) in our episode next week! -
Daniël Lakens
@lakens
Prologus 71: Is the Scientific Paper A Fraud (P. Medawar) nulliusinverba.podbean.com/e/ep71-prologue/
-
Daniël Lakens
@lakens
Interfolio, made by El$evier, is the crappiest service around to get recommendation letters in science. Every university that uses it should abandon it as quickly as possible.
-
Daniël Lakens
@lakens
It was great fun to teach a full day course on interpreting statistics to the Dutch Behavioural Insights Netwerk, a collaboration between all Dutch ministries for the application of behavioural knowledge within the Dutch government. They are doing lots of interesting research!
-
Daniël Lakens reposted
Matthew B. Jané
@MatthewBJane
Giving a talk tomorrow at noon. Link to the zoom is below.
-
Daniël Lakens
@lakens
Beautiful location today to talk to The Netherlands Epidemiological Society (VvE) about open science.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog by Data Colada on the ResearchBox platform, which has excellent improvements (I love the Zenodo back-up!). I will focus more on the platform in my educational material. And we should be very grateful to everyone who makes data sharing platforms!
datacolada.org/130 -
Daniël Lakens
@lakens
I have seen the same people criticize preregistration, based on highly unlikely hypotheses about why they would not be needed, or even counterproductive. Literally *none* of these critics have provided any empirical support for their criticism in the last decade. I wonder why.
-
Daniël Lakens
@lakens
New blog by Data Colada on the ResearchBox platform, which has excellent improvements (I love the Zenodo back-up!). I will focus more on the platform in my educational material. And we should be very grateful to everyone who makes data sharing platforms!
datacolada.org/130 -
Daniël Lakens reposted
Ash Paul
@pash22
How much of our scientific knowledge depends on knowledge that we can't communicate directly? How can we replicate studies, if they might rely on tacit knowledge? And why has the concept itself not been made more explicit in the last 45 years?
nulliusinverba.podbean.com/e/episode-70-scientia-tacita/ via @lakens -
Daniël Lakens
@lakens
New Nullius In Verba episode on 'Tacit Knowledge' nulliusinverba.podbean.com/e/episode-70-scientia-tacita/
How much in science rests on tacit knowledge, and what does this means for replication studies? Or should the idea that tacit knowledge exist be abandoned? -
Daniël Lakens
@lakens
Episode 70: Scientia Tacita nulliusinverba.podbean.com/e/episode-70-scientia-tacita/
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for our Quarto workshop at Paul Meehl Graduate School!
Join us on December 12 in Eindhoven to learn how to use Quarto for reproducible research and communication.
More details and registration here:
paulmeehlschool.github.io/2025-11-03-quarto/ -
Daniël Lakens
@lakens
A recent paper suggests we stop using the terms Registered Report and preregistration. This is a very bad idea by my fellow metascientists. There is no way I am ever gonna drop 'Registered Reports' for 'Two-stage review with in principle acceptance'
journals.sagepub.com/doi/10.1177/25152459251375445 -
Daniël Lakens
@lakens
The null is never true, except when it is true.
Raffaele Piccolo
@piccoloraf
No benefit from beta-blockers in patients with recent myocardial infarction and LVEF equal or greater than 50%. #aha2025
IPD meta-analysis of 5 RCTs including 17,801 patients (REBOOT, REDUCE-AMI, BETAMI, DANBLOCK, and CAPITAL-RCT).
@Borjaibanez1 @dan_atar @RosselloXavier @evaprescott @drjohnm @mmamas1973 @djc795 @mirvatalasnag @EAPCIPresident @SABOURETCardio @Hragy @kaulcsmc @EricSecemskyMD @venkmurthy @DrMarthaGulati @SripalBangalore @sbrugaletta @DLBHATTMD @EricTopol @CMichaelGibson @DFCapodanno @nadig_cardio @Ortega_Paz @HadyLichaaMD @ACCinTouch @Medscape @AHAScience
Published on @NEJM
www.nejm.org/doi/full/10.1056/NEJMoa2512686?query=feature… -
Daniël Lakens
@lakens
My Shiny app containing 3530 Open Science blog posts discussing the replication crisis is updated - you can now use the SEARCH box. I fixed it as my new PhD Julia wanted to know who had called open scientists 'Methodological Terrorists' :) shiny.ieis.tue.nl/open_science_blogs/
-
Daniël Lakens reposted
Sense Jan vd Molen
@sensemolen
Maar Ellemers had de statistiek verkeerd gedaan, waardoor haar resultaat achteraf niet hard te maken bleek. Casper Albers liet dat zien.
Het deed er verder niet toe, want de boodschap was aangekomen en het nieuwe beleid was al ingezet. -
Daniël Lakens reposted
Sense Jan vd Molen
@sensemolen
Met die uitkomsten is bij mijn weten niets gedaan. Na lang aandringen mochten we wel praten met NWO, maar het bleef bij een goed gesprek.
Toen prof Ellemers jaren geleden (onterecht) beweerde dat mannen meer kans hadden op Veni’s, kwamen er meteen Kamervragen en maatregelen. -
Daniël Lakens reposted
Sense Jan vd Molen
@sensemolen
Ik blijf me verbazen over de selectiviteit bij universiteiten en NWO. Twente schaft Cum Laude af na een artikel van Thijs Bol dat zegt dat mannen vaker CL krijgen dan vrouwen. Maar een eerder artikel van Thijs (& mij) liet zien dat vrouwen vaker een Veni-beurs krijgen dan mannen.
-
Daniël Lakens reposted
Collabra: Psychology
@CollabraOA
What can Christmas tell us about the role of environment in shaping people’s implicit biases? doi.org/10.1525/collabra.146223
-
Daniël Lakens reposted
The Dark Fiddling Pirate Jussim
@PsychRabble
Totally agree. Nothing wrong with small studies if they are interpreted appropriately, usually meaning as including great uncertainty, NOT establishing some new discovery, and as preliminary requiring confirmation by larger studies conducted by independent teams, preferably via the Registered Report format, to limit publication biases.
Daniël Lakens
@lakens
New 'Preliminary Report' submission format at Science and Medicine in Football. An idea that should be interesting for other journals! Explicitly make space for honestly reported smaller sample-size studies www.tandfonline.com/doi/full/10.1080/24733938.2025.2580319
-
Daniël Lakens reposted
Nick DeVito
@NDevito1
This sounds really interesting.
Daniël Lakens
@lakens
New 'Preliminary Report' submission format at Science and Medicine in Football. An idea that should be interesting for other journals! Explicitly make space for honestly reported smaller sample-size studies www.tandfonline.com/doi/full/10.1080/24733938.2025.2580319
-
Daniël Lakens reposted
抖阴短剧
@dyaiduanju
wish SLA and applied linguistics journals would follow suit.
Daniël Lakens
@lakens
New 'Preliminary Report' submission format at Science and Medicine in Football. An idea that should be interesting for other journals! Explicitly make space for honestly reported smaller sample-size studies www.tandfonline.com/doi/full/10.1080/24733938.2025.2580319
-
Daniël Lakens reposted
Witold Więcek 🇺🇦
@vientsek
No, look at *this* distribution of z-values from medical research! (329,601 z-values from Cochrane database)
John B. Holbein
@JohnHolbein1
Look at the distribution of z-values from medical research!
-
Daniël Lakens
@lakens
New 'Preliminary Report' submission format at Science and Medicine in Football. An idea that should be interesting for other journals! Explicitly make space for honestly reported smaller sample-size studies www.tandfonline.com/doi/full/10.1080/24733938.2025.2580319
-
Daniël Lakens reposted
Franco Impellizzeri
@francoimpell
Happy to announce @SciMed_Football’s new 𝗣𝗿𝗲𝗹𝗶𝗺𝗶𝗻𝗮𝗿𝘆 𝗥𝗲𝗽𝗼𝗿𝘁 category for small-sample studies, a pragmatic step to address the spread and misreporting of underpowered football research. Grateful to all co-authors for months shaping the submission requirements.
Sci & Med Football
@SciMed_Football
🚨‼️ New Submission Category‼️🚨
Introducing… ✨Preliminary Reports✨— a new submission category for small-sample experimental studies encouraging focused aims, transparent reporting, and responsible interpretation.
Read more here👇
www.tandfonline.com/doi/full/10.1080/24733938.2025.258031… -
Daniël Lakens reposted
Daniël Lakens
@lakens
It is because you want this to be true. Whenever you see something you want to be true, time to turn on your internal critic.
-
Daniël Lakens
@lakens
This is not an accurate picture of how biased the literature is. The authors only analyze p-values in abstracts. If scientists say 'not significant' without stating p for p >. 05, you get this graph with 0 bias.
John B. Holbein
@JohnHolbein1
Look at the distribution of z-values from medical research!
Post not available here, view on X
Daniël Lakens
@lakens
It is because you want this to be true. Whenever you see something you want to be true, time to turn on your internal critic.
-
Daniël Lakens
@lakens
Went on a trip to Paris for the day, giving a talk at Université Paris Cité, saying hi to former lab visitor Ethan Meimoun, and we even had time to pick up gluten free goodies for my wife at Copains on the way!
-
Daniël Lakens reposted
Psicostat
@psicostat
🎉 Il 23 gennaio festeggiamo 10 anni di Psicostat con talk, confronto e aperitivo offerto!
📣 Aperta la call Minitalk Early Careers per studentə, dottorandə e giovani ricercatorə
🔗 Partecipazione gratuita (anche come uditori) con iscrizione obbligatoria: docs.google.com/forms/u/4/d/e/1FAIpQLSdCAuVOjNo9qXaJHD27A… -
Daniël Lakens reposted
Daniël Lakens
@lakens
Part 2 of our episode on p-hacking is out! Tune in to the latest Nullius In Verba episode here:
nulliusinverba.podbean.com/e/p-hacking-ii/ -
Daniël Lakens reposted
Psicostat
@psicostat
🚀 Kicking off the upcoming meetings with a fresh round of online talks! First up: Gabriela Xavier-Quintais ♻️ Join us next Friday, November 7th at 12pm (CEST), for "Metascience in Ecology: the role of statistical tests". Don’t miss it! 🔍📊
-
Daniël Lakens
@lakens
Part 2 of our episode on p-hacking is out! Tune in to the latest Nullius In Verba episode here:
nulliusinverba.podbean.com/e/p-hacking-ii/ -
Daniël Lakens
@lakens
Episode 69: Fraus P-Valoris - II nulliusinverba.podbean.com/e/p-hacking-ii/
-
Daniël Lakens reposted
Ash Paul
@pash22
Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology
daniellakens.blogspot.com/2025/10/why-we-should-stop-usin… via @lakens -
Daniël Lakens reposted
Daniël Lakens
@lakens
New paper finds that selective reporting remains the most replicable finding in science: journals.sagepub.com/doi/full/10.1177/07419325251361907 I especially like their new exploratory metric 'p-values per participant'. Some papers had 11 p-values per participant! 🤯
-
Daniël Lakens
@lakens
New paper finds that selective reporting remains the most replicable finding in science: journals.sagepub.com/doi/full/10.1177/07419325251361907 I especially like their new exploratory metric 'p-values per participant'. Some papers had 11 p-values per participant! 🤯
-
Daniël Lakens
@lakens
Exciting new addition to the AsPredicted and ResearchBox research infrastructure toolbox: AsCollected ascollected.org A platform to document results provenance - where did data come from, who collected it, and who cleaned and analyzed it. So important!
-
Daniël Lakens reposted
Constantin Späth @cspaeth.bsky.social
@ConstantinSpath
New Commentary published in the German Journal for Sport and Exercise Research, emphasizing not only the need to specify a SESOI when testing a claim, but recommending also to start a discussion about best practices to determine a SESOI
link.springer.com/article/10.1007/s12662-025-01072-7 -
Daniël Lakens reposted
Jonathan Brauer
@RealJonBrauer
Good read.
Daniël Lakens
@lakens
New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-we-should-stop-usin… where I reflect on how we should check the quality of novel statistical techniques.
-
Daniël Lakens
@lakens
Are you interested in thinking about which studies are worth replicating? Then you have 10 articles to dig into in Meta-Psychology, representing a very wide range of viewpoint on this topic, out now: open.lnu.se/index.php/metapsychology/issue/view/297
-
Daniël Lakens reposted
Peder M Isager
@peder_isager
This paper is our first stab at solving a very complex problem. Such early attempts benefit enormously from being critiqued from the get-go, and we were really lucky to receive several excellent commentaries from many experts in this field.
-
Daniël Lakens reposted
Peder M Isager
@peder_isager
This paper forms one of Meta-Psychology’s Special Topics. Eight commentaries have been published alongside the paper which criticizes and extends the ideas presented within. We have written a response to these commentaries here open.lnu.se/index.php/metapsychology/article/view/4738.
-
Daniël Lakens reposted
Peder M Isager
@peder_isager
My paper with @lakens and Anna van 't Veer - “Replication value as a function of citation impact and sample size” - is published in @Meta_Psy! open.lnu.se/index.php/metapsychology/article/view/3300
-
Daniël Lakens reposted
Jesper N. Wulff
@jesper_wulff
So many good points here. Highly recommend reading 👇
Daniël Lakens
@lakens
New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-we-should-stop-usin… where I reflect on how we should check the quality of novel statistical techniques.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-we-should-stop-usin… where I reflect on how we should check the quality of novel statistical techniques.
-
Daniël Lakens
@lakens
New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-we-should-stop-usin… where I reflect on how we should check the quality of novel statistical techniques.
Daniël Lakens
@lakens
I reflect on how dangerous authority arguments in science are, and what actual quality control would look like, such as a database where we track how popular techniques are, and which quality standards have been checked.
-
Daniël Lakens reposted
František Bartoš
@BartosFra
Simulation studies have a conflict of interest problem. The same team:
- develops a new method
- designs a simulation study to evaluate it
However, the new method has to show good performance to get published.
We propose living synthetic benchmarks to address the issue. -
Daniël Lakens
@lakens
Nature Psychology Review is a journal created by a commercial publisher because it is easy profit. Review articles are easy to write, get cited a lot. Slap Nature on it and you get a high IF.
It should flip to a diamond open access journal. Submit your review articles elsewhere. -
Daniël Lakens reposted
Daniël Lakens
@lakens
We got our first review stating that as p-curve was recently criticized, and z-curve was also not 'vetted by a real statistician', the method should not be trusted.
No statistical tool could ever damage science as much as this horrendously flawed authority argument. -
Daniël Lakens
@lakens
We got our first review stating that as p-curve was recently criticized, and z-curve was also not 'vetted by a real statistician', the method should not be trusted.
No statistical tool could ever damage science as much as this horrendously flawed authority argument. -
Daniël Lakens reposted
Daniël Lakens
@lakens
I am writing a paper "Why small samples are all you need, small effects are really important, you do not need to correct for multiple comparisons, and invalid measures are useful'. But because I really believe it, and really not just for the attention and citations, I swear. 🙄
-
Daniël Lakens
@lakens
I am writing a paper "Why small samples are all you need, small effects are really important, you do not need to correct for multiple comparisons, and invalid measures are useful'. But because I really believe it, and really not just for the attention and citations, I swear. 🙄
-
Daniël Lakens reposted
Hsin-Yuan Chen
@ThomasChenUoN
Dr Daniël Lakens presents 'When & How to Deviate From a Preregistration.' Essential for research integrity. Register: www.eventbrite.co.uk/e/when-and-how-to-deviate-from-a-pre…
#preregistration #openscience #datascience #statistics #academicresearch @lakens -
Daniël Lakens reposted
ReproducibiliTea
@ReproducibiliT
UCL/UNIL - Topic: When and How to Deviate From a Preregistration, with Prof Daniël Lakens
Tuesday, 28 October at 2 pm CET
Registration:
www.eventbrite.co.uk/e/when-and-how-to-deviate-from-a-pre… -
Daniël Lakens
@lakens
End of the Paul Meehl Graduate School Symposium, with wonderful chats over dinner. Last year was great, this year even better! So nice to talk to this crowd of metascientists.
Thanks to @sajedeh_rasti for the organization of my favorite metascience event of the year! -
Daniël Lakens
@lakens
Now Rasoul Norouzi from Tilburg University, talking about reproducibly automatically extracting causal statements from the scientific literature.
-
Daniël Lakens
@lakens
Very cool: Lisa Spitzer has made a deviation from preregistrations template, which you can find here: apps.leibniz-psychology.org/prp-dev/
-
Daniël Lakens
@lakens
Second keynote of the day, Lisa Spitzer talking about how to foster preregistration, with a lot of insights into how it was to be a metascience PhD researcher!
-
Daniël Lakens
@lakens
Cas Goosen is presenting the idea of multi-stage preregistration, a way to improve the evaluations of the severity of deviations (which are common).
-
Daniël Lakens
@lakens
The first keynote of the day at the Paul Meehl Graduate School symposium by Duygu Uygun-Tunç introducing a philosophical perspective on metascience, asking appropriately big questions 😅
-
Daniël Lakens
@lakens
Now Gabriela Xavier Quintais talking about at the Paul Meehl Graduate School symposium whether the use of hypothesis testing is a good fit for ecology. Can error rates for studies in this field be controlled, is replication possible, can researchers randomize to conditions?
-
Daniël Lakens
@lakens
Now Johanna Einsiedler who is reflecting on which aspects of systematic reviews we can automate, how we quantify uncertainty, and keep the human in the loop, at the Paul Meehl Graduate School Symposium.
-
Daniël Lakens
@lakens
Now Max Primbs talking about his super exciting Many-Dags project, asking experts in his field of implicit researchers to draw DAGs about a theory in his field.
-
Daniël Lakens
@lakens
Everything ready and set up for the Paul Meehl Graduate Symposium. Looking So forward to today, with lots of great presentations and lovely visitors! paulmeehlschool.github.io/2025-08-17-programsymposium2025/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
In an incredibly fun, interesting, and educational Paul Meehl Graduate School workshop by Aaron Peikert on collaboration through Github.
By far the best educational material I have seen in any Git course. Feels like a game, played on GitHub! 🕹️ -
Daniël Lakens reposted
Dr. David Nolan
@DrDavidNolan
It is ironic how confidently this incorrect explanation of confidence intervals in presented.
Anyone looking to brush up on stats @lakens has some fantastic free courses available on these exact concepts. -
Daniël Lakens
@lakens
In an incredibly fun, interesting, and educational Paul Meehl Graduate School workshop by Aaron Peikert on collaboration through Github.
By far the best educational material I have seen in any Git course. Feels like a game, played on GitHub! 🕹️ -
Daniël Lakens reposted
Charlotte Pennington
@drcpennington
Do you have an interest in social cognition and skills in neuroscience? Come and do a fully-funded(!) PhD with Dr Daniel Shaw, Dr Enrico Amico, and me! 🧠 📚
Find out more and apply here: warwick.ac.uk/fac/cross_fac/mibtp/phd/supervisors/DJShaw/
#phdstudentship #fundedphd #phdjobs #findaphd -
Daniël Lakens reposted
Daniël Lakens
@lakens
Yesterday with the META/e group at Eindhoven we discussed the draft of the new Dutch Code for Research Integrity: www.nwo-i.nl/en/news/2025/09/28/denk-mee-over-wetenschapp…
It is great to see how much transparency and FAIR data and code will become the default expectation to adhere to research integrity. -
Daniël Lakens
@lakens
Yesterday with the META/e group at Eindhoven we discussed the draft of the new Dutch Code for Research Integrity: www.nwo-i.nl/en/news/2025/09/28/denk-mee-over-wetenschapp…
It is great to see how much transparency and FAIR data and code will become the default expectation to adhere to research integrity. -
Daniël Lakens
@lakens
Call For Papers: Journal of Business Economics – Special Issue – Behavioral Ethics Under Review: Replications and Theoretical Foundations
Call for Regular Articles & Registered Reports
sbeonline.org/2025/10/call-for-papers-journal-of-business… -
Daniël Lakens
@lakens
"The Case for Expanding the Domain of Registered Reports: Confronting Academic Dishonesty and Declining Confidence in Science" by Robert Andersen doi.org/10.1111/1748-8583.70019
-
Daniël Lakens
@lakens
It is interesting how LinkedIn has become the academic networking site. After giving a talk in a new community, you used to get new followers in places like this. But now, you only get new connection requests on LinkedIn. Different times!
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New Nullius In Verba episode on p-hacking (Part 1 of 2). We discuss why it is useful for a single term to describe all practices that untransparently inflate error rates, and what these practices are.
nulliusinverba.podbean.com/e/p-hacking-i/ -
Daniël Lakens
@lakens
New Nullius In Verba episode on p-hacking (Part 1 of 2). We discuss why it is useful for a single term to describe all practices that untransparently inflate error rates, and what these practices are.
nulliusinverba.podbean.com/e/p-hacking-i/ -
Daniël Lakens
@lakens
Did I need to by a golden vinyl single of 'Golden' by Huntrix?
No.
Did I buy it anyway?
Yes, yes I did. -
Daniël Lakens
@lakens
Episode 68: Fraus P-Valoris - I nulliusinverba.podbean.com/e/p-hacking-i/
-
Daniël Lakens reposted
AJ Thurston, PhD
@AJThurston
When you really think about it, it's almost miraculous that we have this standard. I wonder if the modern equivalents of Fisher, Neyman, Pearson could compromise as they did? I guess Cattel-Horn-Carrol would be a good modern example in the affirmative.
-
Daniël Lakens
@lakens
Tomorrow I will be in Nijmegen to talk at the VVSOR meeting on when and why we make dichotomous claims based on statistical tests at a meeting about philosophy in statistics education. www.vvsor.nl/education/events/thinking-in-grey-statistica…
I always a joy explaining why p<.05 is such an important tool. -
Daniël Lakens reposted
Daniël Lakens
@lakens
-
Daniël Lakens
@lakens
-
Daniël Lakens reposted
Christian Magelssen
@magelssen_chr
I was constantly faced with that argument at my Uni (not UofG). My response was: If they do poor research with errors, it takes long time for the literature to correct itself. So it better and more effective to do research properly. They didn’t buy that argument
-
Daniël Lakens
@lakens
People sometimes say they do not have time to do open science. But they can use the time they would have needed 40 years ago to send and request paper copies of articles to read the literature.
Somehow we never acknowledge where we gained time that we can invest in open science. -
Daniël Lakens
@lakens
People kept buying incandescent light bulbs, even though LED lights were better for the environment and cheaper in the long run. But incandescent light were cheap in the short term. In the end, the EU had to ban incandescent lights.
The adoption of Open Science works same. -
Daniël Lakens
@lakens
I have updated the blog post with a long reply y Andrew Gelman, in which he clarifies he still recommends the regular use of Type S and M errors. daniellakens.blogspot.com/2025/09/type-s-and-m-errors-as-… This makes our preprint criticising Type S and M errors all the more important.
Daniël Lakens
@lakens
New blog post on Gelman's recent claim that Type S and M errors are intended as a 'rhetorical tool', and if I was wrong to believe they were recommended more routinely in our recent preprint criticizing the idea of Type S and M errors. daniellakens.blogspot.com/2025/09/type-s-and-m-errors-as-…
-
Daniël Lakens
@lakens
I am really looking forward to having so many excellent meta-scientists over for the Paul Meehl Graduate School symposium. It looks like this year will be even more fun than last, with interesting talks and lots of opportunity to network.
Sajedeh Rasti
@sajedeh_rasti
Today is the last day to sign up for the upcoming workshops (on the 16th) and the symposium (on the 17th) of Paul Meehl Graduate School. Don't miss it.
Sajedeh Rasti
@sajedeh_rasti
Time is running out to register for our pre-symposium workshop and Meta Research Symposium event at Paul Meehl Graduate School
Check the information and registration links below:
paulmeehlschool.github.io/workshops/ -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Today is the last day to sign up for the upcoming workshops (on the 16th) and the symposium (on the 17th) of Paul Meehl Graduate School. Don't miss it.
Sajedeh Rasti
@sajedeh_rasti
Time is running out to register for our pre-symposium workshop and Meta Research Symposium event at Paul Meehl Graduate School
Check the information and registration links below:
paulmeehlschool.github.io/workshops/ -
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog post on Gelman's recent claim that Type S and M errors are intended as a 'rhetorical tool', and if I was wrong to believe they were recommended more routinely in our recent preprint criticizing the idea of Type S and M errors. daniellakens.blogspot.com/2025/09/type-s-and-m-errors-as-…
-
Daniël Lakens
@lakens
Episode 67: Investigatio Inhonesta nulliusinverba.podbean.com/e/unethical-research/
-
Daniël Lakens
@lakens
New blog post on Gelman's recent claim that Type S and M errors are intended as a 'rhetorical tool', and if I was wrong to believe they were recommended more routinely in our recent preprint criticizing the idea of Type S and M errors. daniellakens.blogspot.com/2025/09/type-s-and-m-errors-as-…
-
Daniël Lakens reposted
Peder M Isager
@peder_isager
Extremely honored to recieve @oslonyehoyskole's science award for 2025. ONH has been a fantastic base to conduct my research at for the past 4 years, and I have an amazing team of colleagues around me to thank for that. From the bottom of my heart, thank you all!
-
Daniël Lakens
@lakens
Position part B of the image on top of part A in such a way that the 2 riders are sitting on top of the horses, right side up. From Scheerer, Goldstein, and Boring, 1941 doi.org/10.2307/1417695
(answer in the reply)
Daniël Lakens
@lakens
-
Daniël Lakens
@lakens
New Grandbrothers album is out! For anyone who likes the opening tune of our Nullius in Verba podcast, give them a listen, or even better, just get a copy because the record is great!
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Gelman all of a sudden calling Type S and M errors 'rethorical devices' after we pointed out how flawed his ideas were osf.io/preprints/psyarxiv/2phzb_v1 reminds me too much of Daryl Bem saying data was a rethorical device to support pre-cognition.
Andrew Gelman et al.
@StatModeling
“On the poor statistical properties of the P-curve meta-analytic procedure”
statmodeling.stat.columbia.edu/2025/09/25/on-the-poor-sta… -
Daniël Lakens
@lakens
You have not seen second generation p-values used anywhere because they were badly reinvented equivalence tests, and people should use equivalence tests instead. See open.lnu.se/index.php/metapsychology/article/view/933
ᴅʀ ᴍɪʀᴄᴇᴀ ᴢʟᴏᴛᴇᴀɴᴜ 🍂 🍃
@mzloteanu
#statstab #423 Second-generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses
Thoughts: I appreciated the thought behind sgpv but I've not seen them used anywhere.
#sgpv #pvalue #error #bias #inference
journals.plos.org/plosone/article?id=10.1371%2Fjournal.po… -
Daniël Lakens
@lakens
Gelman all of a sudden calling Type S and M errors 'rethorical devices' after we pointed out how flawed his ideas were osf.io/preprints/psyarxiv/2phzb_v1 reminds me too much of Daryl Bem saying data was a rethorical device to support pre-cognition.
Andrew Gelman et al.
@StatModeling
“On the poor statistical properties of the P-curve meta-analytic procedure”
statmodeling.stat.columbia.edu/2025/09/25/on-the-poor-sta… -
Daniël Lakens reposted
Rolf Zwaan 🦢
@RolfZwaan
New post!
-
Daniël Lakens reposted
Ethan
@SEthanMilne
For anyone coming to ACR:
I will be presenting some conceptual work on "prosocial consumer behavior". Preview: we review existing literature and survey ~100 researchers and find that there is no consensus definition for this construct.
Come say hi! -
Daniël Lakens reposted
Daniël Lakens
@lakens
Not having to deviate from your preregistration is how you impress peers. You demonstrate you know something, because knowledge allows you to make good predictions. The more deviations, the less impressive the tests in a study are.
-
Daniël Lakens
@lakens
This is also why people are currently not transparent about deviations from the preregistration in their paper. It makes their tests look less impressive. As it should.
Psychology has a history of valuing narratives more than good tests. Hopefully this will change. -
Daniël Lakens
@lakens
Not having to deviate from your preregistration is how you impress peers. You demonstrate you know something, because knowledge allows you to make good predictions. The more deviations, the less impressive the tests in a study are.
-
Daniël Lakens
@lakens
New blog post from Data Colada, responding to the recent criticisms on p-curve analysis. It is a *very* good response. As in, it addresses exactly the points I would have expected in a reply, and it explains why I will still teach p-curve analysis. datacolada.org/129
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Like too much of the advice by @f2harrell on the foundations of statistics, this should also be ignored. Second generation p-values should never be used, as we explained years ago open.lnu.se/index.php/metapsychology/article/view/933
Frank Harrell
@f2harrell
It’s interesting that abandoning p-values and false discovery rates for 2nd generation p-values results in major reductions in decision errors: journals.plos.org/plosone/article?id=10.1371%2Fjournal.po…
-
Daniël Lakens reposted
Dong Nguyen
@DongNguyeb
I think Harrell is strongly biased regarding P-values and hypothesis testing, so we should ignore his advice.
Returning to his shared article,
IMO, I believe that the so-called second-generation P-value (SGPV) actually pursues a different goal from Fisher’s original P-value.
Fisher’s P-value reflects how unusual the observed data are if the null hypothesis (Ho) were hold.
In other words, P-value measures the compatibility of the data with the randomization distribution of the test statistic under Ho.
In contrast, the SGPV pursue on a different definition of Ho, now it is a practical null interval effects that is so small as to be clinically unimportant.
In this new framework, testing the Ho use P- value quantifies the degree to which the data are compatible with a null interval of practical point.
Two above things are totally different in purpose. -
Daniël Lakens
@lakens
Like too much of the advice by @f2harrell on the foundations of statistics, this should also be ignored. Second generation p-values should never be used, as we explained years ago open.lnu.se/index.php/metapsychology/article/view/933
Frank Harrell
@f2harrell
It’s interesting that abandoning p-values and false discovery rates for 2nd generation p-values results in major reductions in decision errors: journals.plos.org/plosone/article?id=10.1371%2Fjournal.po…
-
Daniël Lakens
@lakens
The work on 'tacit knowledge' in the STS literature is of such low quality, I was amazed when I read through classic work like Collins, 1985. Maybe it is because STS scholars lack knowledge of psychology, or the general low quality in the field. But the concept should not exist.
Daniël Lakens
@lakens
If internal criticism in the STS literature would exist, the concept would have been pointed out as the mistake it is, and abandoned decades ago. But because the STS field lacks any form of internal criticism, bad ideas live on forever.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New episode of Nullius In Verba! www.podbean.com/ep/pb-8zza5-193b78d We discuss controversy in psychology, and explain why new groups seem to need to unfairly attack older groups (from structuralists to metascientists) to (try to) move science forward.
-
Daniël Lakens
@lakens
New episode of Nullius In Verba! www.podbean.com/ep/pb-8zza5-193b78d We discuss controversy in psychology, and explain why new groups seem to need to unfairly attack older groups (from structuralists to metascientists) to (try to) move science forward.
-
Daniël Lakens
@lakens
Episode 66: Psychologia Controversiae nulliusinverba.podbean.com/e/episode-66-psychologia-contr…
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Time is running out to register for our pre-symposium workshop and Meta Research Symposium event at Paul Meehl Graduate School
Check the information and registration links below:
paulmeehlschool.github.io/workshops/ -
Daniël Lakens
@lakens
Made my own hangboard to practice my bouldering muscles!
-
Daniël Lakens
@lakens
New prologue episode of Nullius In Verba! We read Boring's paper on The Psychology of Controversy nulliusinverba.podbean.com/e/prologus-66-the-psychology-o… Written in 1929, but relevant as if he based in on scientists arguing on social media for the last decade. A must read - or, you can just listen to it!
-
Daniël Lakens
@lakens
Prologus 66: The Psychology of Controversy (E. G. Boring) nulliusinverba.podbean.com/e/prologus-66-the-psychology-o…
-
Daniël Lakens
@lakens
New office furniture. :)
-
Daniël Lakens
@lakens
First day of teaching, starting with our Metascience course. 73 master students (up from 42 last year) is a lot, but hopefully also a lot of fun! Today we discussed what metascience is, and summarize the concerns metascientists have raised in the last decade.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Part 2 of our episode of Nullius In Verba asking 'What is metascience' is out now! Enjoy :)
nulliusinverba.podbean.com/e/metascience-ii/ -
Daniël Lakens
@lakens
Very happy with this pristine Sons of Kemet record (2017 pressing). I kind of liked them, but got hooked after watching Transcending Dimensions at the International Film Festival Rotterdam (for a trailer, see www.youtube.com/watch?v=2aSFZcWZ6AI)
-
Daniël Lakens
@lakens
Episode 65: Scientia de Scientia - II nulliusinverba.podbean.com/e/metascience-ii/
-
Daniël Lakens
@lakens
Part 2 of our episode of Nullius In Verba asking 'What is metascience' is out now! Enjoy :)
nulliusinverba.podbean.com/e/metascience-ii/ -
Daniël Lakens reposted
Daniël Lakens
@lakens
If you are preparing your bachelor statistics course and would like to add optional material for students to better understand statistics on a conceptual level (see topics in the screenshot) my free textbook provides a state of the art overview. lakens.github.io/statistical_inferences/
-
Daniël Lakens
@lakens
If you are preparing your bachelor statistics course and would like to add optional material for students to better understand statistics on a conceptual level (see topics in the screenshot) my free textbook provides a state of the art overview. lakens.github.io/statistical_inferences/
-
Daniël Lakens
@lakens
Some of you noted you could not access our new preprint criticizing the idea of Type S and M errors. The wonderful PsyArXiv moderators have just approved it, so if you didn't read it before, take a look: osf.io/preprints/psyarxiv/2phzb_v1
-
Daniël Lakens
@lakens
If even critical metascientists can't find anything to criticize about your work, I must be doing something right.
I spend a lot of time justifying all my recommendations in a philosophy of science. This leads to a strongly coherent view and why and how to improve science.
Daniël Lakens
@lakens
New paper by Mark Rubin confirms that my conceptual approach to justify the practice of preregistration is coherent, and a solid and strong logic that provides a basis for preregistrstion in science: link.springer.com/article/10.1007/s11229-025-05191-4
-
Daniël Lakens
@lakens
New paper by Mark Rubin confirms that my conceptual approach to justify the practice of preregistration is coherent, and a solid and strong logic that provides a basis for preregistrstion in science: link.springer.com/article/10.1007/s11229-025-05191-4
Daniël Lakens
@lakens
He concludes: "Mayo’s (1996, 2018) error statistical approach justifies the use of a well-specified preregistered research plan to allow others to evaluate the capability of a fixed test procedure to license inferences severely."
-
Daniël Lakens
@lakens
The 2024 edition was my favorite scientific meeting last year, lots of time to socialize, a nice dinner together, hearing about cutting-edge research from early career metascientists.
Daniël Lakens
@lakens
The full program for the PMGS Meta Research Symposium 2025 is online: docs.google.com/document/d/1bL-hOlc0g21JxLk2w_87tUy3dQVWY… If you are interested in causal inference, systematic review, hypothesis testing, and preregistration, join is October 17th in Eindhoven! Attendance is free!
-
Daniël Lakens
@lakens
The full program for the PMGS Meta Research Symposium 2025 is online: docs.google.com/document/d/1bL-hOlc0g21JxLk2w_87tUy3dQVWY… If you are interested in causal inference, systematic review, hypothesis testing, and preregistration, join is October 17th in Eindhoven! Attendance is free!
-
Daniël Lakens
@lakens
Unlike what the Bayesian priests want you to think, interim analyses are very problematic in Bayesian statistics: "repeated Bayesian interim analyses can affect operating characteristics of trials, particularly when vague or noninformative priors are used" arxiv.org/abs/2508.07403
-
Daniël Lakens reposted
Berry Consultants
@BerryConsultant
Episode 25 of "In the Interim…" features Dr. Scott Berry and Dr. Michael Proschan in a discussion of the merits of response-adaptive randomization (RAR).
Known to be on opposite ends of the debate, they highlight where RAR supports efficiency—primarily in multi-arm and platform trials with substantial simulation and strict protocols. While also covering the risks: loss of statistical power in two-arm studies, temporal trends, and high operational demands.
Real-world examples discussed include Ebola (PALM trial), ECMO (extracorporeal membrane oxygenation), I-Spy2, and COVID-19 platform studies.
Where are you on the use of RAR in clinical trials?
Don't miss this one! Tune in today: www.berryconsultants.com/resource/25-a-discussion-with-mi… -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
The PMGS Meta-Research Symposium program is out!
📅 Oct 16–17, Eindhoven University of Technology
🧑🏫 Workshop: Aaron Peikert
🎤 Keynotes: Duygu Uygun-Tunç & Lisa Spitzer
👩🔬 Talks by early-career researchers
Check the program + register by Oct 1 👉 paulmeehlschool.github.io/2025-08-17-programsymposium2025/ -
Daniël Lakens reposted
Daniël Lakens
@lakens
Too often, I see people talk about a replication as if the first study has established something, and the replication study is a double-check. What people often fail to understand is that we do not do replication studies to *check* a finding, but to *establish* a finding. 1/x
-
Daniël Lakens
@lakens
Too often, I see people talk about a replication as if the first study has established something, and the replication study is a double-check. What people often fail to understand is that we do not do replication studies to *check* a finding, but to *establish* a finding. 1/x
Post not available here, view on X
Daniël Lakens
@lakens
We distinguish chance findings from actual observations by examining if the effect occurs regularly, or not. If it does, we can conclude we have observed something. After a single study, there is no way to know if an effect will replicate – except by replicating it. 3/x
Daniël Lakens
@lakens
The replication studies are not an optional extra – they are an essential part of the methodological procedure we use to make scientific claims. That is also why replications should be done way more often than they are. For more, see lakens.github.io/statistical_inferences/17-replication.html
-
Daniël Lakens
@lakens
New episode of Nullius In Verba, part 1 of 2: What is metascience? nulliusinverba.podbean.com/e/metascience-i/
We dig into the history, starting from 1938, and discuss the role of philosophy, sociology, and psychology, and what, if anything, makes metascience it's own discipline. -
Daniël Lakens
@lakens
Episode 64: Scientia de Scientia - I nulliusinverba.podbean.com/e/metascience-i/
-
Daniël Lakens
@lakens
Amazing, the OSF has added a 'download this folder' link to the top of file repositories! That's a very nice improvement!
Daniël Lakens
@lakens
New blog post: Easily download files from the Open Science Framework with Papercheck daniellakens.blogspot.com/2025/07/easily-download-files-f…
Downloading all files in an OSF project can be a hassle. Not anymore, with Papercheck! Just run:
osf_file_download("6nt4v") -
Daniël Lakens
@lakens
An abbreviation (ABB) in a journal article (JA) or Grant Application (GA) is rarely worth the words it saves. Every ABB requires cognitive resources (CR) and at my age by the time I'm halfway through a JA or GA I no longer have the CR to remember what your ABB stood for.
-
Daniël Lakens
@lakens
Here's Selvin and Stuart, in 1966, discussing 'data dredging' in surveys, including 'fishing' and 'hunting' for significance. We've known about alpha inflation for so long! Glad preregistration is finally being more widely adopted! www.jstor.org/stable/2681493
-
Daniël Lakens
@lakens
One thing I like about this paper is that we worked together with some of the people we were criticizing, and who had promoted Type S and M errors. They are friends, which helps, but it was not given we could co-author. The paper is much better and nuanced for it in the end.
Daniël Lakens
@lakens
New Preprint: Rethinking Type S and M Errors. We argue Type S and M errors are not useful when designing or evaluating studies, and present alternative solutions to address the issues they were supposed to address (minimum effect tests, and p-uniform). osf.io/preprints/psyarxiv/2phzb_v1
-
Daniël Lakens
@lakens
Here's Selvin and Stuart, in 1966, discussing 'data dredging' in surveys, including 'fishing' and 'hunting' for significance. We've known about alpha inflation for so long! Glad preregistration is finally being more widely adopted! www.jstor.org/stable/2681493
-
Daniël Lakens
@lakens
New prologue episode of our podcast Nullius in Verba! Smriti reads the first chapter of Bunge's (1959) book Metascientific Queries, which asks 'Why Metascience? nulliusinverba.podbean.com/e/why-metascience/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Prologus 64: Why Metascience? (M. Bunge) nulliusinverba.podbean.com/e/why-metascience/
-
Daniël Lakens
@lakens
Prologus 64: Why Metascience? (M. Bunge) nulliusinverba.podbean.com/e/why-metascience/
-
Daniël Lakens
@lakens
J. D. Bernal in 1939 discussing how graduate students (only a thing from 1917 in the UK) are evaluated based on number and bulk, the publication pressure stifling originality, and filling the literature with useless papers.
-
Daniël Lakens reposted
Gilad Feldman
@giladfeldman
Been doing workshops about Peer Community In @PeerCommunityIn / PCI Registered Reports @PCI_RegReports / PCI Psychology & associated Peer Community Journal @PeerComJournal for 3 years now.
Most common concern: "PCJ has no impact factor"🤦
PCJ now has IF 2.
A science revolution. -
Daniël Lakens
@lakens
Saddened to hear about the death of Prof. Andrea Evers. Such an impressively talented scholar. A great loss for Dutch academia. www.universiteitleiden.nl/en/news/2025/08/in-memoriam-and…
-
Daniël Lakens
@lakens
We found that the results of 42% of registered studies on the OSF are never publicly shared journals.sagepub.com/doi/10.1177/25152459241296031. Most common reasons are null results and bad logistics. Plan your studies carefully (equivalence testing and good time management) to prevent research waste!
-
Daniël Lakens
@lakens
In our next episode of Nullius In Verba we will dive into the history of metascience. The first mention of 'metascience' in the literature we could find is Morris, 1938. Do you know of an earlier mention of the term 'metascience'?
-
Daniël Lakens reposted
ᴅʀ ᴍɪʀᴄᴇᴀ ᴢʟᴏᴛᴇᴀɴᴜ 🍂 🍃
@mzloteanu
Read the paper, some v convincing points. I saw you mention Kaizer's work (gamma error) and using two one-tailed tests, I wondered what your thoughts were on Lombardi and Hurlbert's work on these biology.sdsu.edu/pub/stuart/2009MisprescriptionOneTailed.pdf
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New Preprint: Rethinking Type S and M Errors. We argue Type S and M errors are not useful when designing or evaluating studies, and present alternative solutions to address the issues they were supposed to address (minimum effect tests, and p-uniform). osf.io/preprints/psyarxiv/2phzb_v1
-
Daniël Lakens reposted
Riccardo Padovan
@PadovanRiccardo
Just completed Improving Your Statistical Inferences 🎓
Thanks to @lakens for making such a high‑quality, free course on stats, reproducibility & #openscience.
Strongly recommend to all PhD, Master’s & undergrad students!
www.coursera.org/account/accomplishments/verify/YZKI0HDUH3TV -
Daniël Lakens
@lakens
New Preprint: Rethinking Type S and M Errors. We argue Type S and M errors are not useful when designing or evaluating studies, and present alternative solutions to address the issues they were supposed to address (minimum effect tests, and p-uniform). osf.io/preprints/psyarxiv/2phzb_v1
Post not available here, view on X
Daniël Lakens
@lakens
We admit Type S and M errors might be referenced in the discussion of badly designed and/or p-hacked studies, and they can be used in education to teach about one-sided tests and selection bias (although critical effect sizes might be more intuitive).
Daniël Lakens
@lakens
I always found it surprising so many people discussed Type S and M errors as if they are useful, and I hope our paper will make the scientific community rethink the usefulness of these novel statistical concepts, and appreciate the alternative approaches available a bit more!
-
Daniël Lakens reposted
Daniël Lakens
@lakens
We will soon put up a preprint where we criticize the concepts of Type S and M errors. They are not useful when designing and interpreting studies. Better solutions exist for what they intend to do.
If you like these concepts which points would we need to address to convince you? -
Daniël Lakens
@lakens
We will soon put up a preprint where we criticize the concepts of Type S and M errors. They are not useful when designing and interpreting studies. Better solutions exist for what they intend to do.
If you like these concepts which points would we need to address to convince you? -
Daniël Lakens
@lakens
New episode of @nulliusverbapod on Premature Experimentation: nulliusinverba.podbean.com/e/episode-63-experimenta-praem… Are experiments overused, and should we make greater use of other approaches to knowledge generation? If so, which methods should we use instead? Enjoy!
-
Daniël Lakens
@lakens
Episode 63: Experimentatio Exploratoria www.podbean.com/site/error?msg=user-site-page-not-found-r…
-
Daniël Lakens reposted
Jim Coan 🇺🇦
@jimcoan
🤦♂️
Neetu Arnold
@neetu_arnold
It’s 2025 and the American Psychological Association is trying to make implicit bias tests a thing again:
“The IAT has not been without its detractors…Yet various meta-analyses have repeatedly found support for the test’s validity” -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Only a few days remained to submit your abstracts for presentation at PMGS Meta-Research Symposium 2025. Make sure to submit them before the end of July by filling the following form:
forms.office.com/Pages/ResponsePage.aspx?id=R_J9zM5gD0qdd…
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for the PMGS Meta-Research Symposium 2025!
Abstract submissions are still welcome—deadline: July 31.
Plus: join our mini-workshop on coordination using GitHub on the afternoon of October 16.
Find all the details & sign up here:
paulmeehlschool.github.io/2025-07-16-symposium2025/ -
Daniël Lakens
@lakens
The first version of the open science blog archive is complete, with 3530 posts from 52 blogs. Browse them in the online Shiny app: shiny.ieis.tue.nl/open_science_blogs/ All blog posts are ordered chronologically. Have fun exploring part of the open science history in psychology!
Daniël Lakens
@lakens
You can click the ‘Happening Today’ button to see how Dorothy Bishop wrote the blog “Why we need preregistration’ in 2013 in response to Sophie Scott’s critique of preregistration. deevybee.blogspot.com/2013/07/why-we-need-pre-registratio… years later, preregistration is common practice.
Daniël Lakens
@lakens
If you work in history of psychology and want to grab the dataframe containing all blog posts to perform text analyses, just go the github repo: github.com/Lakens/open_science_blogsIf you have comments, suggestions, improvements, let me know.
-
Daniël Lakens reposted
Psicostat
@psicostat
In this interview for the Italian Association of Psychology (AIP), we retrace Psicostat’s path: from early workshops to Winter & Summer Schools, up to the launch of the Methodological Review Board and beyond!
👉 Here’s the link to the full interview: aipass.org/psicostat-un-laboratorio-aperto-tra-dati-idee-… -
Daniël Lakens reposted
Ben Grodeck🔸
@benleo_econ
I'm a big fan of Equivalence Testing. Simple and transparent way to provide evidence that a null result is robust (and not just an underpowered result).
Highly recommend to my colleagues in econ.
Daniël Lakens
@lakens
The first step to publishing null results is to learn about equivalence testing, and design studies that statistically support the conclusion that there is no meaningful effect. lakens.github.io/statistical_inferences/09-equivalencetes…. If you are a statistics teacher, include this into 2nd year courses.
nature
@Nature
Scientists overwhelmingly recognize the value of sharing null results, but rarely publish them in the research literature
go.nature.com/472poOC -
Daniël Lakens reposted
Marcus Munafò
@MarcusMunafo
When I was interviewed I started with exactly this - design your studies to be informative no matter what the outcome - but I think it was deemed too esoteric / technical for the article. Pity.
-
Daniël Lakens
@lakens
The first step to publishing null results is to learn about equivalence testing, and design studies that statistically support the conclusion that there is no meaningful effect. lakens.github.io/statistical_inferences/09-equivalencetes…. If you are a statistics teacher, include this into 2nd year courses.
nature
@Nature
Scientists overwhelmingly recognize the value of sharing null results, but rarely publish them in the research literature
go.nature.com/472poOC -
Daniël Lakens
@lakens
New blog post: Easily download files from the Open Science Framework with Papercheck daniellakens.blogspot.com/2025/07/easily-download-files-f…
Downloading all files in an OSF project can be a hassle. Not anymore, with Papercheck! Just run:
osf_file_download("6nt4v") -
Daniël Lakens
@lakens
There might not be 'a scientific method' but there is a desirable property of scientific methods. If you are right, the procedure you use to make claims should tell you that you are right. And when you are wrong, it should tell you that you are wrong.
-
Daniël Lakens
@lakens
I've created a shiny app to browse through all (now 3400) open science blog posts. There is a 'happening today' button. Among others, Rolf Zwaan in 2020 on sequential analyses rolfzwaan.blogspot.com/2020/07/how-we-did-not-fool-oursel…
-
Daniël Lakens
@lakens
Still playing around with creating an archive of open science blogs discussing the replication crisis. Here's an overview of when blogs first and last posted. 2016 and 2017 were a peak for psychologists interacting and communicating through in total more than 3000 blog posts.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Interested in the latest metascience? Come to Eindhoven October 17th to listen to Dr. Duygu Uygun-Tunç (University of Chicago) and Dr. Lisa Spitzer (Leibniz-Institut für Psychologie) as keynote speakers, and Dr. Aaron Peikert for a workshop collaboration through Git! All free!
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for the PMGS Meta-Research Symposium 2025!
Abstract submissions are still welcome—deadline: July 31.
Plus: join our mini-workshop on coordination using GitHub on the afternoon of October 16.
Find all the details & sign up here:
paulmeehlschool.github.io/2025-07-16-symposium2025/ -
Daniël Lakens
@lakens
Interested in the latest metascience? Come to Eindhoven October 17th to listen to Dr. Duygu Uygun-Tunç (University of Chicago) and Dr. Lisa Spitzer (Leibniz-Institut für Psychologie) as keynote speakers, and Dr. Aaron Peikert for a workshop collaboration through Git! All free!
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for the PMGS Meta-Research Symposium 2025!
Abstract submissions are still welcome—deadline: July 31.
Plus: join our mini-workshop on coordination using GitHub on the afternoon of October 16.
Find all the details & sign up here:
paulmeehlschool.github.io/2025-07-16-symposium2025/ -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for the PMGS Meta-Research Symposium 2025!
Abstract submissions are still welcome—deadline: July 31.
Plus: join our mini-workshop on coordination using GitHub on the afternoon of October 16.
Find all the details & sign up here:
paulmeehlschool.github.io/2025-07-16-symposium2025/ -
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
No new episode this week! Smriti is busy teaching an introduction to psychology course, and Daniel is relaxing in his garden with a new puppy.
-
Daniël Lakens
@lakens
Just checking, before I draw too general conclusions, but did anyone enjoy the Metascience conference in London? People from my lab absolutely hated it. Not a positive comment from any of my peers. Did anyone think it was worthwhile?
-
Daniël Lakens
@lakens
As I am going over old blog posts, here is a nice blog by Brent Roberts on qualities in graduate students that, in his opinion, lead to a successful career in academia. Still very good advice. pigee.wordpress.com/2012/11/08/a-lay-theory-of-the-succes…
-
Daniël Lakens
@lakens
A lot of discussions about open science in psychology happened on blogs. I have been archiving them and will make their content available. I remember 37 blogs (with 2827 posts!) but which blogs did I forget and are not on my list? Please share and reply with any that are missing!
-
Daniël Lakens
@lakens
Do you want to go to a metascience conference where AI will not solve all problems, you can ask critical questions, and El$evier is not invited? Sign up for the mailing list for the 8th Perspectives on Scientific Error conference in 2026 in Leiden! perspectivesonscientificerror2026.wordpress.com
-
Daniël Lakens
@lakens
Excellent opportunity for a PhD position with the wonderful supervisors Frank Renkewitz and Moritz Heene job-portal.lmu.de/jobposting/5f8aaaa92946fd18f937129b473a… “Models, Measures, Moderators - Exploration and Explanation of Heterogeneity in Psychological Replications” - great topic for a PhD!
-
Daniël Lakens reposted
Grant Abt
@grantabt
Today I pushed the ‘Accept in Principle’ button on another @JSportsSci Registered Report.
If you’re interested in finding out more about Registered Reports then read our editorial.
www.tandfonline.com/doi/full/10.1080/02640414.2021.1950974 -
Daniël Lakens reposted
The Dark Fiddling Pirate Jussim
@PsychRabble
Best recent piece of phil of sci I know of is by @HSJSpeaks. Social pressure NOT to make some point can corrupt an entire scientific literature. Original article on left; basic process on right.
-
Daniël Lakens
@lakens
New episode of @nulliusverbapod on exploratory experimentation. Why does exploratory research seem to be valued less? How can we do it well? nulliusinverba.podbean.com/e/experimenta-exploratoria/
-
Daniël Lakens
@lakens
Episode 62: Experimenta Exploratoria nulliusinverba.podbean.com/e/experimenta-exploratoria/
-
Daniël Lakens
@lakens
New blog post: Are meta-scientists ignoring philosophy of science? daniellakens.blogspot.com/2025/07/are-meta-scientists-ign… I don't think so, but I fear for reasons philosophers of science will not enjoy reading.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
"The intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not. The importance of the strength of our conviction is only to provide a proportionately strong incentive to find out if the hypothesis will stand up to critical evaluation."
-
Daniël Lakens
@lakens
"The intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not. The importance of the strength of our conviction is only to provide a proportionately strong incentive to find out if the hypothesis will stand up to critical evaluation."
Daniël Lakens
@lakens
Medawar, 1979, Advice to a Young Scientist
-
Daniël Lakens reposted
Muhammed Bulutay
@mvbulutay
I recently discovered this podcast and immediately fell in love with it. I vividly recommend it!
Daniël Lakens
@lakens
Our podcast @nulliusverbapod consists of 60+ episode on what science is, and what it could be. Smriti and I are passionate about learning, so we read out classic papers and chapters in our separate 'Prologue' episodes. We have 17 so far. Here's a list of all these episodes:
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
I'm excited to share that the first paper of my PhD project is now published. In this paper, my coauthors, @lakens and Krist Vaesen, and I discuss a systematic approach from different stakeholders to improve coordination.
You can read the paper at: doi.org/10.1038/s41562-025-02260-z -
Daniël Lakens reposted
Zac Crowley (PhD)
@crowley_zac
If you are a researcher in health sciences and haven't already done so, I would encourage you to give @nulliusverbapod a listen. Some very thought-provoking discussions and readings that I wish I had been exposed to much earlier in my career.
Daniël Lakens
@lakens
Our podcast @nulliusverbapod consists of 60+ episode on what science is, and what it could be. Smriti and I are passionate about learning, so we read out classic papers and chapters in our separate 'Prologue' episodes. We have 17 so far. Here's a list of all these episodes:
-
Daniël Lakens reposted
Nature Human Behaviour
@NatureHumBehav
While individualism and isolated work remain common in #academia, coordination offers substantial benefits. This Comment urges systemic changes from all stakeholders toward more coordinated #science. @sajedeh_rasti @lakens
www.nature.com/articles/s41562-025-02260-z -
Daniël Lakens reposted
Daniël Lakens
@lakens
Our podcast @nulliusverbapod consists of 60+ episode on what science is, and what it could be. Smriti and I are passionate about learning, so we read out classic papers and chapters in our separate 'Prologue' episodes. We have 17 so far. Here's a list of all these episodes:
-
Daniël Lakens
@lakens
Our podcast @nulliusverbapod consists of 60+ episode on what science is, and what it could be. Smriti and I are passionate about learning, so we read out classic papers and chapters in our separate 'Prologue' episodes. We have 17 so far. Here's a list of all these episodes:
Post not available here, view on X
Daniël Lakens
@lakens
The Anticreativity Letters (R. E. Nisbett). A creative article in which a senior devil and a junior devil discuss how they can manipulate scientists so that they will generate as little scientific knowledge as possible. Full of nuggets of wisdom. nulliusinverba.podbean.com/e/prologus-23-the-anticreativi…
Daniël Lakens
@lakens
The Fixation of Belief (C. S. Peirce),🔍 Dive into C.S. Peirce’s classic “The Fixation of Belief” and explore how we form convictions. A must-listen for fans of epistemology and scientific reasoning! nulliusinverba.podbean.com/e/prologus-25-the-fixation-of-…
Post not available here, view on X
Daniël Lakens
@lakens
We Have to Break Up (R. B. Cialdini),💔 Cialdini’s “We Have to Break Up” challenges the status quo in psychological science. A bold call for a focus on more applied research. nulliusinverba.podbean.com/e/prologus-49-we-have-to-break…
Daniël Lakens
@lakens
Probability Pyramiding (A. Neher),"📊 Before Ioannidis, there was Neher. This 1967 gem warns of research error and the need for replication. Still relevant today. nulliusinverba.podbean.com/e/prologus-56-probability-pyra…
-
Daniël Lakens reposted
Psicostat
@psicostat
Il gruppo di ricerca Psicostat esprime il proprio cordoglio per la scomparsa del Professore Carlo Arrigo Umiltà, Honorary Member del gruppo e figura centrale nella nostra comunità scientifica. Alla sua famiglia e a chi gli ha voluto bene va il nostro affetto e la nostra vicinanza
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Neher 1967: "The general adoption of independent replication as a requirement for acceptance of findings in the behavioral sciences will require the efforts of investigators, readers, and publishing editors alike. It seems clear that such a policy is both 1/2 >
-
Daniël Lakens
@lakens
Neher 1967: "The general adoption of independent replication as a requirement for acceptance of findings in the behavioral sciences will require the efforts of investigators, readers, and publishing editors alike. It seems clear that such a policy is both 1/2 >
Daniël Lakens
@lakens
development of a sound body of knowledge concerning human behavior’ If you want to listen to the entire paper,
@nulliusverbapod
has you covered. It's a classic so give it a listen: nulliusinverba.podbean.com/e/prologus-56-probability-pyra… -
Daniël Lakens reposted
AJ Thurston, PhD
@AJThurston
Academic influencers hyping results because p < .05 😃
Actual academics:
Angela Duckworth
@angeladuckw
📱 Just seeing your phone—even face down—can lower test scores.
Why? It drains mental energy.
The farther the phone, the higher the GPA.
This isn’t about willpower—it’s about situation modification.
Change your environment. Change the outcome.
That’s why I launched Phones in Focus—the first national study on school phone policies.
👉 Teachers & school leaders: Take the 5-min survey at phonesinfocus.org
#phonesinfocus #studentsuccess #behavioralscience #angeladuckworth #focusnotforce #educationmattersThis media couldn't be displayed. See it on the original post. -
Daniël Lakens reposted
Psicostat
@psicostat
Today’s presentations and all materials related to the MRB project are available on the following website: methodological-review-board.github.io
-
Daniël Lakens reposted
Psicostat
@psicostat
Lively session on setting up the Methodological Review Board today! 🔎
After presenting the MRB and the April survey results, a roundtable took place with voices from all three departments in our School of Psychology. Thanks to all and to @lakens for the closing remark! -
Daniël Lakens
@lakens
What is the most compelling book providing an overview of the successes and worthwhile contributions that psychology has made over the last century? Has it even been written? As one example, in my Human Factors course I discuss the 3rd break light. stanley-r-harris.co.uk/blog/why-do-cars-have-a-3rd-brake-…
-
Daniël Lakens reposted
Rosana Ferrero 📈📊🙌
@RosanaFerrero
💥 ¡Conoce {Papercheck}! 🛠️
Una herramienta AUTOMATIZADA para verificar buenas prácticas en artículos científicos. Desarrollada por @lakens y su equipo, usa IA y código para mejorar la transparencia en investigación.👇🧵
🔗 scienceverse.github.io/papercheck/
#openscience #rstats #stats -
Daniël Lakens
@lakens
The Paul Meehl Graduate School Meta Research Symposium 2025 is on October 17. Keynote speakers are Duygu Uygun-Tunç and Lisa Spitzer. The symposium is free to attend for everyone. And will soon announce an extra workshop the afternoon of the 16th. paulmeehlschool.github.io/2025-05-26-pmgs-symposium/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
Call for submissions: Paul Meehl Graduate School Meta Research Symposium 2025!
October 17, 2025, at Eindhoven University of Technology
Keynotes: Dr. Duygu Uygun-Tunç and Dr. Lisa Spitzer
ECR's, submit your talk before 31st of June!
paulmeehlschool.github.io/2025-05-26-pmgs-symposium/ -
Daniël Lakens reposted
Ross Tucker
@Scienceofsport
🎙️New podcast: The Replication Crisis Hits Sports Science.
72% of sports science studies could NOT be replicated.
Effect sizes dropped 75%. Nearly 90% of researchers wouldn’t even share data. We explore bad incentives, a crisis of trust & how to fix it
🎧 podcasts.apple.com/za/podcast/most-sports-science-researc… -
Daniël Lakens reposted
Daniël Lakens
@lakens
🚨 New blog post: "Retrieving Planned Sample Sizes from AsPredicted Preregistrations"
🔗daniellakens.blogspot.com/2025/06/retrieving-planned-samp…
We show how Papercheck can automatically extract planned sample sizes — which are the most common deviation in preregistrations. 1/3 -
Daniël Lakens
@lakens
🚨 New blog post: "Retrieving Planned Sample Sizes from AsPredicted Preregistrations"
🔗daniellakens.blogspot.com/2025/06/retrieving-planned-samp…
We show how Papercheck can automatically extract planned sample sizes — which are the most common deviation in preregistrations. 1/3
Daniël Lakens
@lakens
Optionally, we show how an LLM can infer if there are deviations from the preregistration. In line with Papercheck’s philosophy, any use of LLM’s is optional, and should not replace human judgment. It might prove useful, and sometimes works well, but further validation is needed.
Daniël Lakens
@lakens
We also show an example where the LLM gets it wrong. But we hope that automatically retrieving this information will remind authors to discuss deviations and help peer reviewers to check the preregistration. Lots of future developments are possible. Reach out if you want to help!
-
Daniël Lakens
@lakens
New episode! The 3rd and final part of our discussion of Paul Meehl's unpublished book 'The Seven Sacred Cows of Academia' nulliusinverba.podbean.com/e/seven-sacred-cows-iii/
Daniël Lakens
@lakens
Episode 61: Septem Vacae Sacrae III nulliusinverba.podbean.com/e/seven-sacred-cows-iii/
-
Daniël Lakens
@lakens
Episode 61: Septem Vacae Sacrae III nulliusinverba.podbean.com/e/seven-sacred-cows-iii/
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New blog post: Checking Data and Code in Repositories with Papercheck daniellakens.blogspot.com/2025/06/checking-data-and-code-… We show how our Papercheck tool can automatically check and list files in data repositories linked to in a scientific manuscript.
-
Daniël Lakens
@lakens
We also show how Papercheck can automate checks *inside* files, such as R scripts, to check for best practices! There is a lot of cool ways this idea can be expanded (e.g., checking for IP addresses in data files, etc). daniellakens.blogspot.com/2025/06/checking-data-and-code-…
-
Daniël Lakens
@lakens
New blog post: Checking Data and Code in Repositories with Papercheck daniellakens.blogspot.com/2025/06/checking-data-and-code-… We show how our Papercheck tool can automatically check and list files in data repositories linked to in a scientific manuscript.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
We also illustrate how Papercheck can easily be used to do meta-science. For example, is there some improvement in how often are effect sizes reported for t-tests in Psychological Science over time? Take a few lines of code to see: Yes, but not enough.
daniellakens.blogspot.com/2025/06/automatically-checking-… -
Daniël Lakens
@lakens
We also illustrate how Papercheck can easily be used to do meta-science. For example, is there some improvement in how often are effect sizes reported for t-tests in Psychological Science over time? Take a few lines of code to see: Yes, but not enough.
daniellakens.blogspot.com/2025/06/automatically-checking-… -
Daniël Lakens reposted
Panteleimon ("Paddy") Ekkekakis
@Ekkekakis
Arguably the most important project in the history of exercise science thus far. Equally enlightening is the accompanying "Reflections" paper. This conversation is long overdue. Congrats to @JennyMurphy2 for this incredible achievement and to @SportsMedicineJ for publishing these papers.
doi.org/10.1007/s40279-025-02201-w
doi.org/10.1007/s40279-025-02200-x -
Daniël Lakens
@lakens
New Papercheck blog: Automatically Checking Journal Article Reporting Standards. We demonstrate how Papercheck can flag inexactly reported p-values and missing effect sizes, and remind researchers to follow the APA JARS guidelines.
daniellakens.blogspot.com/2025/06/automatically-checking-… -
Daniël Lakens reposted
Daniël Lakens
@lakens
Very excited to publicly share news about a new tool, Papercheck, that Lisa DeBruine and me started to develop more than a year ago! In an introductory blog post, we explain our philosophy to automatically check scientific papers for best practices. daniellakens.blogspot.com/2025/06/introducing-papercheck.…
-
Daniël Lakens reposted
♕Deborah Mayo♕
@learnfromerror
New blogpost: "Sennsible significance" Commentary on Senn's guest post.
errorstatistics.com/2025/06/17/sennsible-significance-com… -
Daniël Lakens
@lakens
Very excited to publicly share news about a new tool, Papercheck, that Lisa DeBruine and me started to develop more than a year ago! In an introductory blog post, we explain our philosophy to automatically check scientific papers for best practices. daniellakens.blogspot.com/2025/06/introducing-papercheck.…
Daniël Lakens
@lakens
In the next few days, we will illustrate some modules we built that automatically check some of the JARS reporting guidelines, the contents of data repositories, preregistered sample sizes and deviations of them, and best practices in reporting power analyses.
Daniël Lakens
@lakens
As we explain, the module based tool makes it possible for anyone to contribute automated checks. The software is still an alpha version – things might change – but people we work with already find it so useful, we wanted to share it with others.
Daniël Lakens
@lakens
If you’d like to dive into how it works, see the articles on the GitHub page: scienceverse.github.io/papercheck/. If you want to work with us to make new modules, or have ideas for useful modules, let us know! And keep an eye out for the next blog posts!
-
Daniël Lakens reposted
Aaron Caldwell
@ExPhysStudent
I'm pleased to announce the publication of two important papers in Sports Medicine on the first every large replication project in sport and exercise science. Read the full papers:
link.springer.com/article/10.1007/s40279-025-02201-w
link.springer.com/article/10.1007/s40279-025-02200-x -
Daniël Lakens reposted
Daniël Lakens
@lakens
New Replication Project in sport and exercise science shares it results: Out of 25 replication studies performed, 56% yielded significant results, but only 7 (28%) revealed a similar effect size. Several problems related to the quality of are discussed.
link.springer.com/article/10.1007/s40279-025-02201-w -
Daniël Lakens
@lakens
New Replication Project in sport and exercise science shares it results: Out of 25 replication studies performed, 56% yielded significant results, but only 7 (28%) revealed a similar effect size. Several problems related to the quality of are discussed.
link.springer.com/article/10.1007/s40279-025-02201-w -
Daniël Lakens
@lakens
This was an incredibly interesting story about how amazing it is we now have a treatment for HIV, and the complexities of drug development. It is basically 5 episodes of an hour released at once. Haven't listened to anything where I learned so much in 5 hours in a while!
Saloni
@salonium
LAUNCH DAY 🚀 Today I’m launching a new podcast, Hard Drugs, with Jacob Trefethen (@JacobTref)
Our first episode is about lenacapavir — a new HIV drug that blocks infections with an efficacy rate of nearly 100%, and which could completely change the fight against HIV worldwide.
It’s also about the scientific journey leading up to this point, and how HIV was transformed from a lethal to a manageable condition.
3:52 How was HIV discovered? Where did it come from, and how does it attack the body and cause AIDS?
38:10 Antiretrovirals: How did scientists develop breakthrough HIV drugs — from azidothymidine to protease inhibitors to PrEP?
1:51:35 How does prevention and treatment work today?
2:19:03 HIV’s capsid and the breakthrough of lenacapavir, the first-approved HIV capsid inhibitor
2:50:36 How to develop long-lasting treatments
3:14:45 Lenacapavir’s near 100% efficacy in clinical trials
3:48:40 The impact of global programs against HIV, and can we now end HIV?This media couldn't be displayed. See it on the original post. -
Daniël Lakens reposted
Jonathan Mannhart 🔎
@JMannhart
I want to remind everybody again that there is a vaccine for Lyme disease. It‘s just that… nobody cared.
It‘s right on the article for Lyme disease. One of the most insane paragraphs ever on Wikipedia:
Mason
@webdevMason
So what I'm led to believe is that like half a million people contract Lyme Disease from hellbugs EVERY YEAR in this corner of the country and risk arthritis, heart problems and brain damage and they basically don't talk about it? It doesn't consume their every waking thought??
-
Daniël Lakens
@lakens
Venn (1866) described the problem with the "sharpshooter fallacy": 🎯 painting the bullseye on the spot where a bullet hits a door.
In psychology, we call this "HARKing" — Hypothesizing After Results Are Known.Post not available here, view on X
Daniël Lakens
@lakens
Preregistration might have other goals — but its main aim is to perform tests in a way that others can evaluate their severity. 🔍
Daniël Lakens
@lakens
For more on this, see:
www.tandfonline.com/doi/full/10.1080/2833373X.2024.2376046 -
Daniël Lakens reposted
Daniël Lakens
@lakens
Isn't it weird that different governments can take the exact same scientific knowledge about the state of science, and respond in completely opposite ways? This is what I see in The Netherlands vs the USA. Almost as if the problem is not the knowledge, but the government 🤷
-
Daniël Lakens
@lakens
Isn't it weird that different governments can take the exact same scientific knowledge about the state of science, and respond in completely opposite ways? This is what I see in The Netherlands vs the USA. Almost as if the problem is not the knowledge, but the government 🤷
-
Daniël Lakens reposted
Rafael Grigoryan
@eegdude
Why most published research findings are false, including "Why most published research findings are false"
Daniël Lakens
@lakens
New episode of @nulliusverbapod ! We (very) critically discuss the classic paper by John Ioannidis 'Why most published research findings are false'. What did he get wrong, and why is it still interesting to read the paper? nulliusinverba.podbean.com/e/episode-56-most-published-re…
-
Daniël Lakens reposted
UCD Public Health, Physiotherapy & Sports Science
@ucd_sphpss
Many thanks to Professor Daniel Lakens, Eindhoven University of Technology, who gave an insightful and engaging talk about open science at UCD last week. Thanks also to those who joined us for the talk from UCD, UCC, University of Maynooth, TUD and elsewhere. We hope you enjoyed the discussion and it has sparked some ideas for your own research.
osf.io/ejqa2/
@lakens
#openscience #metascience #goodresearch -
Daniël Lakens
@lakens
I often see Ioannidis' paper 'Why most published research findings are false' on reading lists. And by all means - I found it educational when I first read it. But, you might also want a summary of the criticisms on the paper, and we dive into these in our podcast episode.
Daniël Lakens
@lakens
New episode of @nulliusverbapod ! We (very) critically discuss the classic paper by John Ioannidis 'Why most published research findings are false'. What did he get wrong, and why is it still interesting to read the paper? nulliusinverba.podbean.com/e/episode-56-most-published-re…
-
Daniël Lakens reposted
Saloni
@salonium
LAUNCH DAY 🚀 Today I’m launching a new podcast, Hard Drugs, with Jacob Trefethen (@JacobTref)
Our first episode is about lenacapavir — a new HIV drug that blocks infections with an efficacy rate of nearly 100%, and which could completely change the fight against HIV worldwide.
It’s also about the scientific journey leading up to this point, and how HIV was transformed from a lethal to a manageable condition.
3:52 How was HIV discovered? Where did it come from, and how does it attack the body and cause AIDS?
38:10 Antiretrovirals: How did scientists develop breakthrough HIV drugs — from azidothymidine to protease inhibitors to PrEP?
1:51:35 How does prevention and treatment work today?
2:19:03 HIV’s capsid and the breakthrough of lenacapavir, the first-approved HIV capsid inhibitor
2:50:36 How to develop long-lasting treatments
3:14:45 Lenacapavir’s near 100% efficacy in clinical trials
3:48:40 The impact of global programs against HIV, and can we now end HIV?This media couldn't be displayed. See it on the original post. -
Daniël Lakens reposted
ReproducibiliTea UEA
@ReproTeaUEA
What an interesting talk with @grantabt, with quite a few references to @lakens
Check the video out - www.youtube.com/watch?v=oD8Ip_YTVRQ
and the comments for the slides.
ReproducibiliTea UEA
@ReproTeaUEA
Join our talk with @grantabt on ethical implications of poor research practices on 9 June at 2 pm UK time! See full Teams link in comments. uea-uk.libguides.com/researchrepro #openscience #academictwitter #openresearch #reproducibilitea
-
Daniël Lakens
@lakens
Listen to the incompetence and unfairness at the Journal for Population Economics in the @I4Replication podcast open.spotify.com/episode/4Eo0DFGixG5i4RYKdRvLb9
I4R
@I4Replication
A re-analysis of Ciacci's (2024) "Banning the purchase of sex increases cases of rape: Evidence from Sweden" Journal of Population Economics reveals major issues. A year ago, reproducers Adema, Folke, and Rickne found coding errors driving the paper's key results. Let's unpack
⬇️ -
Daniël Lakens
@lakens
Back from Dublin, where I gave 2 talks (NeuroPsychoEconomics conference and UCD). But also, the first time trad climbing in the amazingly beautiful Glendalough valley.
-
Daniël Lakens
@lakens
Why are so many critiques on how we do science stuck at the level of proto-criticism? 'P-values are not the answer', 'we need methodological pluralism'. What is this? Metascience for kindergartners?
-
Daniël Lakens reposted
Gregory R. Hancock
@GregoryRHancock
Hi friends! Here’s a new paper with @TheYiFeng that might be the summer beach read you’re looking for. 😉
Hancock, G. R., & Feng, Y. (2025, June 6). nmax and the Quest to Restore Caution, Integrity, and Practicality to the Sample Size Planning Process.
doi.org/10.31234/osf.io/ye75b_v1 -
Daniël Lakens
@lakens
Interesting fact: Bayes factor design analysis is a frequentist method based on the desired long run probability of reaching a fixed cutoff (BF>X).
ᴅʀ ᴍɪʀᴄᴇᴀ ᴢʟᴏᴛᴇᴀɴᴜ 🍂 🍃
@mzloteanu
#statstab #360 Bayes Factor Design Analysis {bfda}
Thoughts: Sample size planning is confusing at first with Bayesian. But BFDA is the quick answer.
#bfda #bayesian #bayesfactor #samplesize #power #errorrate
shinyapps.org/apps/BFDA/ -
Daniël Lakens
@lakens
Second part of @nulliusverbapod discussing Paul Meehl's 7 sacred cows of academia. Meehl criticizes one of our favorite topics: The idea that all research professors do is worthwhile. nulliusinverba.podbean.com/e/episode-60-septem-vacae-sacr…
-
Daniël Lakens
@lakens
Episode 60: Septem Vacae Sacrae II www.podbean.com/site/error?msg=user-site-page-not-found-r…
-
Daniël Lakens reposted
Yiğit Aşık
Sorulara baktım da bayağı güzel sorular, bu konularda sezgisellik sağlaması açısından iki güzel kaynak:
@lakens 'in Improving Your Statistical Inferences: lakens.github.io/statistical_inferences/
@StatModeling 'ın Regression and Other Stories
ezgi🦊
@ezgiturali
veri biliminde önemli istatistik becerilerinizi test edebileceğiniz eski google çalışanları tarafından hazırlanan bi soru seti buldum. mülakatlara girmeden göz atmak da iyi olabilir, bazen kafanızın çalışma mantığını anlamak için böyle sorular geliyor:
www.unofficialgoogledatascience.com/2025/03/quantifying-s… -
Daniël Lakens
@lakens
Had to say hi to Francis Bacon, such a regular guest on @nulliusverbapod , while visiting the old library in Trinity College.
-
Daniël Lakens
@lakens
At the Guinness brewery, raising a glass to William Gosset for inventing the t-test.
-
Daniël Lakens
@lakens
The journal of sport sciences recently made a sample size justification section mandatory. More and more journals are implementing this, as it is a central part of a study! Learn how to create a state of the art sample size justification from my paper: online.ucpress.edu/collabra/article/8/1/33267/120491/Samp…
-
Daniël Lakens
@lakens
It might be a bit niche (but isn't that the great thing about podcasts?) but the 'In the interim' podcast is a very educational insight into more complex trial design using sequential and adaptive testing. open.spotify.com/episode/6Gxk3ntfLY3DnCooZcCGCb
-
Daniël Lakens
@lakens
Don't trust anything scientifically illiterate people say.
The mortality salience literature is probably the biggest mess in psychology. It is immediately clear if you read any of the original articles.
For the recent final spasms: x.com/lakens/status/1917204389158371644
Bryan Johnson
@bryan_johnson
TIL, Mortality Salience
When people are reminded of their death (through news, illness, or even subtle cues), they are more likely to:
+ Defend cultural worldviews more strongly.
+ Show increased in-group favoritism and out-group derogation.
+ Seek self-esteem boosts (via success, social validation).
+ Become more punitive toward those who threaten their beliefs. -
Daniël Lakens reposted
František Bartoš
@BartosFra
Unfortunately not only psychology... we compared medicine, economics, psychology, and environmental sciences and all fields show notable signs of bias
onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1703 -
Daniël Lakens reposted
Brazilian Reproducibility Network
@BrRepNetwork
Our coordinator @olavoamaral presents results from the Brazilian Reproducibility Initiative this Friday, May 30, at 12h Brasília/15h GMT in a @OSFramework webinar. Register at cos-io.zoom.us/webinar/register/7717467385713/WN_6vqsQ_Gm…
-
Daniël Lakens reposted
Paul Hough
@the_hough
Improving Your Statistical Inferences 📊. This is an excellent (free) resource for students and academics: lakens.github.io/statistical_inferences/ Thanks to @lakens
-
Daniël Lakens reposted
Ambra
@ambra_prg
Ever feel disappointed by a non-significant result?
You should ask yourself: Could my study have even detected an effect that matters?
A simple way to find out is by computing critical effect size values.
doi.org/10.1177/25152459251335298 -
Daniël Lakens reposted
-
Daniël Lakens reposted
Daniël Lakens
@lakens
The Dutch budget for the transition to Open Science was 20 million per year. Due to budget cuts, it now is 10 million. Yesterday they launched a 3 million euro metascience call to contribute to the evidence base for open science. This is what a well-functioning country does.
-
Daniël Lakens
@lakens
Social psychology today, it seems to me, is in a state of profound intellectual disarray. We approach our work with a kind of restless pioneer spirit: a new [.] territory is discovered, explored for a while, and then usually abandoned when the going gets rough or uninteresting.
Daniël Lakens
@lakens
Ring, 1967, doi.org/10.1016/0022-1031(67)90016-9
-
Daniël Lakens
@lakens
If you work in psychology in an area without Registered Reports, you will see that the main hypotheses in papers are almost always supported. If your peers publish Registered Reports, you will also have access to null results, which should be very common. doi.org/10.1177/25152459211007467
-
Daniël Lakens
@lakens
The Dutch budget for the transition to Open Science was 20 million per year. Due to budget cuts, it now is 10 million. Yesterday they launched a 3 million euro metascience call to contribute to the evidence base for open science. This is what a well-functioning country does.
-
Daniël Lakens reposted
Constantin Späth @cspaeth.bsky.social
@ConstantinSpath
"If the use of a tool is detached from a philosophy of science it risks becoming a heuristic."
Daniël Lakens
@lakens
This is in my opinion one of the best papers I have written. It moved preregistration from 'let's do it because it is good' to 'let's do it when we have this philosophy of science'. It still provides the most coherent argument for why to adopt it in the metascientific literature.
Carlisle Rainey
@carlislerainey
"The value of preregistration for psychological science: A conceptual analysis"
from Lakens
paper: www.jstage.jst.go.jp/article/sjpr/62/3/62_221/_pdf/-char/ja -
Daniël Lakens
@lakens
Although maybe it is now better to read the 2024 paper we wrote on this topic: www.tandfonline.com/doi/full/10.1080/2833373X.2024.2376046 We explain why Registered Reports are the best implementation of preregistration, summarize the surveys on p-hacking, and counter common criticisms on preregistration.
Daniël Lakens
@lakens
This is in my opinion one of the best papers I have written. It moved preregistration from 'let's do it because it is good' to 'let's do it when we have this philosophy of science'. It still provides the most coherent argument for why to adopt it in the metascientific literature.
Carlisle Rainey
@carlislerainey
"The value of preregistration for psychological science: A conceptual analysis"
from Lakens
paper: www.jstage.jst.go.jp/article/sjpr/62/3/62_221/_pdf/-char/ja -
Daniël Lakens
@lakens
This is in my opinion one of the best papers I have written. It moved preregistration from 'let's do it because it is good' to 'let's do it when we have this philosophy of science'. It still provides the most coherent argument for why to adopt it in the metascientific literature.
Carlisle Rainey
@carlislerainey
"The value of preregistration for psychological science: A conceptual analysis"
from Lakens
paper: www.jstage.jst.go.jp/article/sjpr/62/3/62_221/_pdf/-char/ja -
Daniël Lakens reposted
Carlisle Rainey
@carlislerainey
"The value of preregistration for psychological science: A conceptual analysis"
from Lakens
paper: www.jstage.jst.go.jp/article/sjpr/62/3/62_221/_pdf/-char/ja -
Daniël Lakens reposted
Open Research Europe
@OpenResearch_EU
Need help making your research reproducible? Check out these simple, effective tips!
ore.eu/news
#openscience #research -
Daniël Lakens
@lakens
Goverments come and go, scientific progress remains. I am proud of the work I have done to improve science, and I will continue to call out the problems that make science less reliable and efficient, and spend my life trying to fix them.
-
Daniël Lakens
@lakens
And if you are a senior researcher, attend! This was such a fun day full of cutting edge metascience last year.
Daniël Lakens
@lakens
Call for submissions: Paul Meehl Graduate School Meta Research Symposium 2025!
October 17, 2025, at Eindhoven University of Technology
Keynotes: Dr. Duygu Uygun-Tunç and Dr. Lisa Spitzer
ECR's, submit your talk before 31st of June!
paulmeehlschool.github.io/2025-05-26-pmgs-symposium/ -
Daniël Lakens
@lakens
Call for submissions: Paul Meehl Graduate School Meta Research Symposium 2025!
October 17, 2025, at Eindhoven University of Technology
Keynotes: Dr. Duygu Uygun-Tunç and Dr. Lisa Spitzer
ECR's, submit your talk before 31st of June!
paulmeehlschool.github.io/2025-05-26-pmgs-symposium/ -
Daniël Lakens reposted
♕Deborah Mayo♕
@learnfromerror
There's a new guest post up by Stephen Senn on relevant significance, power, and the misinterpretations in viewing an observed difference that is clinically relevant as evidence for a clinically relevant population effect. errorstatistics.com/2025/05/21/stephen-senn-guest-post-re…
-
Daniël Lakens reposted
Psicostat
@psicostat
It's important to be able to go beyond statistical folklore and yesterday Jelle Goeman reminded us of that! Visiting from Leiden University, he shared an insightful view on correcting for multiple testing within the Bayesian framework. A real pleasure to hear and exchange ideas!
-
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
If you like our theme music, Grandbrothers will be releasing a new LP that you can pre-order: shop.grandbrothersmusic.com/products
-
Daniël Lakens
@lakens
New study by @NWONieuws finds relatively good agreement when the same proposals are ranked by 2 committees. Differences are due to interpretations of high risk, or how convincinf a rebuttal by the applicant is. They see no reason to change the procedure. www.nwo.nl/en/news/nwo-conducts-double-blind-study-on-ass…
-
Daniël Lakens
@lakens
New episode of @nulliusverbapod Part 1 of a discussion of Paul Meehl's unpublished book 'The Seven Sacred Cows of Academia'. How can we make universities better and more efficient by questioning some basic ideas? nulliusinverba.podbean.com/e/seven-sacred-cows/
-
Daniël Lakens
@lakens
Episode 59: Septem Vacae Sacrae nulliusinverba.podbean.com/e/seven-sacred-cows/
-
Daniël Lakens reposted
irsp_rips
@irsp_rips
🔔‼️ New papers alert
In these papers, @lakens takes a deep dive into the two major crises in psychology.
Concerns about replicability, theorizing, relevance, generalizability, and methodology are discussed.
A must-read for every psychology researcher!
Available in #oa 👇👇 -
Daniël Lakens reposted
Daniël Lakens
@lakens
What makes exploratory research valuable? We discuss 6 factors: 1) Large and 2) Uncommon sample 3) Rigorous data collection 4) Widely used measures that 5) Reveal a coherent pattern across variables and 6) The potential to generate questions.
doi.org/10.1080/02640414.2025.2486871 -
Daniël Lakens
@lakens
What makes exploratory research valuable? We discuss 6 factors: 1) Large and 2) Uncommon sample 3) Rigorous data collection 4) Widely used measures that 5) Reveal a coherent pattern across variables and 6) The potential to generate questions.
doi.org/10.1080/02640414.2025.2486871 -
Daniël Lakens
@lakens
Save the date for the 8th Perspectives on Scientific Error conference on 11, 12, and 13 Februari 2026 in Leiden, the Netherlands. perspectivesonscientificerror2026.wordpress.com
This is an event not to miss, bringing together philosophers, statisticians, and metascientists. -
Daniël Lakens
@lakens
A recent attempt by psychologists to create 'consensus' on smartphones and social media was a huge failure:
www.newscientist.com/article/2480657-attempt-to-reach-exp…
The lesson here is that our field needs to learn how other disciplines actually reach consensus (much more slowly and inclusively!). -
Daniël Lakens
@lakens
We sometimes need to deviate from a preregistration to increase the validity of our inferences. Not all deviations reduce the severity of our tests - but they often do. Be transparent about all deviations, and evaluate the consequences for test severity. online.ucpress.edu/collabra/article/10/1/117094/200749/Wh…
-
Daniël Lakens reposted
Daniël Lakens
@lakens
I used the 'guess the correlation' game for teaching, but a while ago it started to collect data (Google ID, IP, guesses) It's amazing you can now make a Shiny app in 10 minutes using ChatGPT. If you want to play, a GDPR friendly version is online here: shiny.ieis.tue.nl/guesscorrelation/
-
Daniël Lakens reposted
Tarun Bhatnagar (he/him/his)
@tarunb20
Truly state of the art….a wonderful resource for up-to-date thinking and application in statistics
Daniël Lakens
@lakens
Finally got around to incorporating all Github feedback on my open access textbook, making around 20 minor improvements, and updating some references. 17 chapters of state of the art stats and methods education, freely available for any course you teach. lakens.github.io/statistical_inferences/
-
Daniël Lakens
@lakens
I used the 'guess the correlation' game for teaching, but a while ago it started to collect data (Google ID, IP, guesses) It's amazing you can now make a Shiny app in 10 minutes using ChatGPT. If you want to play, a GDPR friendly version is online here: shiny.ieis.tue.nl/guesscorrelation/
-
Daniël Lakens
@lakens
Finally got around to incorporating all Github feedback on my open access textbook, making around 20 minor improvements, and updating some references. 17 chapters of state of the art stats and methods education, freely available for any course you teach. lakens.github.io/statistical_inferences/
-
Daniël Lakens
@lakens
Researchers should not make claims based on exploratory analyses because the probability that these claims are wrong can be unacceptably high.
doi.org/10.1080/02640414.2025.2486871 -
Daniël Lakens
@lakens
Both Bayes factors and equivalence tests will in real-life cases lead to similar inferences about the effect - if any - being too small to matter. If you are more experienced with frequentism, you'll probably do better by performing equivalence tests.
academic.oup.com/psychsocgerontology/article-abstract/75/… -
Daniël Lakens
@lakens
Excellent presentation by @sajedeh_rasti today on the importance of more coordination in psychological science at the theory and open science meeting in Freiburg, which was so very lovely!
-
Daniël Lakens
@lakens
You can still apply to this position until May 25. If this is a topic that sounds interesting, you want to work in an intellectually stimulating environment on a research question that matters, at a department with nice colleagues, with the resources to do what we want, apply!
Daniël Lakens
@lakens
I am hiring a PhD student on the meaningful interpretation of effect sizes as part of my VICI funded project. This is a 4 year paid position in a welcoming and collaborative environment. Find out more or apply at www.tue.nl/werken-bij-tue/vacatureoverzicht/phd-on-metasc…
-
Daniël Lakens reposted
Jack Fitzgerald | @jackfitzgerald.bsky.social
@FitzgeraldJack_
Dutch academics, please share - replication games are coming to Vrije Universiteit Amsterdam on July 19! If you live in the Netherlands, or want a good reason to visit, then come network, hone your skills, and potentially coauthor a paper. Registration: www.surveymonkey.ca/r/Replication_Games_Amsterdam_2025
I4R
@I4Replication
Registration is open for 7 upcoming replication games: i4replication.org/games.html
Looking for researchers in econ/ecology/polsci/psych/pub health
San Francisco (June 19)
Amsterdam (July 19)
Tilburg (Aug 21)
Bordeaux (Aug 24)
Brno (Sept 7)
Paris (Oct 3)
Lyon (Oct 9)
Short 🧵 -
Daniël Lakens reposted
Daniël Lakens
@lakens
If you design a study to test a prediction, you need to be able to falsify it. In frequentist statistics this is achieved through an equivalence test. You can't have an informative test if you can't prove yourself wrong. Learn about equivalence testing.
lakens.github.io/statistical_inferences/09-equivalencetes… -
Daniël Lakens
@lakens
If you design a study to test a prediction, you need to be able to falsify it. In frequentist statistics this is achieved through an equivalence test. You can't have an informative test if you can't prove yourself wrong. Learn about equivalence testing.
lakens.github.io/statistical_inferences/09-equivalencetes… -
Daniël Lakens
@lakens
My papers summarizing the strong similarities between the crises in psychology in the 1970's and the 2010's, including reflections on why we are not solving most of them, are out in open access print. You'll enjoy this I promise.
rips-irsp.com/articles/10.5334/irsp.1036
rips-irsp.com/articles/10.5334/irsp.1038 -
Daniël Lakens reposted
Dan Williams
@danwilliamsphil
A recent article in Time Magazine cites me as an example of someone “cultivated” by authoritarian leaders like Trump to “defend the authoritarian agenda.” Why? Because I’ve criticised some research into misinformation and disinformation. I’m a bit annoyed by this. 1/2
-
Daniël Lakens
@lakens
Heading over to Zurich to teach a 4 day workshop on designing better experiments, followed by a visit to Freiburg for the Theory Development meeting. Looking forward to a busy but fun week!
-
Daniël Lakens reposted
Gilad Feldman
@giladfeldman
Now officially published:
doi.org/10.1098/rsos.250367 -
Daniël Lakens reposted
I4R
@I4Replication
We discuss the comment and more in our new podcast episode: spotifycreators-web.app.link/e/ngjZPoVfeTb. Two of the three replicators join us.
-
Daniël Lakens reposted
I4R
@I4Replication
the methods stated in the paper, for example by using incorrect variables, adding variables to regressions, using incorrect bandwidth numbers for the RD analysis, or changing clustering methods.
-
Daniël Lakens reposted
I4R
@I4Replication
This reanalysis finds that errors in the paper’s statistical modeling and coding create the illusion of an effect not just in the main analysis, but in all five strategies. The methods implemented by the original author differ in various ways from...
-
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
A little late, but the new episode on science communication is out now!
We discuss the Carl Sagan effect, pop science books, science journalism, YouTube channels, podcasts... and coffee and sandwiches.
nulliusinverba.podbean.com/e/science-communication/ -
Daniël Lakens
@lakens
Episode 58: Communicatio Scientiae nulliusinverba.podbean.com/e/science-communication/
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
There is still time to sign up for the last workshop of this academic year at PMGS, "Scientific Criticism and Peer Review," by @renebekkers
Check out the details here:
paulmeehlschool.github.io/workshops/criticism/ -
Daniël Lakens reposted
Quentin André
@andre_quentin
Data Colada has submitted a motion for sanction against Francesca Gino and her former lawyer team.
I hope the court rules in their favor. It will be a powerful message that sleuths can expose fake research without fearing legal consequences. -
Daniël Lakens reposted
Internet Archive
@internetarchive
Thank you, Dr. Carla Hayden, for your leadership & commitment to ensuring access to knowledge for all. Your service as Librarian of Congress has inspired a generation of librarians, archivists & readers.
📸 Dr. Hayden meets with @internetarchive founder @brewster_kahle, 2017. -
Daniël Lakens
@lakens
Truly excellent workshop by @peder_isager on Causal Inferences, and lovely to be able to hang out afterwards to talk metascience!
Daniël Lakens
@lakens
We are starting today's Paul Meehl Graduate School workshop on Causal Inferences taught by the wonderful @peder_isager Great turnout with people from Italy, Belgium, and Germany. Looking forward to learning about this myself! paulmeehlschool.github.io/workshops/causal/
-
Daniël Lakens
@lakens
We are starting today's Paul Meehl Graduate School workshop on Causal Inferences taught by the wonderful @peder_isager Great turnout with people from Italy, Belgium, and Germany. Looking forward to learning about this myself! paulmeehlschool.github.io/workshops/causal/
-
Daniël Lakens
@lakens
"Meta-science, in essence, involves the application of more rigorous methods than are presently used in the history and philosophy of science to answer questions about how science does and does not work best." Faust, 2005. doi.org/10.1002/jclp.20185
-
Daniël Lakens reposted
Grant Abt
@grantabt
Sometimes you need a carrot, sometimes you need a (small) stick. @JSportsSci we've introduced a mandatory sample size estimation and justification section to manuscripts. The rationale is outlined in the editorial.
Editorial
www.tandfonline.com/doi/full/10.1080/02640414.2025.249940…
IFA
www.tandfonline.com/action/authorSubmission?show=instruct… -
Daniël Lakens
@lakens
The journal of Sports and Sciences will now require mandatory sample size estimation and justification www.tandfonline.com/doi/full/10.1080/02640414.2025.249940… This is an excellent step that will improve scientific papers. I expect many other journals will follow.
-
Daniël Lakens
@lakens
Reviewer 2 here: If your intro has an Appeal to Extremes fallacy (e.g. If all of science consisted of X, if people only do X, only X will not solve...) - especially without a citation (because no one said only X would be enough) I will be annoyed reading the rest of your paper.
-
Daniël Lakens
@lakens
Look at what I found at a flea market in Kyoto on this May The Fourth!
-
Daniël Lakens reposted
Andreas Ortmann
@aortmannphd
The journal where the editor-in-chief withdrew one of the Islam et al papers after I4R asked for the replication package and two of the authors asked for the removal of their names. See line 10 in I4R 31 March spreadsheet update. I guess: Lesson learned.
I4R
@I4Replication
Good news in the field of development economics! The journal Economic Development and Cultural Change just implemented a data & code availability policy: www.journals.uchicago.edu/journals/edcc/data-availability…
Required to post data/codes/Readme + dataset may be randomly selected for review! -
Daniël Lakens reposted
Juan (el del blog) Fernández
@JuanEldelblog
NUEVA entrada del blog adaptando los excelentes materiales de @lakens sobre inferencia estadística y ciencia abierta. Con apuntes relevantes sobre el uso de evidencias en educación:
investigaciondocente.com/2025/05/01/entre-el-sesgo-de-con… -
Daniël Lakens reposted
Daniël Lakens
@lakens
It is weird everyone knows ego-depletion as one of the research lines where hundreds of studies were performed, but there was no effect - but the Mortality Salience literature has become a historical example of much larger research waste. 825 studies, nothing real about it.
Daniël Lakens
@lakens
R.I.P Terror Management: A Z-Curve Analysis. replicationindex.com/2025/04/27/r-i-p-terror-management-a… by Uli Schimmack
-
Daniël Lakens reposted
novemberrain1108
@Tatsunori_Ishii
Many Lab 4の結果からも、死の顕現化の効果はばらつきがある(ように見える)、効果の平均的は0に近いということだったけど、新しいメタ分析(Chen et al., 2025, JPSP)でも同様らしい。
psycnet.apa.org/doiLanding?doi=10.1037%2Fpspa0000438
Daniël Lakens
@lakens
It is weird everyone knows ego-depletion as one of the research lines where hundreds of studies were performed, but there was no effect - but the Mortality Salience literature has become a historical example of much larger research waste. 825 studies, nothing real about it.
Daniël Lakens
@lakens
R.I.P Terror Management: A Z-Curve Analysis. replicationindex.com/2025/04/27/r-i-p-terror-management-a… by Uli Schimmack
-
Daniël Lakens reposted
Rob Sica
@robsica
David Buss, 1997:
"TMT [Terror Management Theory] is anchored in an outmoded evolutionary biology"
www.researchgate.net/publication/246399872_Human_Social_M…
Daniël Lakens
@lakens
It is weird everyone knows ego-depletion as one of the research lines where hundreds of studies were performed, but there was no effect - but the Mortality Salience literature has become a historical example of much larger research waste. 825 studies, nothing real about it.
Daniël Lakens
@lakens
R.I.P Terror Management: A Z-Curve Analysis. replicationindex.com/2025/04/27/r-i-p-terror-management-a… by Uli Schimmack
-
Daniël Lakens
@lakens
Duygu and Mehmet are such sharp minds! They have convinced me that setting the alpha level to different thresholds for different studies is not a good thing beyond actual decision making, and that it is unwise to adopt context specific alpha levels for scientific claims.
-
Daniël Lakens reposted
Prof Neil Martin
@ThatNeilMartin
I tried my best, Daniel, I tried my best.
-
Daniël Lakens
@lakens
It is weird everyone knows ego-depletion as one of the research lines where hundreds of studies were performed, but there was no effect - but the Mortality Salience literature has become a historical example of much larger research waste. 825 studies, nothing real about it.
Daniël Lakens
@lakens
R.I.P Terror Management: A Z-Curve Analysis. replicationindex.com/2025/04/27/r-i-p-terror-management-a… by Uli Schimmack
-
Daniël Lakens
@lakens
R.I.P Terror Management: A Z-Curve Analysis. replicationindex.com/2025/04/27/r-i-p-terror-management-a… by Uli Schimmack
-
Daniël Lakens
@lakens
On a holiday in Japan, staking out the next location for our writing retreat. This is the Engyoji temple complex at the edge of Himeji.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New episode of Nullius In Verba on censorship in science.
Daniël Lakens
@lakens
Episode 57: Censura nulliusinverba.podbean.com/e/episode-57-censura/
-
Daniël Lakens
@lakens
New episode of Nullius In Verba on censorship in science.
Daniël Lakens
@lakens
Episode 57: Censura nulliusinverba.podbean.com/e/episode-57-censura/
-
Daniël Lakens
@lakens
Episode 57: Censura nulliusinverba.podbean.com/e/episode-57-censura/
-
Daniël Lakens reposted
Lucija Batinovic
@Luce10101
For people who published/conducted Stage 2 registered reports, have you ever had issues that required significant deviations from the Stage 1 and how have you handled this? Are there any instances where this did not lead to rejection?
-
Daniël Lakens reposted
Matthew B. Jané
@MatthewBJane
Error Report #4: matthewbjane.quarto.pub/meta-analysis-error-reports/repor…
The meta-analysis on Fluoride and IQ by Taylor et al. (2025) is methodologically flawed and has serious data integrity issues. The full detailed report co-authored by me, James Heathers, and David Robert Grimes is available as a pre-print on OSF: osf.io/preprints/osf/zhm54_v3 -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Registration is open for our final workshop of the year: Scientific Criticism and Peer Review (June 6, Eindhoven). Led by René Bekkers, info: paulmeehlschool.github.io/2025-04-22-criticism/
Great opportunity to attend both the workshop and PYMS symposium on June 5 in Tilburg: youngmetascientists.github.io/symposium2025 -
Daniël Lakens reposted
Daniël Lakens
@lakens
I am hiring a PhD student on the meaningful interpretation of effect sizes as part of my VICI funded project. This is a 4 year paid position in a welcoming and collaborative environment. Find out more or apply at www.tue.nl/werken-bij-tue/vacatureoverzicht/phd-on-metasc…
-
Daniël Lakens reposted
ReproducibiliTea UEA
@ReproTeaUEA
An interesting two-part reading on two replicability crises in Social Psychology, with Part 1 highlighting the continuing lack of incentives to perform replication studies and the importance of large-scale research projects to instigate change by @lakens osf.io/preprints/psyarxiv/dtvs7_v2
-
Daniël Lakens
@lakens
I am hiring a PhD student on the meaningful interpretation of effect sizes as part of my VICI funded project. This is a 4 year paid position in a welcoming and collaborative environment. Find out more or apply at www.tue.nl/werken-bij-tue/vacatureoverzicht/phd-on-metasc…
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New paper: Blaming the Thermometer for the Fever: Separating Misapplication from Method in Null Hypothesis Significance Testing, with @laabho and @jesper_wulff . We - strongly - push back on criticisms against NHST, and argue for its proper application. papers.ssrn.com/sol3/papers.cfm?abstract_id=5212117 >
-
Daniël Lakens reposted
Journal of Sports Sciences
@JSportsSci
🆕 Exploratory research in sport and exercise science: Perceptions, challenges, and recommendations
👉 www.tandfonline.com/doi/full/10.1080/02640414.2025.2486871
Written by M Ditroilo, Cristian Mesquida, @grantabt & Daniel @lakens -
Daniël Lakens
@lakens
New paper: Blaming the Thermometer for the Fever: Separating Misapplication from Method in Null Hypothesis Significance Testing, with @laabho and @jesper_wulff . We - strongly - push back on criticisms against NHST, and argue for its proper application. papers.ssrn.com/sol3/papers.cfm?abstract_id=5212117 >
Post not available here, view on X
Daniël Lakens
@lakens
We are in good company. Remember that in the special issue on ‘moving beyond p < .05’ most proposals were actually to keep using p < .05! The only additions were to use p < .05 more thoughtfully errorstatistics.com/2024/07/01/guest-post-daniel-lakens-h… >
Daniël Lakens
@lakens
Will this paper do what we hope – stop the century-old discussion about NHST, get people to realize we have bigger fish to fry, and that we should train people to become thoughtful dichotomous claim-makers instead of abandoning NHST? I claim it will. 😊
-
Daniël Lakens reposted
Benjamin Spivak
@BenjaminSpivak
Lancet Psychiatry now requiring manuscripts to acknowledge lack of lived experience as a limitation if absent (which means pressure to reveal lived experience). This is not a good development imo.
www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(2… -
Daniël Lakens reposted
Jesper N. Wulff
@jesper_wulff
New preprint. We tackle the most common criticisms of NHST and show how to address them. With NHST 🤯
Lau Holgersen
@laabho
New preprint with @lakens and @jesper_wulff:
"Blaming the Thermometer for the Fever: Separating Misapplication from Method in NHST"
NHST gets a lot of blame—mostly for issues it did not cause!
Read it here: ssrn.com/abstract=5212117 -
Daniël Lakens reposted
Lau Holgersen
@laabho
As promised, the preprint is now out:
"Blaming the Thermometer for the Fever: Separating Misapplication from Method in NHST"
We push back against calls to ban dichotomous testing and defend error control—properly applied.
Read here: ssrn.com/abstract=5212117
Daniël Lakens
@lakens
We will soon post the corresponding preprint 'Blaming the Thermometer for the Fever: Separating Misapplication from Method in Null Hypothesis Significance Testing' which pushes back on the idea to ban error control in science by not allowing dichotomous tests.
Jesper N. Wulff
@jesper_wulff
Paper with @laabho and @lakens and PDW with Sirio Lonati accepted for AOM. Copenhagen, here we come! #aom2025 @AOMConnect
-
Daniël Lakens reposted
Lau Holgersen
@laabho
New preprint with @lakens and @jesper_wulff:
"Blaming the Thermometer for the Fever: Separating Misapplication from Method in NHST"
NHST gets a lot of blame—mostly for issues it did not cause!
Read it here: ssrn.com/abstract=5212117 -
Daniël Lakens
@lakens
The Dutch rail service NS internally replicated all their A/B tests. 50% did not replicate. A familiar story for scientists, very intriguing to hear companies have the same problem in their own research. Food for thought! (Link in Dutch) www.linkedin.com/posts/nsinternational_abtesten-datadrive…
-
Daniël Lakens
@lakens
At the Nethics meeting at Eindhoven University of Technology on AI and Ethics review. The new Dutch Code of Ethics for Social and Behavioral Sciences is available, and an interesting program and whether and how AI should play a role in ethics review.
-
Daniël Lakens reposted
Grant Abt
@grantabt
As a companion piece to our recent paper (www.tandfonline.com/doi/full/10.1080/02640414.2025.2486871), at @E_C_S_S Rimini this year we present a poster on exploratory research.
IS EXPLORATORY RESEARCH MISUNDERSTOOD IN SPORT AND EXERCISE SCIENCE?
04.07.2025, 11:00 - 12:00, Hall 5 (Poster Session 3)
@lakens -
Daniël Lakens reposted
Kert Viele
@KertViele
One of my core interests is the massive love/hate relationship statisticians have with data. Want to approve a drug or treat a patient today? Look at the available evidence!! Want to design an experiment today…many statisticians answer…IGNORE the available evidence!! Why?
Berry Consultants
@BerryConsultant
Join Scott Berry and Kert Viele as they discuss the complex topic of using external data in clinical trials.
Explore the debate on whether incorporating historical and real-world evidence enhances or complicates trial design. Listen in as they dissect the benefits, risks, and future of clinical trials.
Tune in today to episode 8, "External Data in Clinical Trials": berryconsultants.com/podcast -
Daniël Lakens reposted
Daniël Lakens
@lakens
My paper on concerns about replicability, theorizing, relevance, generalizability, and methodology across 2 crises is now in press at @irsp_rips. After revisions it was 17500 words, so it is split in 2 parts: osf.io/dtvs7_v2 and osf.io/g6kja_v1
-
Daniël Lakens
@lakens
5 years ago we bought the house below ours. We use both (especially the garden). We thought the notary would inform the municipality, but recently realized we needed to. After 7 rounds of paperwork, it finally was approved. The publication process perfectly prepared me for this!
Daniël Lakens
@lakens
We wanted to merge both houses formally. They told us we needed to make a constructive change. OK. We will make a door behind the front door. We needed a drawing whether the fire compartments of our house changed. We made a drawing. How much CO2 would construction generate? Etc.
Daniël Lakens
@lakens
After round 4 the friendly employee at the municipality told me: No, you should be frustrated by this process, this procedure is way too much of a hassle for what you are trying to achieve. Upon which I replied the peer review process had trained me for this moment in my life. 😂
-
Daniël Lakens
@lakens
You can sign up until April 22 for the next Causal Inference and Variable Control workshop on May 9 by Peder Isager for the Paul Meehl Graduate School. Workshops are free, and fun, so sign up if interested! paulmeehlschool.github.io/workshops/causal/
-
Daniël Lakens reposted
Vincent Yzerbyt
@VincentYzerbyt
Fascinating pair of papers in which Daniël Lakens reflects on the similarities and differences between the two crises in social psychology. A must read ! @lakens @irsp_rips @EASP @SPSPnews
Daniël Lakens
@lakens
My paper on concerns about replicability, theorizing, relevance, generalizability, and methodology across 2 crises is now in press at @irsp_rips. After revisions it was 17500 words, so it is split in 2 parts: osf.io/dtvs7_v2 and osf.io/g6kja_v1
-
Daniël Lakens
@lakens
My paper on concerns about replicability, theorizing, relevance, generalizability, and methodology across 2 crises is now in press at @irsp_rips. After revisions it was 17500 words, so it is split in 2 parts: osf.io/dtvs7_v2 and osf.io/g6kja_v1
Post not available here, view on X
Daniël Lakens
@lakens
Why would we have success this time around? As you see, I am pessimistic our current efforts lead to noticeable improvements. We can improve our methods – this can be achieved by individuals. For the rest, we will need a more collective efforts, which are currently lacking.
Daniël Lakens
@lakens
If you want a detailed (and sometimes personal) complete history of the concerns people have raised concerning the quality of psychological science (much of which applies elsewhere) this might make for a good journal club read.
-
Daniël Lakens reposted
Gianmarco Altoè
@gianmarco_altoe
The call for applications to our PhD Program in Psychological Sciences is now open – find out how to apply!
#unipd -
Daniël Lakens reposted
Daniël Lakens
@lakens
Another study showing the huge uptake, and very favorable evaluations, of preregistration (this time in experimental economics). This is by now a very established result. Mainly the older generation who is negative about is. www.econstor.eu/bitstream/10419/315047/1/I4R-DP220.pdf 1/3
-
Daniël Lakens reposted
Psychometric Society
@pmetricsoc
At IMPS 2025, Jessica Flake (University of British Columbia) will give an invited talk on Methodological Research for the Open Science Era. For more about Dr. Flake: www.psychometricsociety.org/invited-speaker/jessica-flake…
#imps2025 #psychometrics #quantitativepsychology
@umn_edpsych -
Daniël Lakens
@lakens
If you want to attend the next Paul Meehl Graduate School course on Causal Inference, sign up: paulmeehlschool.github.io/2025-04-02-causal-announce/
-
Daniël Lakens
@lakens
Another great workshop, with a inspiring crowd of lovely early career researchers. Creating a free graduate school for metascience with @sajedeh_rasti has been some work, but every meeting is soooo worth it!
Daniël Lakens
@lakens
We are starting off with our workshop on Qualitative Open Science by Sarahanne Field for the Paul Meehl Graduate School for Metascience. Looking forward to learning about the state of the art in qualitative research by one of the leading experts on this topic!
-
Daniël Lakens reposted
Edmunds Vanags
@e_vanags
Inspiring, practical and brilliant as ever!
Daniël Lakens
@lakens
We are starting off with our workshop on Qualitative Open Science by Sarahanne Field for the Paul Meehl Graduate School for Metascience. Looking forward to learning about the state of the art in qualitative research by one of the leading experts on this topic!
-
Daniël Lakens
@lakens
Another study showing the huge uptake, and very favorable evaluations, of preregistration (this time in experimental economics). This is by now a very established result. Mainly the older generation who is negative about is. www.econstor.eu/bitstream/10419/315047/1/I4R-DP220.pdf 1/3
Post not available here, view on X
Daniël Lakens
@lakens
The authors write “Signaling credibility and meeting journal requirements emerged as the primary motives for pre-registering a study, while the need to explore the data and the time costs of pre-registration were among the main reasons against pre-registration.” 1 more post then:
Daniël Lakens
@lakens
I think at some point we need to move from taking this as a serious ‘reason’ to be against preregistration, and call these people out for being too incompetent on research methods to take their opinion seriously. As we wrote doi.org/10.1080/2833373X.2024.2376046
-
Daniël Lakens
@lakens
We are starting off with our workshop on Qualitative Open Science by Sarahanne Field for the Paul Meehl Graduate School for Metascience. Looking forward to learning about the state of the art in qualitative research by one of the leading experts on this topic!
-
Daniël Lakens reposted
Dr Theodoros M. Bampouras
@DrTheoBampouras
Thank you for a very interesting article
Daniël Lakens
@lakens
New paper by former lab visitor Max Ditroilo, "Exploratory research in sport and exercise science". We propose narrower definitions where hypothesis tests are confirmatory when error rates are controlled, and exploratory when not and give reporting advice www.tandfonline.com/doi/full/10.1080/02640414.2025.2486871
-
Daniël Lakens
@lakens
The long awaited results of the @BrRepNetwork are in! Their final sample consists of 97 replications of 47 studies. The only coherent measure of replication, p<.05, shows a replication rate of 19%. www.biorxiv.org/content/10.1101/2025.04.02.645026v2
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Check out PYMS Summer Symposium 2025.
Registrations & submissions are open now!
Join us on June 5th at Tilburg University for talks, a hackathon, dinner, and networking. Travel funds available!
Deadline: April 30th! Sign up: forms.gle/NtkeVLpk91qrmmW66 -
Daniël Lakens reposted
Mariane Bacelar
@bacelar_mariane
Discussed the preprint with my students in one of our lab meeting and they all loved it! So informative!
#openscience
Daniël Lakens
@lakens
New paper by former lab visitor Max Ditroilo, "Exploratory research in sport and exercise science". We propose narrower definitions where hypothesis tests are confirmatory when error rates are controlled, and exploratory when not and give reporting advice www.tandfonline.com/doi/full/10.1080/02640414.2025.2486871
-
Daniël Lakens
@lakens
New paper by former lab visitor Max Ditroilo, "Exploratory research in sport and exercise science". We propose narrower definitions where hypothesis tests are confirmatory when error rates are controlled, and exploratory when not and give reporting advice www.tandfonline.com/doi/full/10.1080/02640414.2025.2486871
-
Daniël Lakens reposted
Grant Abt
@grantabt
The latest from Max Ditroilo, Cris Mesquida, me, & @lakens on exploratory studies in sport/exercise science. Inter alia, we outline the differences between exploratory & confirmatory studies and best practice for conducting/reporting an exploratory study.
www.tandfonline.com/doi/full/10.1080/02640414.2025.248687… -
Daniël Lakens
@lakens
The Eindhoven Young Academy of Engineering has made a podcast on what 'excellence' in academia is, and why we need to take context into account. The first episode, on being an academic with challenging care duties, is a very important conversation to have!
open.spotify.com/episode/4S43XWb7E8hhAqLjxDU5vR -
Daniël Lakens reposted
Juan (el del blog) Fernández
@JuanEldelblog
Thanks to @SamuelPLeon I´ve discovered @lakens podcast: nulliusinverba.podbean.com
Great to understand the nuances of science.
Daniel, do you mind if I translate to Spanish and adapt (essentially making it shorter for teachers) this: lakens.github.io/statistical_inferences/16-confirmationbi… -
Daniël Lakens reposted
Daniël Lakens
@lakens
New episode of @nulliusverbapod ! We (very) critically discuss the classic paper by John Ioannidis 'Why most published research findings are false'. What did he get wrong, and why is it still interesting to read the paper? nulliusinverba.podbean.com/e/episode-56-most-published-re…
-
Daniël Lakens
@lakens
Episode 56: Cur Plerumque Investigation Publica Falsa Est nulliusinverba.podbean.com/e/episode-56-most-published-re…
-
Daniël Lakens
@lakens
New episode of @nulliusverbapod ! We (very) critically discuss the classic paper by John Ioannidis 'Why most published research findings are false'. What did he get wrong, and why is it still interesting to read the paper? nulliusinverba.podbean.com/e/episode-56-most-published-re…
-
Daniël Lakens reposted
Peder M Isager
@peder_isager
I'll finally get to teach a workshop on one of my favourite subjects - causal inference - at the Paul Meehl graduate school may 9th! Ever wondered what the point is of controlling for variables in your regression analysis? Come find out with us!
Daniël Lakens
@lakens
A new free workshop of the Paul Meehl Graduate School is open for registration! On May 9 @peder_isager will teach on Causal Inference and Variable Control. Check out the announcement below, and see you all @TUeindhoven
paulmeehlschool.github.io/2025-04-02-causal-announce/ -
Daniël Lakens
@lakens
A new free workshop of the Paul Meehl Graduate School is open for registration! On May 9 @peder_isager will teach on Causal Inference and Variable Control. Check out the announcement below, and see you all @TUeindhoven
paulmeehlschool.github.io/2025-04-02-causal-announce/ -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
The new workshop of Paul Meehl Graduate School, Causal Inference and Variable Control by Peder Isager, is open for registration. Check out the announcement below:
paulmeehlschool.github.io/2025-04-02-causal-announce/ -
Daniël Lakens reposted
ᴅʀ ᴍɪʀᴄᴇᴀ ᴢʟᴏᴛᴇᴀɴᴜ 🍂 🍃
@mzloteanu
#statstab #313 Analyzing Replication Studies
Thoughts: Replications and RR are on the rise, but not all are conducted appropriately, and become meaningless.
#stats #replication #metascience #openscience #metaresearch #metaanalysis
lakens.github.io/statistical_inferences/17-replication.html. -
Daniël Lakens
@lakens
Back from Berlin after our impromptu trip to meet up with metascientists and see the city. Had a blast!
-
Daniël Lakens
@lakens
It is so weird! People have been writing for 60 years about selective reporting, fishing for significance, and probability pyramiding.
You would think that if this behavior has been described for decades, it should have some effect on the replicability of claims people make?
🤔
Daniël Lakens
@lakens
A new prologue episode! Who first made the claim that more than 50% of published findings might not be true? It was Neher, 1967! We read his full paper as preparation for next week, where we will critically discuss Ioannidis, 2005, repeating this claim. nulliusinverba.podbean.com/e/prologus-56-probability-pyra…
-
Daniël Lakens reposted
Alex Chalmers
@chalmermagne
new from me: what went wrong with the Alan Turing Institute?
I dive into the failures in institutional design, focus, and leadership that led the UK's national AI institute to become irrelevant. It's a story of drift, stakeholderism, and wasted potential. 🧵 -
Daniël Lakens
@lakens
Thanks for linking to this! Very relevant to a preprint we will release soon, where we will criticize the idea of Type S and M errors, basically relegating their use to education (and even there, we will recommend more useful concepts).
ᴅʀ ᴍɪʀᴄᴇᴀ ᴢʟᴏᴛᴇᴀɴᴜ 🍂 🍃
@mzloteanu
#statstab #310 Directional (One-Sided) versus Non-Directional (Two-Sided) Tests
Thoughts: This topic is more nuanced than just power, but useful to know that you save on sample size *if* you're right!
#hypothesis #nhst #power #onetail #directionaltest
lakens.github.io/statistical_inferences/05-questions.html -
Daniël Lakens
@lakens
A new prologue episode! Who first made the claim that more than 50% of published findings might not be true? It was Neher, 1967! We read his full paper as preparation for next week, where we will critically discuss Ioannidis, 2005, repeating this claim. nulliusinverba.podbean.com/e/prologus-56-probability-pyra…
-
Daniël Lakens
@lakens
Prologus 56: Probability Pyramiding (A. Neher) nulliusinverba.podbean.com/e/prologus-56-probability-pyra…
-
Daniël Lakens reposted
Psicostat
@psicostat
We’re excited to share our April meeting! 🎉 Join us on April 4th for a talk with Simone Gastaldon on "Reanalyzing Published Science: The Power and Perils of Commentary Articles as an Early Career Researcher". 👥📢Don’t miss our next meetings and subscribe to the mailing list!
-
Daniël Lakens reposted
Daniël Lakens
@lakens
If we follow how many studies that have been registered on the OSF are eventually publicly shared, only 58% of studies are. This suggests a substantial filedrawer. Researchers indicate time management and null results as causes. Perhaps not efficient. doi.org/10.1177/25152459241296031
-
Daniël Lakens
@lakens
If we follow how many studies that have been registered on the OSF are eventually publicly shared, only 58% of studies are. This suggests a substantial filedrawer. Researchers indicate time management and null results as causes. Perhaps not efficient. doi.org/10.1177/25152459241296031
-
Daniël Lakens reposted
John V. Kane
@UptonOrwell
Indeed! I especially enjoyed this one. You both often force me to think about my own positions, and also get me thinking about the field of political science vis-a-vis psychology. Thank you!!! 🙏
p.s., at this point I am reasonably convinced that hypnosis is a real thing 😁 -
Daniël Lakens
@lakens
Went bouldering with @Noah_van_Dongen and Caspar van Lissa. Noah got a Veni grant this year, Caspar a Vidi, and I received a Vici, so we had to make this picture 😊 We will also work together in our projects on improving theorising in psychology!
-
Daniël Lakens
@lakens
I often say 'this episode is one of our best ones'. But our discussion on how psychology mirrors a pseudoscience, and how we can learn from pseudosciences such as parapsychology which tricks psychologists will use now they can not get away with p-hacking is really good.
Daniël Lakens
@lakens
New episode of @nulliusverbapod on Pseudoscience www.podbean.com/ew/pb-yggka-185819a We discuss the demarcation problem, why we should not ignore Pseudoscience but teach students why it is so bad, and complain about the too popular Telepathy Tapes.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New episode of @nulliusverbapod on Pseudoscience www.podbean.com/ew/pb-yggka-185819a We discuss the demarcation problem, why we should not ignore Pseudoscience but teach students why it is so bad, and complain about the too popular Telepathy Tapes.
-
Daniël Lakens
@lakens
This work is the result of @laabho 's visit to my lab while he was completing his PhD. We currently have 3 lab visitors, which is a lot of fun, and we are always open to hosting young scientists passionate about good science!
Jesper N. Wulff
@jesper_wulff
Paper with @laabho and @lakens and PDW with Sirio Lonati accepted for AOM. Copenhagen, here we come! #aom2025 @AOMConnect
-
Daniël Lakens
@lakens
We will soon post the corresponding preprint 'Blaming the Thermometer for the Fever: Separating Misapplication from Method in Null Hypothesis Significance Testing' which pushes back on the idea to ban error control in science by not allowing dichotomous tests.
Jesper N. Wulff
@jesper_wulff
Paper with @laabho and @lakens and PDW with Sirio Lonati accepted for AOM. Copenhagen, here we come! #aom2025 @AOMConnect
-
Daniël Lakens
@lakens
Joined in on the first qualitative interview @sajedeh_rasti is doing for her PhD, interviewing scientists involved in large scale coordinated projects. Incredibly interesting. Qualitative research and talking to scientists in other disciplines is providing a wealth of information
-
Daniël Lakens
@lakens
New record, a new release of old work by Hiroshi Yoshimura, FLORA. One of my favorite artists to put on while working.
-
Daniël Lakens
@lakens
This podcast episode so perfectly illustrates how preregistration allows others to evaluate the severity of a test. Authors opportunistically deviate. The analyses are clearly not reliable. Now we can see. Preregistration delivers on it promise - especially when - authors deviate
Two Psychologists Four Beers
@fourbeerspod
New episode! @minzlicht is back to talk about cannabis research, empathic AI, and some stuff we saw at SPSP.
share.fireside.fm/episode/Ah1OZyuo+DWffg-MH -
Daniël Lakens reposted
Bia
@BeatrizGietner
This episode is great
It’s about pseudoscience but first @lakens and @smreeteemehta define what science is and go a bit into the history of science -
Daniël Lakens
@lakens
New episode of @nulliusverbapod on Pseudoscience www.podbean.com/ew/pb-yggka-185819a We discuss the demarcation problem, why we should not ignore Pseudoscience but teach students why it is so bad, and complain about the too popular Telepathy Tapes.
-
Daniël Lakens
@lakens
Episode 55: Pseudoscientia nulliusinverba.podbean.com/e/pseudoscience/
-
Daniël Lakens reposted
Maximiliano Escaffi
@MaxEscaffi
🚀 New Publication! 🚀
Excited to share our latest article in International Journal of Psychology with @gempp & Dr. Julien P. Irmer:
📖 "Meta-Analysing the Factor Structure and Reliability of Measurement Instruments: An R-Based Tutorial" -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
You still have time to sign up for the "Open Qualitative Research" workshop by Sarahanne Field in Paul Meehl Graduate School.
Check the details here:
paulmeehlschool.github.io/workshops/qualitative/ -
Daniël Lakens reposted
Daniël Lakens
@lakens
New modeling paper that examines the effect of mandating preregistration, finds that "universally mandatory preregistration, compared to mere incentivization, promotes a more balanced coexistence of research methodologies" contingent upon incentive system. link.springer.com/article/10.1007/s11229-024-04828-0
-
Daniël Lakens reposted
Raphael Merz
@RaphaelMerz
🚀Very excited to finally see our paper on nonsignificance misinterpretations published! 📈
Together, @ste_lee_murphy, @aurelio_fdez, Linda Reimann and I investigated the prevalence of "p > .05 = absence of an effect" interpretations. (1/4)
doi.org/10.1098/rsos.242167 -
Daniël Lakens
@lakens
New modeling paper that examines the effect of mandating preregistration, finds that "universally mandatory preregistration, compared to mere incentivization, promotes a more balanced coexistence of research methodologies" contingent upon incentive system. link.springer.com/article/10.1007/s11229-024-04828-0
Daniël Lakens
@lakens
Without preregistration, labs that engage in HARKing will overwhelmingly dominate the landscape.
If preregistration is only incentives, conditions remain where HARKing continues to dominate.
Daniël Lakens
@lakens
I think here the model should have been improved, as not all papers are worth the same. Highly novel papers would be cited a lot more. Furthermore, in real life single researchers do a mix of safe and risky projects, not only either or.
Daniël Lakens
@lakens
As always, Registered. Reports are an even better solution. This is why critics of preregistration typically remain completely silent about Registered Reports, as they resolve most criticisms of preregistration AND combat publication bias.
Daniël Lakens
@lakens
It also warns against treating preregistered studies as intrinsically more valuable, which they are not, and reminds us that is not how preregistration should be used. It allows us to see how severely a hypothesis is tested. That is all.
Daniël Lakens
@lakens
Overall, the paper strongly dismisses many concerns raised against preregistration, points out it effectively prevents HARKing (when implemented well, which today it often is not) and shows any negative effects are due to the incentive system - which should also be fixed!
-
Daniël Lakens reposted
Daniël Lakens
@lakens
I am taking my lab and visiting PhD and master students for a 3 day trip to Berlin. If any metascientists there would like to join us for dinner March 31 or April 1, let me know. Let me know if you would like to meet up - especially if you are an early career researcher!
-
Daniël Lakens
@lakens
I am taking my lab and visiting PhD and master students for a 3 day trip to Berlin. If any metascientists there would like to join us for dinner March 31 or April 1, let me know. Let me know if you would like to meet up - especially if you are an early career researcher!
Daniël Lakens
@lakens
I could even squeeze in an impromptu talk or chat in the morning of the 1st of April. This would be early (around 9) and can be very small, it is always fun to meet fellow metascientists!
-
Daniël Lakens
@lakens
Was watching Vaiana 2 (or Moana 2 outside of Europe) and the statement by Loto, the engineer character in the movie, is such good advice for early career researchers!
'Perfection's a myth. There is only failing, then learning, then death'. -
Daniël Lakens
@lakens
Very good start of the new Metaror journal as the public peer reviews are as critical of @jamesheathers recent claim that 1 in 7 papers are fraud as I want my peer metascientists to be of such badly supported bold claims. Read the reviews here: metaror.org/kotahi/articles/18/index.html
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Check out the information about our first journal club of the year at PYMS. We gonna discuss the paper on intellectual humility by Hoekstra and Vazire, 2021.
youngmetascientists.github.io/journal-club-first -
Daniël Lakens reposted
Daniël Lakens
@lakens
Registered Reports are such an excellent solution to many of the challenges we face in knowledge generation in science. It is a shame you are not doing more of them.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
The goal of direct replication studies is first of all to identify Type 1 or Type 2 errors in the literature. This is an important part of having a reliable scientific knowledge base. lakens.github.io/statistical_inferences/17-replication.html
-
Daniël Lakens
@lakens
Registered Reports are such an excellent solution to many of the challenges we face in knowledge generation in science. It is a shame you are not doing more of them.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
New super cool paper by some of the meta-scientists that most shape my current thinking. osf.io/preprints/psyarxiv/t53np_v1
To be FAIR: Theory Specification Needs an Update -
Daniël Lakens
@lakens
There are 448 million people in the EU and 341 million Americans. I am hearing a lot of sentiments to stop buying American products and stop visiting the US. A good time to focus on strengthening science in Europe, store data with European providers and build European networks.
-
Daniël Lakens
@lakens
The goal of direct replication studies is first of all to identify Type 1 or Type 2 errors in the literature. This is an important part of having a reliable scientific knowledge base. lakens.github.io/statistical_inferences/17-replication.html
-
Daniël Lakens
@lakens
New super cool paper by some of the meta-scientists that most shape my current thinking. osf.io/preprints/psyarxiv/t53np_v1
To be FAIR: Theory Specification Needs an Update -
Daniël Lakens reposted
Nick DeVito
@NDevito1
Check out the OSIRIS Reproducibility Network!
osiris4r.eu/newsevents/2657/ -
Daniël Lakens
@lakens
New episode of @nulliusverbapod on conspiracy stories, out now! Enjoy! nulliusinverba.podbean.com/e/fabulae-coniurationis/
-
Daniël Lakens
@lakens
Episode 54: Fabulae Coniurationis nulliusinverba.podbean.com/e/fabulae-coniurationis/
-
Daniël Lakens reposted
Saint Javelin
@saintjavelin
The Dutch public broadcaster made a sketch on the current situation of the world
This media couldn't be displayed. See it on the original post. -
Daniël Lakens
@lakens
The Centre for Open Science is hosting a competition to predict replicability using AI or ML. www.cos.io/predicting-replicability-challenge I think such efforts are misguided. There is room for a philosopher of science to explain why empiricism can not be replaced by an algorithm.
-
Daniël Lakens reposted
Replication Network
@TheReplicateNet
ARTICLE OF INTEREST ("Decisions, Decisions, Decisions: An Ethnographic Study of Researcher Discretion in Practice” by van Drimmelen et al., 2024) replicationnetwork.com/2025/03/05/aoi-decisions-decisions…
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Registration is now open for our new workshop, Open Qualitative Research, at the Paul Meehl Graduate School. Check out the details here:
paulmeehlschool.github.io/workshops/qualitative/ -
Daniël Lakens
@lakens
I'm very grateful to have received a VICI grant. I'll work on the meaningful interpretation of effect sizes in psychology. It's an amazing opportunity to be able to create a team to work on this challenge in the next five years. www.nwo.nl/onderzoeksprogrammas/nwo-talentprogramma/proje…
-
Daniël Lakens reposted
Carlisle Rainey
@carlislerainey
My commentary on Isager, van ‘t Veer, and Lakens’ “Replication value as a function of citation impact and sample size” is now accepted at Meta-Psychology.
CC: @lakens @peder_isager
Preprint: osf.io/preprints/metaarxiv/8z45v_v2 -
Daniël Lakens reposted
Statistical Horizons
@StatHorizons
Are you a researcher burdened by costly and time-consuming data collection? Join @lakens for Sample Size Justification on March 19-21 to learn how to justify your sample size, make the most of your resources, and efficiently test your hypothesis.
-
Daniël Lakens reposted
Kert Viele
@KertViele
Berry Consultants now has a podcast...
www.berryconsultants.com/podcast
First episode live...
Challenges faced in pioneering adaptive trials
Influence of FDA approvals in adaptive trials
Impact of platform trials during COVID
Aspirations for the next 25 years of clinical trial design -
Daniël Lakens reposted
Open Access Tracking Project @oatp@fediscience.org
@oatp
An Inception-Cohort Study Quantifying How Many Registered Studies Are Publicly Shared - Eline N. F. Ensinck, Daniël Lakens, 2025 journals.sagepub.com/doi/10.1177/25152459241296031
-
Daniël Lakens
@lakens
Episode 53: Fraus - II nulliusinverba.podbean.com/e/episode-53-fraus-ii/
-
Daniël Lakens reposted
Psicostat
@psicostat
Day 4 Continues! ✈️
We keep the momentum going with @CalignanoGiulia exploring the power of the multiverse approach 🔀🌍, followed by @finos18432 diving into the challenges of inference in the multiverse. 🧮 -
Daniël Lakens
@lakens
Excellent and provocative lecture by @ulrichschimmack.bsky.social on measurement @ 4M winterschool. "It is hard to reward people for thinking."
"There is nothing wrong with exploring the data. How else will we discover new things? Especially when your crappy model does not fit." -
Daniël Lakens
@lakens
Too bad not everyone could make it to the 4M winterschool, would have loved to interact with some of the people who could not pick up their badges.... 😂
-
Daniël Lakens
@lakens
Listening to Lisa DeBruine teaching us how to organize large team science projects, with a deep dive into how to organize data and code into your own R package. Her book and videos are at: psyteachr.github.io/intro-r-pkgs/ Part of the great 4M winterschool by @psicostat.
-
Daniël Lakens reposted
Psicostat
@psicostat
Day 2 🚀
We continued our journey with an insightful session by @lakens , discussing the importance of Registered Reports.✨ Then, @JkayFlake took the stage diving into measurement not shmeasurement, challenging us to think critically about how we measure what matters ⚡️ -
Daniël Lakens reposted
Psicostat
@psicostat
We kicked off our Winter School on 4M’s with great enthusiasm! ❄️
It’s been an incredible experience so far, and we’re thrilled to have hosted amazing speakers like @lakens , @deevybee , and Carlo Umiltà, who shared their insights and expertise with us🚀
#winterschool #4ms -
Daniël Lakens
@lakens
"Replication is more like reliability, it is not like validity. It is not worth replicating research that is invalid."
Daniël Lakens
@lakens
At the wonderful 4M winterschool in Padova, listening to @JkayFlake talking about measurement and replicability. There are so many nice people here, incredibly fun week to interact with fellow metascientists and early career researchers!
-
Daniël Lakens
@lakens
At the wonderful 4M winterschool in Padova, listening to @JkayFlake talking about measurement and replicability. There are so many nice people here, incredibly fun week to interact with fellow metascientists and early career researchers!
-
Daniël Lakens reposted
mtliuzza 🇺🇦
@mtliuzza
@lakens explains how we can address the replication crisis by learning something from how we had addressed the ethics crisis in the past. 4M Winter school at @UniPadova organized by @psicostat
This media couldn't be displayed. See it on the original post. -
Daniël Lakens reposted
Peder M Isager
@peder_isager
Inspiring work by @sajedeh_rasti et al. on identifying the various benefits of coordination in science, and providing a better language to talk about what coordination in science actually *is*.
I personally long to be part of a truly coordinated research line.
Sajedeh Rasti
@sajedeh_rasti
knowledge generation system and highlighted seven ways in which scientific coordination provides epistemic benefits. I would love to hear your thoughts. You can find them here:
doi.org/10.31234/osf.io/eq269_v1
doi.org/10.31234/osf.io/vjcfk_v2 -
Daniël Lakens
@lakens
Thanks to @TomChivers and @StuartJRitchie for the shoutout to my coursera course in their podcast on deduction. We discussed deduction on @nulliusverbapod. If you want to learn why even Popper could not escape induction completely, listen to pca.st/episode/d8716274-0a37-444d-bfd5-2169378b981d
-
Daniël Lakens reposted
Jeroen de Ridder
@ridderjeroen
Vanaf nu overal te bestellen, binnenkort in de boekhandel: “Wat is nou waar? Zeven regels voor helder denken in verwarrende tijden.” Link met meer info hieronder.
-
Daniël Lakens
@lakens
3 new free workshops of the Paul Meehl Graduate School that you can sign up for!
April 11 – Dr. Sarahanne Field on qualitative studies
May 9 – Dr. Peder Isager on causal models
June 6 – Dr. René Bekkers on scientific criticism and peer review
See you there!
Sajedeh Rasti
@sajedeh_rasti
Mark your calenders.
The dates and topics of the upcoming workshops of the Paul Meehl Graduate School are announced.
Stay tuned for more information
paulmeehlschool.github.io/2025-02-10-save-date/ -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
Mark your calenders.
The dates and topics of the upcoming workshops of the Paul Meehl Graduate School are announced.
Stay tuned for more information
paulmeehlschool.github.io/2025-02-10-save-date/ -
Daniël Lakens reposted
Constantin Späth @cspaeth.bsky.social
@ConstantinSpath
This paragraph left the biggest impression
-
Daniël Lakens reposted
Gustav Nilsonne
@GustavNilsonne
Would you like to do a postdoc in metascience at @karolinskainst? I am now inviting candidates to develop joint proposals for @MSCActions and other programmes. For more information, please see:
docs.google.com/document/d/1T6iV9WoLYwUOIhD2tzDxqvpAPBggf… -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
knowledge generation system and highlighted seven ways in which scientific coordination provides epistemic benefits. I would love to hear your thoughts. You can find them here:
doi.org/10.31234/osf.io/eq269_v1
doi.org/10.31234/osf.io/vjcfk_v2 -
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
by identifying its key dimensions from the literature. We then showed how these dimensions can be applied to describe different levels of coordination in scientific projects. In "The Need for Scientific Coordination," we examined the shortcomings of the current
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
#preprint
I recently shared papers online on scientific coordination that I’ve been working on over the past year with @lakens and @KristVaesen. In "A Framework for Describing the Levels of Scientific Coordination," we clarified the concept of scientific coordination -
Daniël Lakens reposted
Chris Kavanagh
@C_Kavanagh
Attacking media & universities. Co-opting industry. Hollowing out the civil service and replacing with loyalists. Stacking courts with partisan judges. Undermining the validity of elections. Encouraging a personality cult. Ruling by decree. Things aren’t looking good for the US.
-
Daniël Lakens reposted
Nullius In Verba
@nulliusverbapod
New episode, the first of two, on Scientific fraud: nulliusinverba.podbean.com/e/episode-52-fraus-i/
-
Daniël Lakens
@lakens
Episode 52: Fraus - I nulliusinverba.podbean.com/e/episode-52-fraus-i/
-
Daniël Lakens reposted
FOS
@FOS_2025
We're thrilled to share the program for the FOS conference in Freiburg. We will hear a keynote on meta-theory by @lakens, incredible talks by ECRs, a workshop on replication by @priyasilverst and a panel discussion on theory-building (vs.) Open Science.
drive.google.com/file/d/1GvGhrj6l7lMzafLRCRb6AtJTb0QtqDEC… -
Daniël Lakens reposted
Daniël Lakens
@lakens
Although some researchers manage to perfectly adhere to preregistrations, it is common to need to deviate. Not all deviations are consequential, but some are, and it is important to evaluate the impact of deviations on the severity of a test. online.ucpress.edu/collabra/article/10/1/117094/200749
-
Daniël Lakens reposted
Ethan
@SEthanMilne
My job market paper is finally published in JMR!
I’ll do a larger thread on it soon, but for now here’s the link: journals.sagepub.com/doi/abs/10.1177/00222437251320021 -
Daniël Lakens
@lakens
Although some researchers manage to perfectly adhere to preregistrations, it is common to need to deviate. Not all deviations are consequential, but some are, and it is important to evaluate the impact of deviations on the severity of a test. online.ucpress.edu/collabra/article/10/1/117094/200749
-
Daniël Lakens
@lakens
Who would have guessed that open access and open science non-profits such as @internetarchive and @OSFramework are useful just in case your government want to delete scientific data?
And yet, here we are. -
Daniël Lakens
@lakens
"Norms are man-made in the sense that we must blame nobody but ourselves for them; neither nature, nor God. It is our business to improve them as much as we can, if we find that they are objectionable." Popper, 1945, The Open Society and Its Enemies. archive.org/details/in.ernet.dli.2015.59272/page/n9/mode/2up
-
Daniël Lakens
@lakens
Which findings should we replicate? Isager et al propose a 'replication value', defined as the expected utility gain when conducting a replication. It is a function of (1) the value of being certain about the claim, and (2) uncertainty about the claim. osf.io/preprints/metaarxiv/2gurz_v1
-
Daniël Lakens
@lakens
Our PhD student @andreakis_psy became a Dr. today after an excellent defense of her thesis on 'Surviving Academia'. It has been such a joy to be part of her supervisory team.
Here's a presentation she gave on her thesis last summer youtu.be/ve5wQSZ0zyA -
Daniël Lakens reposted
Michael Inzlicht
@minzlicht
Roy Baumeister called ego depletion "one of the most replicable findings in social psychology." As someone who spent 20 years studying it—and ultimately had to admit it wasn't real—I have to respectfully disagree. Here's my perspective of what went so wrong.
-
Daniël Lakens reposted
ISSID
@ISSIDOfficial
@lakens makes some
Important observations about the lack of preregistrations that go on to publication or even onto preprint servers.. more information in the post below.
Daniël Lakens
@lakens
We love to share new papers on here. But how many of the studies that scientists preregister on the Open Science Framework are never shared publicly? In a new paper in AMPPS we estimate 40% of preregistered studies are never shared. That’s a lot.
🧵⤵️
journals.sagepub.com/doi/10.1177/25152459241296031 -
Daniël Lakens reposted
Daniël Lakens
@lakens
We love to share new papers on here. But how many of the studies that scientists preregister on the Open Science Framework are never shared publicly? In a new paper in AMPPS we estimate 40% of preregistered studies are never shared. That’s a lot.
🧵⤵️
journals.sagepub.com/doi/10.1177/25152459241296031 -
Daniël Lakens
@lakens
We love to share new papers on here. But how many of the studies that scientists preregister on the Open Science Framework are never shared publicly? In a new paper in AMPPS we estimate 40% of preregistered studies are never shared. That’s a lot.
🧵⤵️
journals.sagepub.com/doi/10.1177/25152459241296031Post not available here, view on X
Daniël Lakens
@lakens
This could be a PhD student completing their PhD, but leaving academia, and no one was left to write up the work. Some papers do not pass peer review – but then are also not posted as a preprint. Why not? It was so much work, and sharing is so easy now!
Daniël Lakens
@lakens
We really hope this paper will make researchers think about how they are investing their time and effort. If 40% of the studies you preregister do not end up shared, why are we doing them? How can we prevent this, and be more efficient as researchers?
-
Daniël Lakens reposted
Nate Breznau
@BreznauNate
eeking people willing to talk about their usage of questionable research practices (QRPs) for panel #metascience2025. DM me if you would be a brave volunteer, willing to engage in discourse to understand how we as researchers internalize perverse incentive structures in science.
-
Daniël Lakens reposted
Koenfucius 🔍
@koenfucius
Ill-defined, poorly understood, hard to measure and often used to dismiss small but significant effects (especially when they’re inconvenient). @orbenamy and @lakens review the ‘crud factor’ and propose ways to address its problems: journals.sagepub.com/doi/pdf/10.1177/2515245920917961
HT @spencrgreenberg -
Daniël Lakens
@lakens
Is there a librarian who can tell me why a DOI of a paper (specifically doi.org/10.1177/0956797615585115) in Psychological Science would point to this weird version of the website: sage.cnpereading.com/paragraph/article/?doi=10.1177/09567… instead of just here: journals.sagepub.com/doi/10.1177/0956797615585115 Is it just a mistake?
Daniël Lakens
@lakens
Relatedly, why does this DOI doi.org/10.1177/0956797614547278 not resolve? I guess there are just quite obvious mistakes in the DOI database that no one cares to resolve? Is there a place where we can inform people about such mistakes?
-
Daniël Lakens
@lakens
In this special two-part celebration, we answer questions submitted by our listeners. Thanks to Don Moore, Leif Nelson, Henry Wyneken, Charlotte Pennington, and Karan Paranganat for the questions!
Episode 51: Quinquagesimus - II www.podbean.com/eas/pb-vegxm-17c5e41 -
Daniël Lakens
@lakens
Episode 51: Quinquagesimus - II nulliusinverba.podbean.com/e/fiftieth-ii/
-
Daniël Lakens reposted
Benjamin Spivak
@BenjaminSpivak
Is scientific relativism taken seriously by epistemologists? It still holds sway in some parts of the social sciences, but was wondering how people who specialise in epistemology see it.
-
Daniël Lakens reposted
Daniël Lakens
@lakens
People, just stop doing Frontiers special issues. Find a nice diamond open access journal, and self-publish and collaboratively peer review the set of papers you all want to write. Do not give Frontiers money. If anything, they hurt your reputation. MDPI is
even worse. -
Daniël Lakens
@lakens
People, just stop doing Frontiers special issues. Find a nice diamond open access journal, and self-publish and collaboratively peer review the set of papers you all want to write. Do not give Frontiers money. If anything, they hurt your reputation. MDPI is
even worse. -
Daniël Lakens
@lakens
The cyber attack at our uni is largely resolved! It only took a week (with some systems up earlier). Education has been postponed one week this and next quarter. But still a small consequence compared to e.g. the 2019 cyberattack at Maastricht. www.tue.nl/en/news-and-events/cyberattack
-
Daniël Lakens reposted
Sajedeh Rasti
@sajedeh_rasti
So excited to be part of this amazing conference hosted by @dwayne_lieck . Check more details here.
FOS
@FOS_2025
We’re excited to announce the Freiburg Open Science Conference to be held on the 15th & 16th of May 2025. We’re happy to welcome @lakens @priyasilverstein @psychedfreek & @sajedeh_rasti as our guests to talk about Theory Building and Open Science!
www.psychologie.uni-freiburg.de/abteilungen/Paedagogische… -
Daniël Lakens reposted
Daniël Lakens
@lakens
If you would like to read what I hope (and tried) to make the best (=most conceptually rigorous) review of preregistration and Registered Reports currently in the literature, see our recent paper: www.tandfonline.com/doi/full/10.1080/2833373X.2024.2376046
-
Daniël Lakens reposted
FOS
@FOS_2025
We’re excited to announce the Freiburg Open Science Conference to be held on the 15th & 16th of May 2025. We’re happy to welcome @lakens @priyasilverstein @psychedfreek & @sajedeh_rasti as our guests to talk about Theory Building and Open Science!
www.psychologie.uni-freiburg.de/abteilungen/Paedagogische… -
Daniël Lakens reposted
Dwayne Lieck
@dwayne_lieck
I'm very excited to host a conference in Freiburg with such amazing guests and support from my institute and research group!
FOS
@FOS_2025
We’re excited to announce the Freiburg Open Science Conference to be held on the 15th & 16th of May 2025. We’re happy to welcome @lakens @priyasilverstein @psychedfreek & @sajedeh_rasti as our guests to talk about Theory Building and Open Science!
www.psychologie.uni-freiburg.de/abteilungen/Paedagogische… -
Daniël Lakens
@lakens
Andrew Gelman on why preregistration is valuable. statmodeling.stat.columbia.edu/2025/01/16/why-i-like-prer… A more precise way to say this is: preregistrstion allows you to evaluate the severity of a test. It is important to understand this is the main aim of preregistration.
Daniël Lakens
@lakens
If you would like to read what I hope (and tried) to make the best (=most conceptually rigorous) review of preregistration and Registered Reports currently in the literature, see our recent paper: www.tandfonline.com/doi/full/10.1080/2833373X.2024.2376046
-
Daniël Lakens
@lakens
Obituary of James Coyne, @CoyneoftheRealm.
www.legacy.com/us/obituaries/theday/name/james-coyne-obit… -
Daniël Lakens
@lakens
After the topic of Phineas Gage came up in our podcast recording with @smreeteemehta, I checked, and his case is not just an anecdote, but was published in a medical journal in 1848: pmc.ncbi.nlm.nih.gov/articles/PMC9904626/pdf/nwmsj144365-…
-
Daniël Lakens
@lakens
Go to the site, hit the heart, and donate some money if you can. This has been such an amazing resource that has made my scientific work better by providing access to information (e.g. old books) that I would have otherwise never read.
Jason Scott
@textfiles
Guess what's back.
Yes, archive.org and openlibrary.org are back, as well as web.archive.org (Wayback Machine) and all those amazing items.
Some of the features are not up yet, but rest assured we're working on it and our tired devs will prevail. -
Daniël Lakens
@lakens
Group sequential analysis is a technique to collect data in batches, analyze the data after each batch, and decide to continue or not – all while controlling your Type 1 error rate. It provides a lot of flexibility, while maintaining rigour. >
-
Daniël Lakens
@lakens
I prioritise review requests based on how much journals adhere to open science principles. I will review for diamond open access journals, PCI, and journals that have pushed for open science practices before everything else.
John B. Holbein
@JohnHolbein1
How do you decide which referee requests to accept and which to reject?
-
Daniël Lakens
@lakens
Today at the Paul Meehl graduate school we have Lisa DeBruine teaching on advanced data simulation. And as an after lunch break we have Nick Brown talking about using simulation to detect 'errors' in research.
-
Daniël Lakens
@lakens
What are relatively simple things that you wish researchers would report, or would report better, that you have to keep pointing out in reviews?
-
Daniël Lakens
@lakens
You can now sign up for the 4th free in person workshop of the Paul Meehl Graduate school. Lisa DeBruine (@LisaDeBruine) will teach about advanced topics in data simulation! You can read more and sign up at: paulmeehlschool.github.io/News_fourth/
-
Daniël Lakens
@lakens
Your friendly reminder that we have tried to fix the theory crisis, practicality crisis, replicability crisis, and generalizability crisis since the 70's. Nothing is new about the current iteration, except some modest progress on replicability osf.io/dtvs7
-
Daniël Lakens
@lakens
I would like to understand the concept of a 'job-market paper' in economics. In many fields PhD's seem to be pushed to publish a lot. Is it correct things work a bit differently in economics? If so, how did this develop? I am happy to call or hear about it in the replies! 🙏
-
Post not available here, view on XPost not available here, view on X
Daniël Lakens
@lakens
And what a keynote by Catarina Dutilh-Novaes @cdutilhnovaes on mistakes in mathematical proofs! Such an intriguing topic! This is a must-watch. youtu.be/xIkh4vi0uNc
-
Daniël Lakens
@lakens
Really happy to announce the 2nd free workshop provided by the Paul Meehl Graduate School in Eindhoven: Theory Development by Dr. Noah van Dongen on March 27. Sign up through paulmeehlschool.github.io/news_second/ and add yourself to our new mailing list to stay updated about new workshops!
-
Daniël Lakens
@lakens
Full program online for our Workshop Perspectives on Scientific Error | 29 Feb, 1 and 2 March 2024! Program: docs.google.com/document/d/19sSFn7ea8KpfKjh6Ewp_AT0ZvudpP… Keynotes by @siminevazire @JudithterSchure @ianhussey @cdutilhnovaes
Registration (only 10 seats left): www.eventbrite.nl/e/workshop-perspectives-on-scientific-e… -
Daniël Lakens
@lakens
First mention of 'Publish or Perish' in the scientific literature comes from Case, 1927, Scholarship in Sociology. Yes, you read that right: 1927.
-
Daniël Lakens
@lakens
A topic that might deserve more attention is the probability that students commit scientific fraud while collecting data. This 1961 study found that when students were asked to complete a too complex experimental procedure a massive number committed fraud. pubmed.ncbi.nlm.nih.gov/13685460/
-
Post not available here, view on XPost not available here, view on X
Daniël Lakens
@lakens
I have never heard of him, but let's hope there is a younger generation to take over that work. Sometimes as professors age, they lose their wit. If that happens to me, please show me this tweet and tell me it is time for me to shut up about things I am clueless about.
-
Daniël Lakens
@lakens
Here is Rosenthal, 1966, on why scientists are harsh with scientists who commit fraud. According to Rosenthal, science is built on trust, and violating this trust can not be forgiven.
-
Daniël Lakens
@lakens
My textbook now includes an updated figure from Carter & McCullough, 2014. It's now generated from the raw data. It's a striking image. The uncorrected effect size is d = 0.62 based on 198 studies. Now, large replications later, we know the true effect size is 0. What a waste.
-
Daniël Lakens
@lakens
New preprint: An Inception Cohort Study Quantifying How Many Registered Studies are Published. We look at how many research studies are performed but remain unpublished after at least 4 years, and estimate this is 42% of registered studies. psyarxiv.com/5hkjz
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@lakens @deevybee @BorisBarbour @sharoz @hardsci @pdakean Key for this kind of collective action to work is the commitment of people with high follower counts and regular producers of content. Are you willing to do this for November? And, who else can you ask to commit?
Daniël Lakens
@lakens
Hi Brian, I am 100% with you and hereby commit to only post new content on Mastodon, and not on Twitter for all November. (I will limit myself to replies, such as these).
-
Daniël Lakens
@lakens
Being a Dungeon Master as a teenager prepares you surprisingly well for chairing academic meetings.
-
Daniël Lakens
@lakens
Very exciting - got my invitation to complete the National Dutch Survey on Research Integrity today! Such an important topic to learn how we can improve our science! If you get your invitation, do fill in the survey!
-
Daniël Lakens
@lakens
As I'm moving most of my writing from Word to R Markdown, stumbling across @writage made my life a lot easier! A Word plugin that allows you to save any old word file as a markdown file! Moving text from word to markdown couldn't be easier! www.writage.com
-
Daniël Lakens
@lakens
When writing a scientific paper, here is a mistake almost everyone (including me) makes: The goal of a theoretical introduction is not to explain how a theory predicts the observed data. The goal is to explain how without the theory the observed data would not be predicted.
-
Daniël Lakens
@lakens
Awesome paper by @RichardWiseman Caroline Watt & Diana Kornrot about the history of registered reports (from 1974!) in parapsychology (trailblazers!). So far, Registered Reports in parapsychology find 8.4% significant results (close enough to a 5% alpha level!) HT @ingorohlfing
-
Daniël Lakens
@lakens
This map also highlights why, if you are a researcher from a rich western country (e.g., The Netherlands, USA, Scandinavia), you might not be in the best position to realize how difficult it is to get access to the scientific literature and how important #openaccess is.
-
Daniël Lakens
@lakens
Very funny talk by @MikeKSmith about the alpha wars and p-values: youtu.be/snoeQYR1ErQ?t=10m10s "To me, this [redefining statistical significance] is like a Jar-Jar Binks argument: It might seem like a good idea when you first propose it...." #justifyalpha
-
Daniël Lakens
@lakens
Heading over to Utrecht today to talk about replication and falsification and listen to @FredHasselman @JelteWicherts and EJ Wagemakers in a James Coleman symposium in honor of @MALMvanAssen getting a prof in mathematical sociology chair at @UtrechtUni .
-
Daniël Lakens
@lakens
I'm looking for a post-doc on a project aimed at increasing the reliability and efficiency of psychological science (V39.3285). 75% research, 25% teaching, for 4 years. Please RT. jobs.tue.nl/nl/vacature/postdoc-on-increasing-the-reliabi…
-
Daniël Lakens
@lakens
15-16 February 2018 2-day Reflections on Replications workshop in Utrecht, NL, organized by @ivanflis with @epskamp, @JelteWicherts , Klaus Fielder, me (see all speakers: roar.sites.uu.nl/speakers/) - free registration: roar.sites.uu.nl/registration/
-
Daniël Lakens
@lakens
Great paper recommendation (by a blog reader): Deconstructing Statistical Questions by David Hand on the importance of learning how to ask statistical questions. Great read, so much goodness: statlab.bio5.org/sites/default/files/fall2014/hand-decons…
-
Daniël Lakens
@lakens
New Preprint: Justify Your Alpha: A Response to “Redefine Statistical Significance” psyarxiv.com/9s3y6
-
Daniël Lakens
@lakens
Now @JelteWicherts gives 34 ways to p-hack your data* (based on journal.frontiersin.org/article/10.3389/fpsyg.2016.01832/…). *Can also be used to prevent p-hacking #icps17vie
-
Daniël Lakens
@lakens
King's College London refuses to share trial data with @CoyneoftheRealm . You will want to read the reasons why: dl.dropboxusercontent.com/u/23608059/PACE F325-15 - Prof.…
-
Daniël Lakens
@lakens
New blog: The relation between p-values and P(H0|D) is not weak enough to ban p-values. daniellakens.blogspot.nl/2015/11/the-relation-between-p-v…
-
Regina Nuzzo
@ReginaNuzzo
Researchers: Which fallacies do you see/experience most? I'm betting Texas Sharpshooter. @NatureNews #science #bias ow.ly/i/dA7Yu
Daniël Lakens
@lakens
@BenAllenDingo @ReginaNuzzo People are being called out. Follow for example @CoyneoftheRealm @Keith_Laws @R__INDEX @StuartJRitchie
-
Daniël Lakens
@lakens
Is It Ever Correct to Use Null-Hypothesis Significance Tests? Of Course It Is. (Meehl, 1990, p. 137). rhowell.ba.ttu.edu/Meehl1.pdf













































































































































































































































































