Same
Brian Nosek
Brian Arthur Nosek is an American social-cognitive psychologist, professor of psychology at the University of Virginia, and the co-founder and director of the Center for Open Science. He also co-founded the Society for the Improvement of Psychological Science and Project Implicit. He has been on the faculty of the University of Virginia since 2002. More on Wikipedia
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
My Jewish spouse loves Christmas. Okay, fine, who doesn’t?
Converting to a “Valentine’s tree” seemed a bit excessive.
Came home from a trip tonight. Dear god, how do I make it stop?Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I will not bend to this obvious manipulation to win my affection.
I cannot.
I… must… not.
Einstein Foundation
@Einstein_Berlin
The €200,000 Institutional #einsteinfoundationaward goes to the Center for Open Science @OSFramework in Charlottesville, VA for its promotion to drive change in global research culture by providing scholars the tools and infrastructure to make #openscience the default.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I… I just…
There’s no escape.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In case you were wondering, it never ended. I am just pretending that it did, for my own sake.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Naturalistic experiment making the most of rigor and transparency with the Registered Report format plus open data, materials, and code.
Congrats to Jordan (what is the statute of limitations for my taking credit for former lab members’ work?) & Aaron for leading a fine project.
Jordan Axt
@jordanaxt
New article in Nature Human Behavior co-led by Aaron Nichols! In a Registered Report, we ran a field study to test whether images of organizational diversity impact the quality and quantity of applicants from majority and minority groups.
www.nature.com/articles/s41562-023-01731-5 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
My new Bluesky account is bsky.app/profile/briannosek.bsky.social
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Zoé Ziani, Ph.D.
@ZoeZiani
Yes... I had to censor my work without which my committee wouldn't sign it... But the great thing with Internet is that you can post the original version and even add a disclaimer explaining why it differs from the published version…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There is a remarkable difference in tone and insight about Gino and her work between a NYT piece that did not appear engage with any co-authors, and a New Yorker piece that did.
NYT: www.nytimes.com/2023/09/30/business/the-harvard-professor…
New Yorker: www.newyorker.com/magazine/2023/10/09/they-studied-dishon… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This great New Yorker piece reveals Zoe as one of the early career researchers who identified problems in Gino's data and acted.
If we want a more just research culture, those of us with position and power must stand and support those without it, in word and deed.
Thank you Zoe
Zoé Ziani, Ph.D.
@ZoeZiani
Gideon Lewis-Kraus interviewed me about my role in the Ariely - Gino scandal. I think it is excellent work!
www.newyorker.com/magazine/2023/10/09/they-studied-dishon… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An important model posits two routes to persuasion. A peripheral route relies on superficial cues of credibility; a central route involves depth consideration of the reasons and evidence.
The Gino arguments seem to bank on readers sticking with the peripheral route.
Stephanie M. Lee
@stephaniemlee
Francesca Gino just launched a new website to publicize her lawsuit. She also (I believe this is new) posted a response to Data Colada’s analysis of her study in the 2012 signing-at-the-top paper, which she claims shows they “cherry-picked” data. www.francesca-v-harvard.org/data-colada-post-1
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Kirsten Morehouse
@kirst_morehouse
Do you collect demographic info about your participants? Do you share open data? If so, check out this preprint w/ @BenedekKurdi & @BrianNosek: osf.io/preprints/metaarxiv/5m3cx We provide a pipeline for assessing & addressing re-identification risk (identifying subjects w/o direct ids) 1/
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Center for Open Science celebrated its 10th anniversary with an event at the National Academies. We told the story of reforming the research culture toward openness, integrity, and reproducibility.
This thread provides some highlights. 1/
www.cos.io/blog/cos-celebrates-10-years -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preregistered studies can & should be critiqued for how they present and interpret planned vs unplanned analyses.
Non-preregistered studies can't & should be critiqued for how they present and interpret planned vs unplanned analyses.
The 2nd is more common and less transparent. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Imagine being head of recruiting for Stanford baseball. 64 games, 7 away weekends + the conference tournament.
Thursday morning departure.
Sunday night or Monday afternoon return.
Better hope to get a bunch of Sat doubleheaders instead of F/S/S games
gostanford.com/sports/baseball/schedule/2023
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Condolences to Cal and Stanford's baseball, softball, and olympic sport student athletes who will lose multiple additional school days to cross-country travel demands.
Being a student athlete is hard enough with regional travel demands. Sheesh.
AP Top 25
@AP_Top25
BREAKING: The Atlantic Coast Conference has cleared the way for Stanford, California and SMU to join the league, two people with knowledge of the decision told the AP.
apne.ws/oDsHg2T -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Condolences to Cal and Stanford's baseball, softball, and olympic sport student athletes who will lose multiple additional school days to cross-country travel demands.
Being a student athlete is hard enough with regional travel demands. Sheesh.
AP Top 25
@AP_Top25
BREAKING: The Atlantic Coast Conference has cleared the way for Stanford, California and SMU to join the league, two people with knowledge of the decision told the AP.
apne.ws/oDsHg2T -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"The GoFundMe response has, right now, given us the freedom to make decisions about how to proceed in this case without worrying about how much our legal fees will accrue. We can pursue what's right instead of what’s most affordable."
-
Brian Nosek (@briannosek@nerdculture.de) reposted
METRICStanford
@METRICStanford
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Anna Harvey
@annalilharvey
@NIH Acting Director Larry Tabak recently said, "We'll have to have the discipline to stop supporting underpowered boutique studies."
We suggest spending that $$ instead on "master protocols” designed to produce reliable evidence about a smaller set of interventions ⬇️:
SSRC
@ssrc_org
SSRC President @annalilharvey shares a proposal to @NIH_CommonFund to improve the reliability of behavioral evidence, borrowing from oncology:
Master protocols—coordinated multi-site trials & meta-analyses—evaluating interventions across contexts.
items.ssrc.org/from-the-presidents-desk/master-protocols-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A summary of the Many Co-Authors project to audit the findings of Gino-led projects: behavioralscientist.org/amid-uncertainty-about-francesca-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @OSFramework has many users from the humanities, including historians, making their work more open. Also, check out the qualitative data repository: data.qdr.syr.edu
Dr Anton Howes
@antonhowes
I think history has some serious problems when it comes to self-correction. False "facts" spread rapidly and can last for decades.
My modest suggestion is to systematically spread access to archival sources. My *hope* is for an Open History movement. open.substack.com/pub/antonhowes/p/age-of-invention-does-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Daughter, age 9: I figured out how to make my live-forever machine. I’ll get jellyfish DNA and mix it w/chemicals... I’m going to test it on <bully at school>.
Me: What if it works?
Daughter: Ugh. I need a new plan.
[A favorite from the memory box.] -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Major improvements to OSF search released today.
In a few click you can, for example, find [click 1] all registrations [2] that used the OSF Preregistration template, [3] that have data linked, and [4] that link a paper.
Check it out: osf.io/search
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
With recent metadata enhancements, fields such as funder, resource types, institutions, and subjects will get richer and richer for search every week.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Also, if your institution is a member of OSF Institutions, the metadata linking your institution to user content dramatically improves its discoverability.
More information for signing up here (forward to your library!): www.cos.io/products/osf-institutions -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The terms for researchers examining potential violations of research integrity are bad (data police, data thug).
What is an alternative term that:
* doesn't imply anything about the intentions of the people who did the work being investigated (deliberate or accidental)
more>>
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
* doesn't imply the researcher has any power over the people/work being investigated
* does indicate that the investigative work is scholarly, and should be valued
* does indicate that "self-appointment" to investigate is appropriate & ordinary -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I will probably have one published article in 2023 from earlier work. Simultaneously, my current research may be the most important and impactful work that I will ever do. It takes time to do things well.
Three cheers for crock-pot science.
[Different than crack-pot science.] -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Reproducibility Project: Psychology was published eight years ago today.
Feels like a lifetime.
Center for Open Science
@OSFramework
#reproducibilityproject published:100 psych replications observe effects 1/2 size of originals www.sciencemag.org/lookup/doi/10.1126/science.aac4716
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@lakens offers his reflections on the project, and the events surrounding it, in the latest episode of his podcast: nulliusinverba.podbean.com/e/new-replication-crisis/
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
To #reproduciblityproject (osf.io/ezcuj/wiki/home) team and supporters. Proud of doing and finishing this. Cheers.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
SIPS (Account Archived - See Bio)
@improvingpsych
We are very pleased to announce the 2023 Mission Awards Winners for the Society for the Improvement of Psychological Science! SIPS Mission Awards recognize and increase awareness of a limited number of exemplary products that further the SIPS mission. improvingpsych.org/2023/08/22/sips-2023-awards-announced/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How did they figure us out so fast?
I am rethinking my position about doing your own research.
Jikkyleaks 🐭
@Jikkyleaks
BOOM! 💥💥💥💥
The organisation that is behind the fund raiser for #datacolada is....
Drum Roll...
DARPA (the very people that funded GoF research).
Simine Vazire was the primary fundraiser, working on behalf of SCORE.
You literally can't make this stuff up. -
Brian Nosek (@briannosek@nerdculture.de) reposted
SORTEE
@sortecoevo
Nominations for the 2023 SORTEE Awards are open now. This year we have two (redesigned) award categories:
1⃣Student Award
2⃣Researcher Award
For details please see: www.sortee.org/awards/
Submit your nominations via: forms.gle/fez14JDVx2FcGWFp7
➡️Deadline: 1st Oct 2023 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
For the curious, the explanation for this appears to be an extreme version of self-publishing and self-citation. When the self-publishing stopped, so did citations to their work.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Caption this.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
And, in an instant, 2,000 psychologists bookmarked this tweet to share back with @lakens next year, next month, tomorrow, forevermore.
Daniël Lakens
@lakens
I have never heard of him, but let's hope there is a younger generation to take over that work. Sometimes as professors age, they lose their wit. If that happens to me, please show me this tweet and tell me it is time for me to shut up about things I am clueless about.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Caption this.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An urge not quite strong enough to last through the next sentence.
Lessig 🇺🇦
@lessig
On the charges against HBS' Francesca Gino link.medium.com/2xXKNeKKmCb
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The response has been profoundly moving. It demonstrates both the support for Data Colada, and the need for greater institutionalization of aid for whistleblowers and integrity seekers in science. The swell of assistance for this case may create the space for bigger solutions.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @DataColada team has been sued by Francesca Gino for exposing the fraudulent data underlying four of her papers (she claims defamation). The legal defense could get expensive. Please join us in providing financial support for their defense.
www.gofundme.com/f/uhbka-support-data-coladas-legal-defense -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
We've met our goal 🎉
3 reasons to still donate:
1. For just $1, you'll boost the # of donors. That message to Joe, Leif, & Uri, and to the world, is as important as the financial support
2. Our target was a guess about likely legal costs
3. Money that's not needed'll be refunded
simine vazire
@siminevazire
Be a part of the change you wish to see in the world:
Donate to help support the legal expenses of Data Colada.
They're being sued by Prof Gino after posting careful, meticulous blog posts raising concerns about data integrity in 4 papers.
Donate here: gofund.me/70c497b1 -
Brian Nosek (@briannosek@nerdculture.de) reposted
JK Flake 📈📏
@JkayFlake
WE DID IT. WE RAISED 25OK. Y'ALL GIVING ME A TEAR. WE'LL UPDATE THE PAGE SOON WITH MORE INFO ABOUT WHAT'S NEXT. ❤
simine vazire
@siminevazire
Data Colada legal defense fundraiser data show a steady stream of donations around the clock. This is a truly international show of support across all time zones! 🌍 🌏 🌎 🤩
Donate here: gofund.me/58491686 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Uri, Leif, and Joe: Ouch, this one hurts. We might need to ask for some help.
Research community:
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @DataColada team has been sued by Francesca Gino for exposing the fraudulent data underlying four of her papers (she claims defamation). The legal defense could get expensive. Please join us in providing financial support for their defense.
www.gofundme.com/f/uhbka-support-data-coladas-legal-defense -
Brian Nosek (@briannosek@nerdculture.de) reposted
Chris Chambers
@chrisdc77
Have donated. Leif, Joe and Uri have taken a huge personal risk to stand up for good science, so let's make sure they don't stand alone.
If you can donate even $1 to their legal fund please do. Let's show our support.
simine vazire
@siminevazire
Be a part of the change you wish to see in the world:
Donate to help support the legal expenses of Data Colada.
They're being sued by Prof Gino after posting careful, meticulous blog posts raising concerns about data integrity in 4 papers.
Donate here: gofund.me/70c497b1 -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Over 500 people have donated in less than 10 hours. Amazing. Let’s make it 1,000 people, to show Joe, Uri, and Leif that we’re in this together.
Donate here: gofund.me/58491686
simine vazire
@siminevazire
Be a part of the change you wish to see in the world:
Donate to help support the legal expenses of Data Colada.
They're being sued by Prof Gino after posting careful, meticulous blog posts raising concerns about data integrity in 4 papers.
Donate here: gofund.me/70c497b1 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More than $110k from almost 750 donors in 12 hours. That's an amazing show of support for @DataColada.
Your named or anonymous donation of $5, $50, or $500 is a meaningful, concrete show of support for research integrity.
Let's get to 2,000 donors.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @DataColada team has been sued by Francesca Gino for exposing the fraudulent data underlying four of her papers (she claims defamation). The legal defense could get expensive. Please join us in providing financial support for their defense.
www.gofundme.com/f/uhbka-support-data-coladas-legal-defense -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Be a part of the change you wish to see in the world:
Donate to help support the legal expenses of Data Colada.
They're being sued by Prof Gino after posting careful, meticulous blog posts raising concerns about data integrity in 4 papers.
Donate here: gofund.me/70c497b1 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @DataColada team has been sued by Francesca Gino for exposing the fraudulent data underlying four of her papers (she claims defamation). The legal defense could get expensive. Please join us in providing financial support for their defense.
www.gofundme.com/f/uhbka-support-data-coladas-legal-defense -
Brian Nosek (@briannosek@nerdculture.de) reposted
Stuart Buck
@stuartbuck1
It's Saturday night, and I'll probably post about this again, but I just wrote up well over 20k words on my personal history as a metascience venture capitalist (so to speak).
-
Brian Nosek (@briannosek@nerdculture.de) reposted
FORRT
@FORRTproject
We have good news to share 🥳🎉
We partnered up with the Center for Open Science @OSFramework and their educational initiative the Open Science Knowledge Base @OSKB_COS who graciously chose FORRT to host their curated resources in OS, increasing our database by a whopping 60%!🙌
OSKB_COS
@OSKB_COS
OKSB is excited to announce our partnership with @FORRTproject - head over to the blog to learn more and get involved! www.cos.io/blog/forrt-and-cos-join-forces-to-foster-open-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I have publicly stated my belief that the studies from the Gino papers were based on fraudulent data. I would like to be wrong about this. My LinkedIn post is my commitment to retract that claim if compelling non-fraud alternative explanations emerge.
www.linkedin.com/feed/update/urn:li:activity:709261570693… -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Powerful people using sexism as a shield :(
I'm sure Gino & Cuddy are targets of sexism & worse. But scholarly criticism of their work, eg Data Colada's, isn't sexism. Saying it is undermines efforts to combat actual sexism & erodes our field's credibility by silencing criticism.
Stephanie M. Lee
@stephaniemlee
Hard to condense (you can read the story for free), but Gino says she was treated unfairly by Harvard's investigation and by Data Colada, in part because of gender. She says Data Colada has “targeted the work of prominent female academics,” e.g. Amy Cuddy. www.chronicle.com/article/scholar-accused-of-research-fra…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Stephanie M. Lee
@stephaniemlee
Francesca Gino's defamation lawsuit against Harvard and Data Colada is 100 pages, so there's a lot to go through here. Here's my story summarizing it. Harvard and the three professors behind Data Colada declined to comment for this.
www.chronicle.com/article/scholar-accused-of-research-fra… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Bold" hypothesis: This will backfire.
Daniel Engber
@danengber
Francesca Gino has filed a $25 million defamation lawsuit against Harvard University and the Data Colada bloggers (@uri_sohn , Leif Nelson, @jpsimmon).
We've updated our story on her 2012 paper from this morning:
www.theatlantic.com/science/archive/2023/08/gino-ariely-d…Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
These seem transparently non-serious (p. 52-61). Is it the whole substance of the rebuttals, or is it filler content for the purpose of filing the suit and the actual rebuttals to the evidence would be more substantive?
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The responses to Francesca's LinkedIn post about it are quite interesting for content and profiles of who are and are not voicing support. www.linkedin.com/posts/francescagino_i-want-to-be-very-cl…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Veronica Boyce
@veroboyce
New preprint with @mcxfrank and Maya Mathur: Eleven years of student replication projects provide evidence on the correlates of replicability in psychology (psyarxiv.com/dpyn6/) reporting 176 replications from students in a grad methods class @StanfordPsych. (1/5)
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Hartford Insurance on data that was shared with Dan Ariely: "It is clear the data was manipulated inappropriately and supplemented by synthesized or fabricated data."
Full story and statement from Hartford insurance: www.npr.org/2023/07/27/1190568472/dan-ariely-francesca-gi… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brooke N. Macnamara
@BrookeMacnamara
Our systematic review & pre-registered meta-analysis of growth mindset interventions on academic achievement psycnet.apa.org/record/2023-14088-001 and our reply to commentaries is now out in Psychological Bulletin.
A (long) thread. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
Our Executive Director @BrianNosek discusses the need for evidence in new approaches to science philanthropy. Read more from @Philanthropy: www.philanthropy.com/news/quick-grants-from-tech-billiona….
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Nidhi Subbaraman
@NidhiSubs
NEW: Sadly, many happiness studies are flawed
Happily, in the last decade psychologists have been reforming the way they design studies
www.wsj.com/articles/sadly-many-happiness-studies-are-fla… w @DunnHappyLab @DuniganFolk @BrianNosek @slyubomirsky -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Found out.
Michael McDonald
@ElectProject
One-third of New College's faculty are not returning for the fall semester. The school has been destroyed, along with the future dreams of its students www.tampabay.com/news/education/2023/07/18/new-college-fl…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"It identified a culture where Tessier-Lavigne “tended to reward the ‘winners’ (that is, postdocs who could generate favorable results) and marginalize or diminish the ‘losers’ (that is, postdocs who were unable or struggled to generate such data).”"
Theo Baker
@tab_delete
BREAKING: Stanford president to resign his post, retract “at least” 3 papers, after our reporting in @StanfordDaily exposed multiple papers with data manipulation and his refusal to retract studies when given opportunities over the course of 2 decades.
stanforddaily.com/2023/07/19/stanford-president-resigns-o… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If you read @michael_nielsen and @kanjun's piece about metascience entrepreneurship and thought "I want to do that," then here's an opportunity for you!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@AsteraInstitute launches a call for Science Entrepreneurs-in-Residence and a RFI for unblocking scientific publishing.
astera.org/science-opportunities/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@AsteraInstitute launches a call for Science Entrepreneurs-in-Residence and a RFI for unblocking scientific publishing.
astera.org/science-opportunities/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How do you know that a book is grounded in evidence? It has 63 pages of endnotes citing the sources. I am enjoying Nobody’s Fool by @profsimons and @cfchabris for evening bar reading, but you can read it wherever.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Daniel Storey
@danielstorey85
No spoilers, but this is the cleverest football advert I've ever seen.
This media couldn't be displayed. See it on the original post. -
Brian Nosek (@briannosek@nerdculture.de) reposted
CERN
@CERN
This week, over 100 international experts from 70 different institutes in five continents got together to reflect on how scientific organisations can accelerate the adoption of #openscience.
The event, hosted by #cern and @NASA, put the spotlight on how Open Science reshapes the landscape of research and discovery, fosters a global exchange of knowledge for a better and more equitable future, and unlocks limitless scientific possibilities through collaboration.
Stay tuned for the full report about the event. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Ryan McGranaghan
@AeroSciengineer
(2/2) Submit your abstract and join us for “Flourishing Science Commons: Data Science, Open Science, and Knowledge Communities” (studio.m-anage.com/agu/fm23/prelim.cgi/Session/191306)
@NASATOPS @OSFramework @theAGU @NCAR_Science @ChelleGentemann @BrianNosek @RebeccaMorss @HFHarvard @GreaterGoodSC @turingway -
Brian Nosek (@briannosek@nerdculture.de) reposted
Ryan McGranaghan
@AeroSciengineer
(1/2) An uncommon opportunity at this year's AGU Fall Meeting to be a part of the frontier forum for the discussion, action, and cutting-edge research into the new approaches to science and collaboration that are required to respond to our grandest questions
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Threads account www.threads.net/@nosek_brian
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How about universities set their standards for eligibility for admission such that it enables a diverse pool, invest their resources in outreach for applicants, and then select the admits by lottery?
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Data Colada
@DataColada
Data Falsificada Part 4 - "Forgetting the Words"
Data tampering in this JPSP 2020 article is revealed by a variable that was collected but not analyzed, and thus left behind when the other variables were tampered with
datacolada.org/112 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Postulate: A substantial portion of twitter fracas are dueling reasonable points based on different premises.
Example: many arguments about events are based on different assumptions or knowledge about what occurred.
Hard to have productive discussion w/out shared basic facts. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Yoel Inbar
@yorl
I've seen some misinformation floating around about the UCLA grad student letter. To be clear, I didn't post it on Twitter and don't support doing so with the signatories' names attached, because I don't want anyone to be doxxed or harassed.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Ruben C. Arslan
@rubenarslan
the OSF api connection exists already in formr.org if anyone wants to submit a pull request to automatically deposit data and establish a chain of custody.
you could even include an R script to anonymise data on-the-fly. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Josh de Leeuw
@joshdeleeuw
I built a tool that any data collection software can use to do this with OSF:
psyarxiv.com/fv65z/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An outstanding episode unpacking the experience that @yorl had with a job interview at UCLA. Lots of productive disagreement while unpacking the complexities of authenticity versus risk avoidance, and performative versus evidence-based ways to advance our values. Start at 41:30
Very Bad Wizards
@verybadwizards
it's up! A VBW exclusive report - @yorl joins @peez and @tamler to talk about his experience applying to UCLA only to have his candidacy pulled at the last minute because of remarks made on his podcast(!) about diversity statements. Plus more fraud verybadwizards.com/episode/episode-263-free-yoel
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Or listen to the whole thing -- the first half on the Gino fraud case is also very good.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Relating to the unknowns identified in the episode, Val indicates that the decision about not hiring was made before the committee received the letter from a group of students summarizing their concerns.
x.com/VLaughmaker/status/1674167531626704896 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Very Bad Wizards
@verybadwizards
it's up! A VBW exclusive report - @yorl joins @peez and @tamler to talk about his experience applying to UCLA only to have his candidacy pulled at the last minute because of remarks made on his podcast(!) about diversity statements. Plus more fraud verybadwizards.com/episode/episode-263-free-yoel
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Dan Honig (DanHonig.bsky)
@rambletastic
If I'm honest I was frightened when I first heard our paper was being formally replicated. Then I wasn't, because of how @I4Replication engaged us. More generally I think we need a climate where replication is not 'policing' rather 'let's make better science together'. Thanks!
I4R
@I4Replication
First DP this week: A replication analysis of @JamesBisbee and @rambletastic (2022) "Flight to Safety: COVID-Induced Changes in the Intensity of Status Quo Preference and Voting Behavior".
This DP (econpapers.repec.org/paper/zbwi4rdps/36.htm) is from @alicemalmberg1 and @danielscates97 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Well, at least they opted for .0005 (Benjamin et al, 2018).
Ed Coulson
@coulsoncommaed
I hardly know what to make of this. This is an actual question asked in an actual submission to an actual journal.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
This #sips2023 was so amazing! Such an energetic, vibrant group of people. And so well-organized!
Huge thanks to everyone who made it happen, especially someone who you might not know did soooooo much from afar, ✨@katiecorker ✨, and @BalazsAczel, Crystal, & many others! -
Brian Nosek (@briannosek@nerdculture.de) reposted
Joe Simmons
@jpsimmon
Part 3 of Data Falsificada is now up on Data Colada: "The Cheaters Are Out of Order". datacolada.org/111
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Stephanie M. Lee
@stephaniemlee
PSA: You can read this story for free by entering your email. You can also log in through your university or subscribe. Thanks for reading/supporting. www.chronicle.com/article/a-dishonesty-expert-stands-accu…
(Also, if you want to know when I publish new stories, you can subscribe to stephaniemlee.substack.com.) -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The latest, in-depth update.
Stephanie M. Lee
@stephaniemlee
NEW: Following data-fraud allegations against Francesca Gino, a Harvard expert in dishonesty research, scholars who worked with her are shaken.
We talked to some about whether they stand by their research and what working with her was like.
www.chronicle.com/article/a-dishonesty-expert-stands-accu… @nellgluckman -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
📣 Don't miss our upcoming webinar on July 12 at 11 am ET featuring @FORRTproject!
Join us to learn more about open and reproducible research training, with a focus on big-team science approaches for teachers and researchers.
Learn more & register: cos-io.zoom.us/webinar/register/WN_juXhoar6Tu2Nbwq5hOCOSw -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Some fraudsters eventually land in villas on the grounds of 7-star hotels. www.nytimes.com/2023/06/21/opinion/cloning-science-fraud.…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Original study, positive result.
N=101; cited 563 times (20 times in 2023 post retraction)
Replication study, null result.
N=3,232,430; cited 47 times (4 times in 2023)
A replication 3000x more powerful is <1x as influential.
Stewart Kettle
@Stew_Ket
The scale of this academic fraud is outrageous - two trials in one paper! It also had very real-world impacts. We replicated these findings in a large trial with the Guatemalan Tax Authority doi.org/10.3390/bs7020028 which produced a (v. precisely estimated) null effect.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Pardon me, >30,000x not 3,000x. Missed a whole order of magnitude.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Dan Quintana
@dsquintana
A historical review of publication bias onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1452
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Joe Simmons
@jpsimmon
That self-appointed data police vigilante is Steve Haroz (@sharoz). VERY much appreciate his catching this mistake and notifying us.
Uri Simonsohn
@uri_sohn
Some vigilante from the self appointed data police downloaded the posted data, tried to reproduce our figure and found an error. Arrow was pointing at the observation in wrong condition for 'harvard' lower case.
We have corrected it
datacolada.org/110
Very sorry. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
To what extent does open data increase the probability of detecting fraud?
And, what proportion of fraud is undetectable even with open data?
Uri Simonsohn
@uri_sohn
Post 2 in the series on Data Falsificada.
Plus, a message to Gino's 148 co-authors
(which include me)
datacolada.org/110 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Uri Simonsohn
@uri_sohn
Post 2 in the series on Data Falsificada.
Plus, a message to Gino's 148 co-authors
(which include me)
datacolada.org/110 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There are 330660 .xls or .xlsx files on @OSFramework.
In June, 6805 were added, 1236 were updated, and 1348 were deleted.
The Figure shows the daily ratio of removed (blue) or removed+updated (red) versus the number added ("normalizing" usage). The Table shows the raw data.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
To my eye, the leap of updated files on June 17th is more likely random variation in daily usage rather than evidence of insights about excel metadata resulting in some OSF users revisiting their posted excel files. I'd have expected to see increases in deletion especially.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In fact, June 17th was the slowest day on OSF for adding or removing excel files for the entire month.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Daniël Lakens
@lakens
Here is Rosenthal, 1966, on why scientists are harsh with scientists who commit fraud. According to Rosenthal, science is built on trust, and violating this trust can not be forgiven.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Why is the reaction to fraud so strong?
Because scholarship as a social system relies on trust.
Even the most ardent critic of a topic is trusting that the evidence they are critiquing is genuine. If they can't even trust that, then scholarship as a social system falls apart.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If one cannot trust that fraud is rare or absent, then there is no sensible basis to further consider whether research claims are coherent, reliable, valid...
If scholarship is for truth-seeking, then the scholar shouldn't have to wonder if they are in the fiction section.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Transparency is a partial solution. But even with radical transparency, there are always actions by research producers that are unobservable to research consumers.
Ultimately, trust is needed for that unobservable gap.
We aspire to make the unobservable gap as small as possible
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
When a critic of research gets angry when they find out that the research was fraudulent, it signals that they were taking it seriously.
They were criticizing the quality of the evidence or the link between the evidence and claims. Scholarly work.
But it wasn't evidence at all. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Perhaps the worst part of fraud cases is all of the collateral damage, on the field, on people who tried to build on the work, and on direct collaborators.
Juliana's response is the best kind of collaborator response to short-circuit knee-jerk criticism and affirm integrity aims -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A commitment to transparency by researchers, journals, funders, and institutions will not prevent fraud. But, it will make fraud more inconvenient and easier to detect.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Katy Milkman
@katy_milkman
If you’re in or adjacent to the field of #behavioralscience & aren’t talking about the new evidence of data fraud by @HarvardHBS ⭐️ Prof. @francescagino, read this immediately 🤢👇🏻 @DataColada We have to do better. datacolada.org/109
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It is hard to say whether the work on fraud or the work on false-positive psych is a more important contribution from Uri, Leif, and Joe, but the field owes them a debt for both.
My gratitude is deepest for the fraud work bc it is profoundly important and not rewarding for them.
Joe Simmons
@jpsimmon
In 2021, a team of us sent a report to Harvard University detailing evidence of fraud in 4 papers (spanning 11 yrs) co-authored by HBS professor Francesca Gino. This is the 1st in a series of Data Colada posts describing that evidence. datacolada.org/109
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Joe Simmons
@jpsimmon
In 2021, a team of us sent a report to Harvard University detailing evidence of fraud in 4 papers (spanning 11 yrs) co-authored by HBS professor Francesca Gino. This is the 1st in a series of Data Colada posts describing that evidence. datacolada.org/109
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A disquieting read. ($) www.chronicle.com/article/a-weird-research-misconduct-sca…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
ICYMI: We just celebrated our 10th anniversary! Check out the sessions from our anniversary event, showcasing our theory of change to start, scale, and sustain new behavior, norms, incentives, and policies that foster openness, rigor, and reproducibility. www.cos.io/blog/cos-celebrates-10-years
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Doing better leadership science via replications and registered reports"
Introduction to a special issue of The Leadership Quarterly
www.sciencedirect.com/science/article/abs/pii/S1048984323… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A team science replication project for priming effects on judgments and behavior is recruiting collaborators.
Context: www.dropbox.com/s/3upsq116x62c0px/Project Description Rep…
Sign-up: insead.eu.qualtrics.com/jfe/form/SV_0d2zpeVprIhjFXw -
Brian Nosek (@briannosek@nerdculture.de) reposted
TranspariMED
@TranspariMED
New meta-meta-research study finds that most meta-research studies are flawed.
Not joking.
www.sciencedirect.com/science/article/abs/pii/S0895435623… @Strech_Da @LGHemkens @NaudetFlorian @NDevito1 @dirnagl @GustavNilsonne @Nathalie_PdS @ukrepro
h/t @sharpmelk -
Brian Nosek (@briannosek@nerdculture.de) reposted
YF Yang- @ufangyang.bsky.social
@ufangyang
Curious about Chinese researchers in #openscience?
Join us - Open Science Across Geographies Speaker Series @OpenSciChina @QingWan99752335
📅 June 14 | 8:30am ET
Discover insights from publishing, government, and policy experts.
Don't miss out! Sign up now👇 @OSFramework -
Brian Nosek (@briannosek@nerdculture.de) reposted
Joanna Diong
@joannadiong
NIH toughens enforcement of delayed clinical trials reporting | Science | AAAS www.science.org/content/article/nih-toughens-enforcement-…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
German Reproducibility Network
@GermanRepro
📢📢Meta-Research Summer School 📢📢
Want to know more about research on research? Join our Meta-research Summer School in Frankfurt am Main (August 28 - September 1, 2023). Apply here by June 16: www.bihealth.org/en/notices/meta-science-summer-school-de… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Heather Joseph
@hjoseph
NSF releases plan in response to Nelson Memo: "The NSF public access plan recognizes that open and equitable communication of research results and data is inextricably linked to its mission of promoting scientific progress…" #openaccess #openscience nsf-gov-resources.nsf.gov/2023-06/NSF23104.pdf?VersionId=…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Did you just finish your undergrad or graduate thesis?
Share your scholarship on thesiscommons.org
Thesis gets a DOI, discoverable on Google Scholar and elsewhere. Especially useful for those considering grad school and those finishing grad school and wanting to share. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Jordan Dworkin
@jddwor
I’m excited to announce the launch of “Sabbaticals in Service” — a new @MetascienceWG pilot program fostering the exchange of ideas between academic researchers and federal agencies
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Metascientists, consider preregistering your work in the Metascience registry and help that registry set the aspirational standard for high rates of result reporting.
Link: osf.io/registries/metascience/discover
Daniël Lakens
@lakens
New preprint: An Inception Cohort Study Quantifying How Many Registered Studies are Published. We look at how many research studies are performed but remain unpublished after at least 4 years, and estimate this is 42% of registered studies. psyarxiv.com/5hkjz
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is important work and establishes a baseline for benchmarking the effectiveness of improvements to OSF to encourage result reporting of all registered studies.
Thanks to Eline Ensinck and @lakens for pursuing this!
Daniël Lakens
@lakens
New preprint: An Inception Cohort Study Quantifying How Many Registered Studies are Published. We look at how many research studies are performed but remain unpublished after at least 4 years, and estimate this is 42% of registered studies. psyarxiv.com/5hkjz
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Related: If you are a metascientist and want to use OSF data for research, start with the OSF API developer.osf.io. And, if you cannot extract the kind of data you need for your research, don't hesitate to send wish list ideas for how OSF data could be more useful to you.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
CEIRRNetwork
@CEIRRNetwork
With support from @TheFluLab, @OSFramework is launching a pilot Open Science Community Trainer Certification Program. Apply by June 16 to promote open science adoption in #influenza research! Gain skills and recognition as an open science advocate. cos.io/flulab
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Delighted to be attending #iced2023 to present on improving the research culture.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A career arc. Thank you to everyone that lifted me up with opportunities, mentorship, and collaboration.
If not for all of you, I'd still be circling in place at the beginning, goatee included.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Or, maybe even regressed back to the mother-provided bowl cut...
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The #metascience2023 presentations are now publicly available!
www.youtube.com/playlist?list=PLChfyH8TVDGlPTcKrVboBaf6QA…
Lots of great talks examining how science is being done and how it can improve.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Also, don't miss the symposium:
"Starting, scaling, and sustaining improvements to research credibility and trustworthiness"
www.youtube.com/playlist?list=PLChfyH8TVDGmWyGePxToZ4ZcNi… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If you like our “culture change pyramid” watch the videos linked in this thread to get the full story on the theory of change.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Center for Open Science celebrated its 10th anniversary with an event at the National Academies. We told the story of reforming the research culture toward openness, integrity, and reproducibility.
This thread provides some highlights. 1/
www.cos.io/blog/cos-celebrates-10-years -
Brian Nosek (@briannosek@nerdculture.de) reposted
Chris Chambers
@chrisdc77
Nominate me for chief editor if you'd like to see a journal fit for the future including:
-Registered Reports, Exploratory Reports, Verification Reports, Results-blind review, TOP Level 2+
-Close ties to preprints & preprint-based review, inc. @PeerCommunityIn & @PCI_RegReports
Gilad Feldman
@giladfeldman
Please write APS at editorsearch@psychologicalscience.org and nominate Simine Vazire and Chris Chambers for Chief Editor of Psychological Science.
We should try and support open-science supportive editors for such roles.
Maybe, just maybe, if enough people write in... -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Wish me luck! I’m going for Psych Science editor in chief again. Here’s my vision statement from when I applied in 2019, if you want to see what kind of stuff I’d push for: sometimesimwrong.typepad.com/wrong/2019/06/hibeic.html
Gilad Feldman
@giladfeldman
Please write APS at editorsearch@psychologicalscience.org and nominate Simine Vazire and Chris Chambers for Chief Editor of Psychological Science.
We should try and support open-science supportive editors for such roles.
Maybe, just maybe, if enough people write in... -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Center for Open Science celebrated its 10th anniversary with an event at the National Academies. We told the story of reforming the research culture toward openness, integrity, and reproducibility.
This thread provides some highlights. 1/
www.cos.io/blog/cos-celebrates-10-yearsPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Chapter 4: @nicipfeif talked about how user-centered design is critical, and identified several risks that must be managed to accelerate adoption and ensure that the technological solutions are fit-for-purpose for the research domains.
36:00-end: www.youtube.com/watch?v=A25ZvRggkn8
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Chapter 5: @HuajinBioLib talked about the role of norms to move from early adopters into the mainstream. She pointed out how support for open scholarship is higher than people perceive it to be.
0-26:00: www.youtube.com/watch?v=1JYWFr93Qmo
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Chapter 5: @katiecorker gave the history of @improvingpsych as a case example of formation and impact of grassroots communities fostering new norms and standards.
26:00-40:00: www.youtube.com/watch?v=1JYWFr93QmoPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Chapter 7: @pebourne talked about reform from the perspective of the Dean of a new School of Data Science @UVA, and the opportunity of starting from scratch for setting norms and policies for open scholarship right from the start.
25:25-end: www.youtube.com/watch?v=B4a1EnkzcaI
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Conclusion: I closed with a personal story for why reforming the research culture toward greater openness, integrity, & reproducibility matters.
And, I provided a snapshot of evidence that--while there is a long way to go--the movement is scaling.
15-min www.youtube.com/watch?v=rSOLwNdAr1E -
Brian Nosek (@briannosek@nerdculture.de) reposted
SORTEE
@sortecoevo
Evidence from psychology shows that @RegReports have a lower positive result rate than the standard literature & likely reduce #publicationbias & QRPs
📰by @annemscheel Mitchell Schijen & @lakens doi.org/10.1177/25152459211007467
Time for eco & evo to support @RegReports! -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
What is open science/open scholarship?
In this 8-minute kick-off to COS's anniversary event, I offered our team's perspective that advancing openness, integrity, and reproducibility requires embracing a broad conception:
Lifecycle Open Science
1/
www.youtube.com/watch?v=Pn0IBuXgn5Q&list=PLChfyH8TVDGmWyG… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
What is open science/open scholarship?
In this 8-minute kick-off to COS's anniversary event, I offered our team's perspective that advancing openness, integrity, and reproducibility requires embracing a broad conception:
Lifecycle Open Science
1/
www.youtube.com/watch?v=Pn0IBuXgn5Q&list=PLChfyH8TVDGmWyG…Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Lifecycle Open Science recognizes that there are many things that can be opened that support three qualities of "open" -- transparency, sharing, and inclusion.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Lifecycle Open Science recognizes that these activities occur across the research lifecycle, and that improving the quality of research will not be accomplished by merely doing "retrospective" open science.
The open movement is big, diverse, and about the whole lifecycle! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
19% of 755 phase 3 cancer trials had changes to the primary end points in the paper compared to the registration.
70% of those were not disclosed in the paper.
Changes were associated with likelihood of a positive outcome (OR=1.86).
jamanetwork.com/journals/jamanetworkopen/fullarticle/2805005
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
As disappointing as this is, none of it would have been detected without trial registration.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
These challenges are not just occurring in late stage trials. There are also reproducibility and replicability challenges in the pre-clinical work, creating friction in the entire pipeline.
elifesciences.org/articles/71601 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Chris Chambers
@chrisdc77
A perfect illustration of why clinical trial registration needs an upgrade to a full #registeredreports model. @RegReports
There are some logistical challenges, but they are solvable. The alternative is this👇
cc @eturnermd1
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
19% of 755 phase 3 cancer trials had changes to the primary end points in the paper compared to the registration.
70% of those were not disclosed in the paper.
Changes were associated with likelihood of a positive outcome (OR=1.86).
jamanetwork.com/journals/jamanetworkopen/fullarticle/2805005 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This intuitively feels wrong because the research is so obviously important and consequential.
Please consider your own research to be important and consequential, and embrace open science accordingly.
If it is worth doing, it is worth doing well.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
19% of 755 phase 3 cancer trials had changes to the primary end points in the paper compared to the registration.
70% of those were not disclosed in the paper.
Changes were associated with likelihood of a positive outcome (OR=1.86).
jamanetwork.com/journals/jamanetworkopen/fullarticle/2805005 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
G7 leaders affirm support for open science and FAIR principles.
www.whitehouse.gov/briefing-room/statements-releases/2023…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thank you to all contributors at @WHOSTP45 @alondra for making this happen!
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Dr. Priya Silverstein
@priyasilverst
"A Guide for Social Science Journal Editors on Easing into Open Science" -- now preprinted! 🥳
A mammoth effort, with contributions from a wide variety of editors, metascientists, and open science experts across the social sciences.
osf.io/hstcx -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If you love teaching about open scholarship and want to do it full-time, check out this key open position at COS!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New Position @OSFramework: Training and Education Manager
Love open scholarship?
Love to educate?
We need you to lead a key part of our work.
Perfect if you want the best parts of academic teaching blended with a mission to improve the research culture
jobs.lever.co/cos/00a5e1ac-3c58-4b70-8b97-bccd1c3a655a -
Brian Nosek (@briannosek@nerdculture.de) reposted
Alfredo Sánchez-Tójar
@ASanchez_Tojar
📢📢📢 Big News!
Rolling out @RegReports at @NatureEcoEvo. That includes #metaanalysis and analyses of secondary data!!! 🥳🎉🙌 #openscience #evidencesynthesis #systematicreview
More at: www.nature.com/articles/s41559-023-02076-y -
Brian Nosek (@briannosek@nerdculture.de) reposted
Marcus Munafò
@MarcusMunafo
It's ten years (!) since @ButtonKate and myself (w many others) published "Power Failure: why small sample size undermines the reliability of neuroscience" in @NatRevNeurosci. Today, we have the @CommonsSITC report on reproducibility. www.nature.com/articles/nrn3475
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A prominent theme at #metascience2023 and COS's 10-year anniversary was the need for promotion and tenure reform. @emckiernan13 mentioned University of Maryland's Psych department as an exemplar case. Here's a webinar with Michael Dougherty about it: www.youtube.com/watch?v=D8FruoOpMmA
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Science, Innovation and Technology Committee
@CommonsSITC
Our report on Reproducibility and Research Integrity is out today.
📰Read the news story: committees.parliament.uk/committee/135/science-innovation…
📜Read the full report: publications.parliament.uk/pa/cm5803/cmselect/cmsctech/10… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Gabriel Costa
@gabrielgcnut
Showing my update of John Ioannidis' paper of highly cited clinical research to Tim @OSFramework using more modern criteria defined by him and @BrianNosek to define what a replication is. @BrRepInitiative Olavo and Kleber as coauthors/advisors. Thanks for your insights Tim.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
John Arnold
@johnarnold
In just one decade @OSFramework has changed the culture of scientific research by defining and quantifying the replication crisis; building the technological infrastructure; and moving the ecosystem of funders, researchers and institutions to improve research practices. 2/2
-
Brian Nosek (@briannosek@nerdculture.de) reposted
John Arnold
@johnarnold
Proud to celebrate 10th anniversary of the Center for Open Science. Launched and supported by grants from @Arnold_Ventures, COS works to increase the openness, integrity, and reproducibility of research to move the incentives away from finding novel results to finding truth. 1/2
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Charlotte Pennington
@drcpennington
In July 2021, the UK @HouseofCommons launched an inquiry into the reproducibility of research. Evidence was submitted on whether there was a ‘reproducibility crisis’, the roles of different stakeholders in addressing this, & the measures required for openness & rigour. A🧵(1/X):
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
#metascience2023 some people from the meeting are on the 3rd floor here.
Tonic At Quigley's, 2036 G St NW, Washington, DC 20036 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Emily Sena kicks off #metascience2023 noting that she has a new title: Professor of Metascience and Translational Medicine.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Yesterday @OSFramework celebrated its 10th anniversary. Today, #metascience2023 begins.
We are so grateful to be part of a community of change agents that are United in a mission to improve research. Thank you for your advocacy, action, and collaboration. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
We're just a few hours away from the start of the Metascience 2023 Conference! Whether you're attending in person or virtually, join the conversation by using #metascience2023.
ℹ️ Agenda: metascience.info/agenda
💻 Virtual Registration: www.eventbrite.com/e/metascience-2023-virtual-attendance-… -
Brian Nosek (@briannosek@nerdculture.de) reposted
SORTEE
@sortecoevo
📢Reminder📢
Content submissions are still open for the SORTEE conference!
To submit a content proposal (i.e. unconference, hackathon, workshop), please go to: docs.google.com/forms/d/e/1FAIpQLSfrTc4oYrakvD0LSCd5vze2v……
Deadline: June 1st
More at: sortee.org/upcoming/ #sortee2023
SORTEE
@sortecoevo
📢 SAVE THE DATE for our 3rd SORTEE conference: #sortee2023
17-18th Oct 2023: it will run for 24h continuously, starting and finishing at 0700 UTC
Registration & content submission (unconferences, hackathons, workshops) will soon open.
Keep an 👁️ on: www.sortee.org/upcoming -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Global drive for more open, rigorous research is growing www.universityworldnews.com/post.php?story=2023050512562269
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Today, @OSFramework celebrates its 10th anniversary with an event at the National Academies. Our sister repository, @ZENODO_ORG, is celebrating theirs today too!
Thank you supporters, researchers, and change agents that made these repositories a success.
www.cos.io/timeline
Zenodo
@ZENODO_ORG
Today is our 10th anniversary 🎉 Use #zenodo10years to celebrate with us by sharing YOUR story on how Zenodo helped YOU! #openscience @CERN @OpenAIRE_eu zenodo.org/10years
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Alfredo Sánchez-Tójar
@ASanchez_Tojar
Sunday ride around beautiful Bielefeld!
Today is the last day to register for early bird prices and/or submit your abstract to our symposium:
"Meta-research for behavioural ecology: it's time!"
for @2023Behaviour!
Submission: uni-bielefeld.de/fakultaeten/biologie/forschung/veranstal… -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
On our way to #metascience2023 🎉
@fidlerfm @SchiavoneSays -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Faculty looking for a change and recent PhDs, take note!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New Position @OSFramework: Training and Education Manager
Love open scholarship?
Love to educate?
We need you to lead a key part of our work.
Perfect if you want the best parts of academic teaching blended with a mission to improve the research culture
jobs.lever.co/cos/00a5e1ac-3c58-4b70-8b97-bccd1c3a655a -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
One week left until #metascience2023! If you’re unable to join us in person, we are excited to announce that we are offering a free virtual streaming option. Streaming sponsored by @AsteraInstitute.
Agenda: metascience.info/agenda
Virtual Registration: www.eventbrite.com/e/metascience-2023-virtual-attendance-… -
Brian Nosek (@briannosek@nerdculture.de) reposted
BITSS
@UCBITSS
BITSS is extremely excited to be attending #metascience2023 Conference next week, May 9-10, in Washington, DC! There's still time to register to attend and we look forward to chatting with folks in-person: metascience2023.org
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Public Knowledge Project (PKP)
@pkp
Now released! @OASPA webinar, transcripts, slides and notes on "Scholarly Communication in Crisis: Research Integrity and Open Scholarship"
Panelists include @deevybee @ClearSkiesAdam & @BrianNosek Chaired by @catmacOA
oaspa.org/scholarly-communication-in-crisis-research-inte…
#scholcomm #publishing #ethics -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New Position @OSFramework: Training and Education Manager
Love open scholarship?
Love to educate?
We need you to lead a key part of our work.
Perfect if you want the best parts of academic teaching blended with a mission to improve the research culture
jobs.lever.co/cos/00a5e1ac-3c58-4b70-8b97-bccd1c3a655a
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
See also, important engineering openings on our jobs page: www.cos.io/careers
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Registration is now open for the live feed for #metascience2023 on May 9-10 for those that cannot join in person in Washington DC.
If you can join in-person, register now for one of the remaining openings!
metascience2023.org -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Registration is now open for the live feed for #metascience2023 on May 9-10 for those that cannot join in person in Washington DC.
If you can join in-person, register now for one of the remaining openings!
metascience2023.org -
Brian Nosek (@briannosek@nerdculture.de) reposted
Thom Bridge
@thom_g_bridge
@ZoAndBehold works from the lunch counter outside the House of Representatives chamber today because a group of women have been occupying the bench she used last week. #mtleg
-
Brian Nosek (@briannosek@nerdculture.de) reposted
platibus
@the_platibus
same vibes
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Three more virtual symposia for #metascience2023., one on each of the next three days and the in-person meeting is next week in DC!
metascience2023.org
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: The awesome line-up of virtual sessions for the lead-up to #metascience2023 is live. These sessions are free, but you have to register:
metascience.info/virtual-symposia/
And, don't miss the in-person meeting May 9-10: metascience.info/agenda/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Dan Goodman
@neuralreckoning
Starting in an hour and a half. I'll be talking briefly about what (I think) is wrong with standard (pre-publication) peer review, and how we could be doing it better, but most of the fun will be in the discussions.
Dr Elizabeth Gadd @lizziegadd.bsky.social
@LizzieGadd
Academic Journals are Broken: Let's Build a Better Scientific Record. Come and join @deevybee @jamesheathers @neuralreckoning @chrisdc77 @brembs & myself unpack the problems and the hope...
2 May 12BST. Sorry for the short notice...
metascience.info/virtual_symposia/academic-journals-are-b… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Three more virtual symposia for #metascience2023., one on each of the next three days and the in-person meeting is next week in DC!
metascience2023.org
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: The awesome line-up of virtual sessions for the lead-up to #metascience2023 is live. These sessions are free, but you have to register:
metascience.info/virtual-symposia/
And, don't miss the in-person meeting May 9-10: metascience.info/agenda/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Stuart Ritchie
@StuartJRitchie
How many anti-semaglutide articles has the Guardian published now? More than a dozen I think, attacking it from every spurious angle possible. It’s always weird to be reminded that your reasoning (to me it seems obviously a positive thing) is wildly different from someone else’s:
Guardian Opinion
@guardianopinion
A 'skinny jab' is no quick fix for obesity - and no excuse to let junk food companies off the hook | Sarah Boseley www.theguardian.com/commentisfree/2023/apr/25/skinny-jab-…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
The free pre-conference virtual symposia for the #metascience2023 Conference will continue through May 4, 2023!
Check out the sessions and register: metascience.info/virtual-symposia/.
Florian Naudet
@NaudetFlorian
Last minute challenge: talking at this online event tomorrow as part of the MetaScience 2023 conference (@OSFramework) "Promoting Research Integrity Through Meta Research in the Publication Process." This virtual symposia is supported by @RoRInstitute.
metascience.info/virtual_symposia/promoting-research-inte… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Sven E. Hug
@hug_sven
There is a new and exciting alternative to Google Scholar:
OpenAlex, the second largest database after Google Scholar, launched the beta version of its web GUI today.
Go check it out!
Congratulations to the OpenAlex team! 🥳 👏
alpha.openalex.org/works -
Brian Nosek (@briannosek@nerdculture.de) reposted
Steve Haroz 📊👁️🧠
@sharoz
A paper on open practices is being published in #chi2023. But unlike the #openscience it promotes, it does not share its data. What could be an excellent discussion of research practices is instead a poor example of scrutinizability. A call for openness:
steveharoz.com/blog/2023/scrutiny-for-thee-but-not-for-me… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Transparency and Openness Promotion (TOP) Guidelines advisory board has an open call for new members.
See here for more information to apply: www.cos.io/blog/call-for-volunteers
Nominate yourself or others.
Help us increase representativeness of the board: www.cos.io/initiatives/top-guidelines -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preregistration enthusiasts: Apply for COS's Preregistration Template Evaluation Working Group to help select and evaluate new templates for OSF.
We are very interested to have a diverse working group by career stage, discipline, region, and identities.
www.cos.io/blog/call-for-volunteers
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A systematic review preregistration template is now available on @OSFramework, the first of several community-developed templates to be added.
Key contributions from @DenOlmo @matherion in conceiving the template & @katiecorker getting it included.
More: osf.io/preprints/metaarxiv/3nbea/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Transparency and Openness Promotion (TOP) Guidelines advisory board has an open call for new members.
See here for more information to apply: www.cos.io/blog/call-for-volunteers
Nominate yourself or others.
Help us increase representativeness of the board: www.cos.io/initiatives/top-guidelines -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preregistration enthusiasts: Apply for COS's Preregistration Template Evaluation Working Group to help select and evaluate new templates for OSF.
We are very interested to have a diverse working group by career stage, discipline, region, and identities.
www.cos.io/blog/call-for-volunteers
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A systematic review preregistration template is now available on @OSFramework, the first of several community-developed templates to be added.
Key contributions from @DenOlmo @matherion in conceiving the template & @katiecorker getting it included.
More: osf.io/preprints/metaarxiv/3nbea/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Pavithran Narayanan 👨🏻🔬🌱🧬🔬
@pavi_narayanan
"Academic Journals are Broken. Let’s Build a Better Scientific Record"
- A free, virtual symposium from @OSFramework's "Metascience 2023" conference with some interesting topics! @PeerCommunityIn @ASAPbio_ @cshperspectives @thecrispress @ReviewCommons metascience.info/virtual_symposia/academic-journals-are-b… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Metascience 2023 is less than three weeks away.
Register now to engage with the latest creative visioning, evidence, and advocacy for improving and accelerating research and discovery.
And, don't miss the virtual pre-conference events already underway!
metascience.info -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New position opening at the Center for Open Science:
DIRECTOR OF ENGINEERING
See more information and apply here: www.cos.io/careers -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thread with highlights from the @oaspa webinar about paper mills with @deevybee @ClearSkiesAdam and myself presenting.
UK Research Integrity Office
@UKRIO
There's a "perfect storm" of paper mills and reliance on legacy systems that is causing huge problems for publishers, says @catmacOA, chairing the @OASPA webinar "Scholarly Communication in Crisis". What are the solutions? #publicationethics #papermills #openscience
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The 2022 Center for Open Science Impact Report is online: www.cos.io/impact
Thank you to our supporters, partners, and colleagues for advancing open scholarship. And, thank you to everyone contributing to increasing openness, integrity, and reproducibility of research. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Dominique Roche
@dom_roche
Are you heading to #metascience2023?
The @SSHRC_CRSH, Canada's Social Sciences and Humanities Research Council, will be there! We'd like to connect with Canadian participants to discuss RoR in Canada. Pls RT and DM me to connect.
metascience.info -
Brian Nosek (@briannosek@nerdculture.de) reposted
TranspariMED
@TranspariMED
THREAD: Today we publish a report comparing:
(a) what WHO says funders should do
(b) what clinical trials funders actually do
Let's start with the 11 safeguards against research waste set out by the WHO... 1/n
www.transparimed.org/single-post/researchfunders2023 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Catriona MacCallum @catmacOA@mastodon.scot
@catmacOA
Still time to register for the @OASPA public webinar tomorrow on hugely impt topic - papermills, research integrity and Open Scholarship with @deevybee @ClearSkiesAdam and @BrianNosek ~800 registrants so far but all are welcome (slides & recordings made public after as usual)
OASPA
@OASPA
OASPA Webinar: Scholarly Communication in Crisis: Research Integrity and Open Scholarship
19 April
Research integrity underpins the research endeavor but confidence is starting to break down.
Can open research practices and tools help? Join us live: oaspa.org/scholarly-communication-in-crisis-research-inte… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: The awesome line-up of virtual sessions for the lead-up to #metascience2023 is live. These sessions are free, but you have to register:
metascience.info/virtual-symposia/
And, don't miss the in-person meeting May 9-10: metascience.info/agenda/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
OSF recently released new #metadata features to meet the desirable characteristics for federally-funded research. Learn how these updates will enhance your research workflow and compliance in our latest blog post. www.cos.io/blog/osf-adds-new-metadata-features-to-meet-de…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Have you taught about preregistration and Registered Reports? The COS programs team needs your help for developing lesson plans for a new initiative.
Consulting rate US$100/hr, for up to 40 hours of work during the next two months. Contact @HuajinBioLib if you are interested! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: Creating anonymous view-only links for OSF projects now also anonymizes registration metadata to help with blind peer review.
Non-anonymized version left vs Anonymized version right.
Note: Still if the project is public, it is public, limiting anonymizing effectiveness. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The most important action for any nonprofit organization is to align their business model with their mission. When they are out of alignment, the desire for self-preservation will defeat the mission almost every time.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
Don't miss the free pre-conference virtual symposia for the #metascience2023 Conference, supported by @aimos_inc and @RoRInstitute!
💻 Virtual symposia: metascience.info/virtual-symposia/
ℹ️ In-person event: metascience.info -
Brian Nosek (@briannosek@nerdculture.de) reposted
Olmo van den Akker
@DenOlmo
Yes, the preprint lists for every question in the form what the corresponding PRISMA-item is, and also discusses PROSPERO extensively.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Even if science is a strong-link problem, the credibility crisis isn’t about finding weak links, it’s that the “strongest” links are weak. It’s a crisis of confidence in our ability to tell what’s strong vs weak. That kind of miscalibration is disastrous in strong-link contexts.
Adam Mastroianni
@a_m_mastroianni
There are two kinds of problems: strong-link problems and weak-link problems.
Weak-link: quality depends on how good the *worst* things are
Strong-link: quality depends on how good the *best* things are -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
COVID times was filled with extremes for good and bad. On the good side, I am filled with joy each time I think of the family activity that led to this thread.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
For the last 2.5 years, my daughters and I have been rating breakfast places in #charlottesville #cville area. We rated 51 restaurants from 1 (worst) to 10 (best) on taste, presentation, menu, ambiance, & service. We also recorded cost-per-person.
Here's what we learned. 1/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A systematic review preregistration template is now available on @OSFramework, the first of several community-developed templates to be added.
Key contributions from @DenOlmo @matherion in conceiving the template & @katiecorker getting it included.
More: osf.io/preprints/metaarxiv/3nbea/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
BloodAndBugs 🦠
@MedVampire
2022 was an challenging year for me and being selected for the Open Influenza Research fellowship with @OSFramework as a 2022 second Cohort member was one of my greatest highlights. After acquiring knowledge on Open science practices and it's relevance
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Fiona Fidler
@fidlerfm
Many of these lead-up #metascience2023 sessions are AEST friendly
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: The awesome line-up of virtual sessions for the lead-up to #metascience2023 is live. These sessions are free, but you have to register:
metascience.info/virtual-symposia/
And, don't miss the in-person meeting May 9-10: metascience.info/agenda/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: The awesome line-up of virtual sessions for the lead-up to #metascience2023 is live. These sessions are free, but you have to register:
metascience.info/virtual-symposia/
And, don't miss the in-person meeting May 9-10: metascience.info/agenda/
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Virtual session presenters include @EJWagenmakers @kekecs_zoltan @LexBouter @RikPeels @dmoher @PaulGlasziou @itchyshin @deevybee @chrisdc77 @jamesheathers @socpsychupdate @psforscher @FalkLieder @NDevito1 @sampendu @gowrigopala and many more!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
COS needs a lesson planning consultant from now to July (up to 40 hrs total) to prepare instructional materials for a train-the-trainer model for existing open science modules.
If you are interested, please send a CV and info about your curriculum/pedagogy work to huajin@cos.io
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More information about the modules is here: www.cos.io/services/training
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Registered Reports Community Feedback
@Rate_RegReports
Registered Report authors!
🚫Had a manuscript rejected?
↩️Withdrawn a manuscript?
Share your experiences and help build a picture of how @RegReports peer review is implemented across journals:
registeredreports.cardiff.ac.uk/feedback
📊Current stats: 246 pieces of feedback across 54 journals -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Could you be willing to have 1 or more lines of your research entered into randomization to proceed as usual or be submitted as a Registered Report?
Please provide us 2 minutes of feedback to assess what would make participation possible and appealing.
docs.google.com/forms/d/e/1FAIpQLSekVHqcGXQ526IuEsDRLcXO0…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thanks for your very helpful responses! More are welcome.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Could you be willing to have 1 or more lines of your research entered into randomization to proceed as usual or be submitted as a Registered Report?
Please provide us 2 minutes of feedback to assess what would make participation possible and appealing.
docs.google.com/forms/d/e/1FAIpQLSekVHqcGXQ526IuEsDRLcXO0… -
Brian Nosek (@briannosek@nerdculture.de) reposted
BITSS
@UCBITSS
Are you a social science researcher interested in strengthening your open science skills and practicing them in a supportive learning environment? Apply to attend our Research Transparency and Reproducibility Training (RT2) from May 17-19: cega.submittable.com/submit!
-
Brian Nosek (@briannosek@nerdculture.de) reposted
UK Research Integrity Office
@UKRIO
Scholarly Communication in crisis: @OASPA webinar, with panellists @AdamSci12, @BrianNosek, and @deevybee, with co-chairs @catmacOA and Emma Wilson of
@RoySocChem.
More info: oaspa.org/scholarly-communication-in-crisis-research-inte…
#scholcomm #publicationethics #researchintegrity #webinar
Richard Poynder
@RickyPo
Scholarly Communication in Crisis: Research Integrity and Open Scholarship | Apr 19, 2023 03:30 PM in Universal Time UTC us02web.zoom.us/webinar/register/WN_3Q9aOMSFQDiVuvJwStF7Zw
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Chinese Open Science @ chineseopensci.bsky.social
@ChineseOpenSci
Our COSN @OpenSciChina position paper is now online at #ampps @PsychScience!
We celebrate achievements & learn from challenges, so it is also a gift to the global #openscience!
Ever wondered how to build an OS community from scratch?
🧑💻Paper: doi.org/10.1177/25152459221144986
🧵👇!
(1/n -
Brian Nosek (@briannosek@nerdculture.de) reposted
Royal Society Publishing
@RSocPublishing
#rsos supports the goals of the metascience and #openscience movements and we’ve partnered with @OSFramework to sponsor Metascience 2023. In-person and virtual attendance options are still available. Register here: metascience.info
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Imagine a rating system in which universities themselves transparently select the outcomes on which they are evaluated.
Prospective students learn which schools share their priorities and how they are doing.
Schools are incentivized to focus on outcomes that matter to students.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The models for US News university rankings are selected after observing the outcomes. What a scandal.
All universities should walk away from this dysfunctional ranking system. There's a place for ratings, but start with shared values and priorities.
$: www.wsj.com/articles/u-s-news-college-rankings-yale-law-f…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Such a system would embrace the diversity of objectives that schools and students have, rather than apply the same criteria to all schools.
Because there are so many students and so many schools, niche institutional characters would emerge and could thrive. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The models for US News university rankings are selected after observing the outcomes. What a scandal.
All universities should walk away from this dysfunctional ranking system. There's a place for ratings, but start with shared values and priorities.
$: www.wsj.com/articles/u-s-news-college-rankings-yale-law-f…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
h/t Dan Molden
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The full speaker list for #metascience2023 is an outrageously excellent cast of early career and mid-career scholars and change makers (and some of the rest of us).
Check it out! metascience.info/speakers/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Dr Jessie Baldwin
@jessiebaldwin
Are you interested in pre-registering secondary data analysis studies & looking for expert advice?
Join this @ucl workshop with @MarjanBakker to learn how to effectively pre-register secondary data analysis studies: www.eventbrite.co.uk/e/workshop-preregistration-of-second…
30th March, 2-4pm BST. All welcome! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In "Psychology’s Increased Rigor Is Good News. But Is It Only Good News?" Barry Schwartz concludes "My aim here has only been to encourage us to acknowledge that there is a price."
I agree with that, but disagree with most of the rest of the analysis. behavioralscientist.org/psychologys-increased-rigor-is-go…Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Ex: preregistration in a registry improves the discovery of all findings, regardless of outcomes so that meta-analyses are less biased by selective reporting.
Registered Reports removes selection based on significance, ditching the false +/- tradeoff for individual studies.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There are tradeoffs to consider and evaluate: time, allocation of resources, potential impacts on risky hypotheses, etc. But, this analysis does not wrestle with how the rigor-enhancing behaviors actually change practice and evidence to address false positives and false negatives
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Richard Poynder
@RickyPo
Why open data is critical during review: An example steveharoz.com/blog/2023/why-open-data-is-critical-during…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Apply now!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
TRAVEL AWARDS for students, postdocs, and members of underrepresented groups to attend Metascience 2023 at the National Academies in Washington DC, May 9-10.
1-min application: docs.google.com/forms/d/e/1FAIpQLScBQQeOtGCMFHwF9Cjce-cne…
Conference info: metascience.info
Apply by March 20, 4p ET. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael Nielsen
@michael_nielsen
I was writing about this in email this morning, for reasons unrelated to metascience, but I really appreciate the Metascience Conference bringing together so many communities: science of science, open science, STS, history, philosophy, & sociology of science, economics, [etc etc]
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael Nielsen
@michael_nielsen
I'm looking forward to Metascience 2023:
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Metascience 2023 conference agenda is live!
metascience.info/agenda/
A stellar line-up of presenters with huge disciplinary diversity, but all with a shared interest in understanding and improving science. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Metascience 2023 conference agenda is live!
metascience.info/agenda/
A stellar line-up of presenters with huge disciplinary diversity, but all with a shared interest in understanding and improving science. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Heidi L. Williams
@heidilwilliams_
Do you have an idea of how to accelerate scientific progress on socially important problems?
If so — and if seed funding for a research project would help — please apply to a new re-granting program @PaulFNiehaus, @calebwatney, and I are co-leading. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thanks #2023osper for your questions. The slides for my talk are here: osf.io/9rnuc
And, this is the paper that I discussed: journals.plos.org/plosbiology/article?id=10.1371/journal.… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There is an amazing line-up of speakers such as @pierre_azoulay @michael_nielsen @profEmilySena and @siminevazire.
See the full list: metascience.info/speakers/
The program will be posted soon! -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
TRAVEL AWARDS for students, postdocs, and members of underrepresented groups to attend Metascience 2023 at the National Academies in Washington DC, May 9-10.
1-min application: docs.google.com/forms/d/e/1FAIpQLScBQQeOtGCMFHwF9Cjce-cne…
Conference info: metascience.info
Apply by March 20, 4p ET. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
TRAVEL AWARDS for students, postdocs, and members of underrepresented groups to attend Metascience 2023 at the National Academies in Washington DC, May 9-10.
1-min application: docs.google.com/forms/d/e/1FAIpQLScBQQeOtGCMFHwF9Cjce-cne…
Conference info: metascience.info
Apply by March 20, 4p ET.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There is an amazing line-up of speakers such as @pierre_azoulay @michael_nielsen @profEmilySena and @siminevazire.
See the full list: metascience.info/speakers/
The program will be posted soon! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Nature now welcomes Registered Reports!
www.nature.com/articles/d41586-023-00506-2 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Joe Henrich
@JoHenrich
ManyLabs2 is often taken to provide evidence against cross-cultural psychological differences. We had a close look and offer a different perspective. Thanks to the authors for sharing their data! @BrianNosek
journals.sagepub.com/doi/full/10.1177/2515245918810225 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Major enhancement to OSF metadata released today for projects and files. This release has been a long time coming to advance FAIR data on OSF, and it sets the stage for big future releases improving reporting and search and discovery.
Read more about it: www.cos.io/blog/new-osf-metadata-to-support-data-sharing-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Review of proposals starts Monday. Submit yours!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Review of proposals for the Metascience 2023 conference talks, discussions, and virtual events begins on January 23.
More info: metascience.info/call-for-proposals/
Registration is open. In-person attendance is limited by the event space at the National Academies.
www.eventbrite.com/e/metascience-2023-tickets-445211869687 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Job Alert: Research Scientist position at the Center for Open Science.
Jobs page: www.cos.io/careers
Direct link to apply: jobs.lever.co/cos/37d48a6e-5def-4460-a7b9-c510d620448d -
Brian Nosek (@briannosek@nerdculture.de) reposted
Alondra Nelson
@AlondraNelson46
The Biden-Harris Administration is committed to advancing open, equitable research.
Announcing the YEAR OF OPEN SCIENCE, actions to provide access to taxpayer-supported research and data, accelerate innovation, and promote public participation and trust.www.whitehouse.gov/ostp/news-updates/2023/01/11/fact-shee… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Review of proposals for the Metascience 2023 conference talks, discussions, and virtual events begins on January 23.
More info: metascience.info/call-for-proposals/
Registration is open. In-person attendance is limited by the event space at the National Academies.
www.eventbrite.com/e/metascience-2023-tickets-445211869687 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
Researchers in the U.S. and elsewhere are increasingly expected to have data management and sharing plans. Join us on Jan. 25 at 11 am ET for a discussion on data sharing for qualitative and mixed-methods research.
👉 Register: cos-io.zoom.us/webinar/register/WN_Tcsn62RDTlaiAU5f3xr4KA -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Metascience 2023 is open for registration and proposal submissions!
metascience.info
In-person: May 9-10, 2023; National Academy of Sciences, Washington DC.
Virtual: April-May sessions (free).
Call for proposals will be reviewed starting January 12th. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
As 2023 begins, I repeat my yearly reflection and commitment.
I will do my best to use the opportunities available to me for the betterment of humanity, in daily routine and with vision for the future.
I am grateful for the collaboration and support of so many people I admire. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Greta Thunberg
@GretaThunberg
this is what happens when you don’t recycle your pizza boxes
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The people have spoken, louder than the bots.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Perfect description, made 8 months ago.
Yishan
@yishan
I've now been asked multiple times for my take on Elon's offer for Twitter.
So fine, this is what I think about that. I will assume the takeover succeeds, and he takes Twitter private. (I have little knowledge/insight into how actual takeover battles work or play out)
(long 🧵) -
Brian Nosek (@briannosek@nerdculture.de) reposted
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Metascience 2023 is open for registration and proposal submissions!
metascience.info
In-person: May 9-10, 2023; National Academy of Sciences, Washington DC.
Virtual: April-May sessions (free).
Call for proposals will be reviewed starting January 12th. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The deadline for requesting the exploratory data for this unique opportunity is a week away (Dec 15). See thread for details.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Are you interested in ideology, morality, political psych, implicit and explicit measures, or individual differences? We are releasing a massive dataset (>280k sessions) with dozens of individual difference measures, implicit and explicit measures. 1/ docs.google.com/document/d/12jmUlXguhFWjqMx2lHGrF8sFANbrJ…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Oxford e-Research Centre
@OxfordeResearch
The Center for Open Science (COS) has welcomed Professor @SusannaASansone to its Board of Directors to support their mission of promoting open science and reproducibility in research @OSFramework
oerc.ox.ac.uk/news/center-for-open-science-welcomes-susan… -
Brian Nosek (@briannosek@nerdculture.de) reposted
SIPS (Account Archived - See Bio)
@improvingpsych
The Society for the Improvement of Psychological Science is excited to announce it will head to Africa for its ninth annual hybrid meeting. SIPS 2024 will be held in Nairobi, Kenya and will be hosted by @BusaraCenter.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An incredible detail of the critiques that Roberts' perspective is bad for science is that the author of the main paper has quite a history associated with research misconduct.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Translation: Steven Roberts broke the OSF.
Center for Open Science
@OSFramework
OSF is experiencing some technical issues at the moment, and you may experience inability to login or access content. We expect a resolution shortly and will post any further updates.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Brilliantly written, breathtaking circumstance.
I haven't read Steven Roberts' work previously, but I expect that I'll be reading all of it now.
Jamil Zaki
@zakijam
My colleague Steven Roberts was recently invited to comment on a "debate" about diversity at a @PsychScience journal.
His experience was troubling and surreal, and lays bare the very biases he studies. He describes it in this important pre-print.
1/
psyarxiv.com/xk4yu -
Brian Nosek (@briannosek@nerdculture.de) reposted
Einstein Foundation
@Einstein_Berlin
The €200,000 Institutional #einsteinfoundationaward2022 goes to @PsySciAcc @coles_nicholas_. The network brings together scientists from over 70 countries and has been making key contributions to the democratization and diversification of psychological research. Congratulations!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It is the last day of the 1 month commitment to trying out Mastodon. How did it go?
A quick poll...
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them.Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Reflecting on November, I have been very impressed by science twitter's collective action. In my corner of the system, the experience has been dramatic: little posting on twitter, and lots of activity on Mastodon.
In this sense, the expt has been a wild success. We tried it!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More generally, I think it is too early to tell if a long-term transition will occur and stick. The dominant causal force is surely Elon's decision making--change may come even for those of us committed to twitter!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
#mastodonmigration polls: Nov 8 and Nov 18 on twitter and mastodon
Obvious: Selection bias of polling location, Mastodon responders more bullish on the experiment than Twitter responders. It would have been shocking if this had not occurred.
More intriguing...Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Likewise, (over-interpreting) 11% of the moderately optimistic Mastodon responders moving up to high optimism.
Subtle counter-indicator: Response rate dropped more on Mastodon than on Twitter.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
To me, the biggest surprise is to compare both polls to an imagined poll conducted in early October asking about plausibility of the twitter community moving to Mastodon. I can't imagine that more than 5-10% would have said anything other than "It's not going to happen."
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Jess Butler
@JessButler284
Use this to find your tweeps on🐘
pruvisto.org/debirdify/
Top servers my people are on
mastodon.social
fosstodon.org
mastodon.online
mstdn.social
scicomm.xyz
nerdculture.de
fediscience.org
scholar.social -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Time for a replication. 10 days I ago I asked:
"Change is hard. New tech, different features, shifting norms, uncertain homeostasis. One week into the November #mastodonmigration experiment, what is your prediction about its chances for success?"
Now? -
Brian Nosek (@briannosek@nerdculture.de) reposted
Jack Hamilton
@jack_hamilton
both devin chandler and lavel davis were my students. devin was in my class in spring 2022, and lavel is in my class this semester. I am so indescribably sad right now but I wanted to share a few memories of them, because they were wonderful people
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Change is hard. New tech, different features, shifting norms, uncertain homeostasis. One week into the November #mastodonmigration experiment, what is your prediction about its chances for success?
-
Brian Nosek (@briannosek@nerdculture.de) reposted
David Brückner
@d_brueckner
It was great speaking to @mollyrcampbell from @Tech_Networks about why and how people are making a transition from #sciencetwitter🐦 -> #sciencemastodon 🐘
Check out the full article below, including an interview with @BrianNosek and myself!
Technology Networks
@Tech_Networks
In this article, @mollyrcampbell explores how Twitter has supported science communication and outreach, as well as the motivations for migration. Where are you heading, #scitwitter? 🤔
#science #sciencenews #twitter
www.technologynetworks.com/tn/articles/should-scitwitter-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More coverage of the #mastodonmigration experiment for November.
www.technologynetworks.com/tn/articles/should-scitwitter-… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Esteban Moro
@estebanmoro
Something big is happening in Mastodon over the weekend. Here is my graph, updated. Right now Mastodon is growing ~3 times faster than Twitter.
Esteban Moro
@estebanmoro
A little bit of data: evolution of the new Mastodon users in the last month.
To put it in perspective the number of monetizable daily active users on Twitter grew by ~25000 daily in 2022 (apples-to-oranges comparison). -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Day 6 #movember: A full 10% of the people I follow put their Mastodon handles in their twitter name or bio.
fedifinder.glitch.me found them all automatically and then I just imported the CSV and now I am following them on Mastodon too. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Arshiya Sangchooli (He/They)
@Arshiya_San
Just yesterday fedifinder returned 30 handles and today it's up to 66?? I kept hearing of the scale of the fabled Mastodon migration but this is still a surprise!
(Context: fedifinder finds unique mastodon handles in your twitter follower/followee bios to help you follow them!)
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Put your Mastodon handle in your twitter name or bio like pictured (@briannosek@nerdculture.de). Then, when your followers use fedifinder.glitch.me to repopulate who they follow on Mastodon you will automatically be added to their list!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It is hard to count real, active users, but estimating and comparing follower counts, I think 10-15% of my twitter network is on Mastodon 5 days into #movember.
But, none of the 100s of women w/"looking for love" bios and no followers have yet to also follow me on Mastodon. Sad. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Day 4: #movember of #mastodonmigration
www.nytimes.com/2022/11/04/technology/elon-musk-twitter-l… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Delicious that Elon's tweet gets a context warning.
Also, taking it at face value, whose free speech is being destroyed? The activists are expressing theirs, the advertisers are expressing theirs, he is expressing his, and readers are reacting with their own.
#movember -
Brian Nosek (@briannosek@nerdculture.de) reposted
David Brückner
@d_brueckner
Will #sciencetwitter🐦 -> #sciencemastodon🐘 work?
The response to this was massive & my #mastodon has gone from a quiet void to a science party in just 3 days!
Many are worried about servers, network effects & losing followers
Follow these tricks to make it work for you! 🧵 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Day 2: #movember Large uptake of Mastodon accounts continues. I see fewer reports of challenges accessing servers; perhaps ops teams are catching up to demand.
For the experiment to have a chance, those active on Mastodon need to signal here that they are posting content there.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Denis Meuthen
@DenisMeuthen
Seeing that everyone is going to mastodon.social, maybe it is important to advertise alternative science mastodon instances such as this list fediscience.org/server-list.html
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Day 1 of #movember the month for the #mastodonmigration experiment is underway
See the linked thread for the behaviors to solve the collective action problem
Add your handle to your tw bio
Use fedifinder.glitch.me to find others
Check out these tips axbom.com/mastodon-tips/
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
And, here's a guide for getting started: axbom.com/mastodon-guide/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Just 8 hours into #mastodonmigration and there is strong take-up of the 3 actions that are necessary to give the movement a chance to succeed.
And, there's a tip in the next tweet to help your followers find you on Mastodon.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Put your Mastodon handle in your twitter name or bio like pictured (@briannosek@nerdculture.de). Then, when your followers use fedifinder.glitch.me to repopulate who they follow on Mastodon you will automatically be added to their list!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@lakens @deevybee @BorisBarbour @sharoz @hardsci @pdakean Key for this kind of collective action to work is the commitment of people with high follower counts and regular producers of content. Are you willing to do this for November? And, who else can you ask to commit?
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If it IS working, then we will probably need to continue the same three behaviors for another month at least to keep helping consumers see benefit of the new platform.
Maybe by January, there will be little remaining value to being on twitter bc most of the network has moved.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If you will commit to these behaviors for November, retweet this thread.
If even early adopters are unwilling to make these commitments, then we will know VERY quickly if this approach to solving the collective action problem will fail.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
You can contribute $ to the open source Mastodon project to maximize its chances to succeed at managing the user influx and feature needs.
opencollective.com/mastodon
or
www.patreon.com/mastodon
or
sponsor.joinmastodon.org -
Brian Nosek (@briannosek@nerdculture.de) reposted
Nick Byrd, Ph.D.
@byrd_nick
I wasn't interested in the #mastadonmigration until it was posed as a collective action challenge.
So I will be trying the 3 steps below.
Find me at nerdculture.de/@ByrdNick (also in bio).
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Everyone moving from twitter to mastodon is very likely to fail because it is a collective action problem, benefits depend on others actions.
Here's how it can succeed. It will require your commitment for 1 month to do a few things. Read and retweet if you commit to do them. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Daniël Lakens
@lakens
Hi Brian, I am 100% with you and hereby commit to only post new content on Mastodon, and not on Twitter for all November. (I will limit myself to replies, such as these).
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Even if the Mastodon migration only affects #academictwitter, it could be a great success.
Social media is like the local pub. The food quality is irrelevant. As long as the people I want to talk to and hear from are there, I'm happy.
I'm at: nerdculture.de/web/@briannosek -
Brian Nosek (@briannosek@nerdculture.de) reposted
Nate Breznau
@BreznauNate
𝗔 𝗛𝗶𝗱𝗱𝗲𝗻 𝗨𝗻𝗶𝘃𝗲𝗿𝘀𝗲 𝗼𝗳 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵𝗲𝗿 𝗨𝗻𝗰𝗲𝗿𝘁𝗮𝗶𝗻𝘁𝘆
• 73 teams tested same hypothesis w/ same data
• Outcomes varied widely, each workflow unique
• We attempt to explain outcomes from decisions, no easy answer
www.pnas.org/doi/10.1073/pnas.2203150119 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
What happens when fraud is easy enough to do well that not even @MicrobiomDigest can detect it?
Excellent essay highlighting the need to change the system from rewarding getting it published to rewarding getting it right.
www.nytimes.com/interactive/2022/10/29/opinion/science-fr… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This week #academictwitter remembers that there are tradeoffs of leaving academia for industry.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Eric Topol
@EricTopol
Billion-dollar experiments to "escape scientific stagnation"
www.economist.com/finance-and-economics/2022/10/26/how-to… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Marike Schiffer
@MarikeSchiffer
I'm a big fan of replication and generalization studies and I'm really pleased we're launching this call for replications and generalizations with the awesome human behaviour team at NComms.
Open for submissions!
Any questions about suitability, please email.
Communications Psychology
@CommsPsychol
#replication rules.
Communications Psychology and the Human Behaviour Team at @NatureComms invite submissions of direct replication and generalization studies in psychology. We welcome #registeredreports & research Articles #reproducibility www.nature.com/collections/dgcejgihcc -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Do you study ideology, morality, attitudes, or a similar construct? Are you interested in doing a Registered Report using a massive, already collected dataset? Check out this thread, and request the data by December 15! More info: docs.google.com/document/u/0/d/12jmUlXguhFWjqMx2lHGrF8sFA…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Are you interested in ideology, morality, political psych, implicit and explicit measures, or individual differences? We are releasing a massive dataset (>280k sessions) with dozens of individual difference measures, implicit and explicit measures. 1/ docs.google.com/document/d/12jmUlXguhFWjqMx2lHGrF8sFANbrJ…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Finnish Reproducibility Network
@FinRepro
Thank you @shkuure @LaakkonenL @VVoikar for the excellent event today! It was especially motivating to hear the talk by @BrianNosek on how researchers and journals can raise the bar for reproducible and robust science with openness through registered reports. 👏 👏 👏
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @HouseScience Qs to OSTP about public access to papers and data are reasonable. The only part I don't like is a common framing: that costs to communicate research take away from costs for research.
If research isn't communicated, did it even happen?
science.house.gov/imo/media/doc/ostp_letter_research_acce…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
That isn't to say that all publishing costs should be moved to research grants, just that the costs of communicating research (and sharing data, materials, and code) ARE part of the costs of doing research. Research isn't done, or useful, if it isn't communicated.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It’s just one counter data point, but I use my domain knowledge about bias, behavior change, culture, structures and systems constantly in my work. Even more, this knowledge is the basis of our theory of change and organizational strategy to advance our mission.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Poll results imply that most do not equate rigor with certainty of results or established validity of methods (presuming you agree that this is usually less known/established in exploratory phase.
However, most noted indicators of rigor ARE equated/aligned with those things.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Think for a moment of your definition of research rigor before paying attention to the poll.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Based on your definition, can exploratory/discovery research be rigorous?
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This suggests a gap in our beliefs and measures of rigor, with measures being relevant to only a subset of research approaches.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Tip: compose one tweet for each of the 10 #academictwitter topics so that when the topic reappears each quarter you can just quote tweet it and have done your part for the discourse. A real timesaver!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The essential bit in these debates is that it is perfectly legitimate to treat academia or science as a job, a career, or a calling.
Declaring one as correct and judging others for doing it differently helps precisely zero people. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If you are the 8th person reading this tweet, you might be naked right now!
www.reuters.com/technology/exclusive-where-did-tweeters-g…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More seriously, this percentage shocked me because I cannot recall ever seeing NSFW content in my twitter feed (that wasn't an invitation, friends). It brought home how much twitter is a collection of very distinct communities.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Requesting your insight: What is a feature of rigorous research?
Is that feature observable? If not, is it possible to know whether research has that feature? How?
[Like others' responses that you endorse.] -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Think for a moment of your definition of research rigor before paying attention to the poll.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Based on your definition, can exploratory/discovery research be rigorous?
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Explanation of your definition, meaning of your answers, and added nuance to the issue are all welcome!
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Emily Farran
@ekfarran
Our second annual update of "Open Research across disciplines" is out: osf.io/3r8hb/ Better still, @ukrepro have transformed it into a fabulous set of accessible webpages, with additional case studies and more: www.ukrn.org/disciplines/ @CamillaKGilmore @priyasilverst
Emily Farran
@ekfarran
Open research across disciplines is live: osf.io/3r8hb/. We list case studies of open research and resources per discipline. Written with @priyasilverst, @CamillaKGilmore, Aminath Ameen & Iliana Misheva. Additional key resources welcome-use form on page 3. @ukrepro
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Policymakers tend to be self-centered rather than user-centered in policy implementation systems. This just keeps appending more demands on the users, which undermines the policy intent by making them administrative burdens rather than guides for good practice, transparency, etc.
Piali Sengupta
@SenguptaLab
If I ever quit academia, it will be because of the sheer volume of admin work that is being piled on us PIs. It's non-stop and is coming from all directions including the Univ. Attending training sessions, filling out forms etc is becoming my nth full-time job and I am fed up.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The next step is for OSF to provide automated mechanisms for compliance reporting to those journals, funders, and institutions removing yet another burden for researchers who are trying to live up to the policies, and to support the policymakers compliance objectives.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Mantra:
Let researchers focus on their research.
Let policymakers focus on their mission.
Provide software to manage the rest. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The deadline to request the exploratory dataset is December 15, 2022.
The deadline to submit a Stage 1 Registered Report is February 15, 2023. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Are you interested in ideology, morality, political psych, implicit and explicit measures, or individual differences? We are releasing a massive dataset (>280k sessions) with dozens of individual difference measures, implicit and explicit measures. 1/ docs.google.com/document/d/12jmUlXguhFWjqMx2lHGrF8sFANbrJ…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Are you new to writing grants, or want to see some examples of proposals to different private and public agencies?
All of our successful and failed proposals that are not restricted from sharing are publicly available here: osf.io/8km72/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Hypothesis: Increasing rigor makes researchers more appealing to industry.
Consequence: Academic fields that improve rigor will then need to up their game on pay and other parts of culture to compete with industry and retain the benefits of having improved. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Important new paper from @DenOlmo and colleagues examining selective reporting in preregistered studies in psychology research.
It identifies opportunities for improvement to increase visibility of changes between research plans and reports.
Paper: osf.io/preprints/metaarxiv/nf6mq
Jelte Wicherts
@JelteWicherts
Comparison of 459 preregistrations and papers shows that more than half of the preregistered studies contained omitted hypotheses (N = 224; 52.2%) or added hypotheses (N = 227; 56.8%), and about one-fifth of studies contain changed hypotheses (N = 82; 19%)
osf.io/preprints/metaarxiv/nf6mq/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Feature request: When I write my texts in ALL CAPS, I want Alexa and Siri readers to yell it.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Example
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Nature Communications
@NatureComms
Transparent peer review for all!
We will publish reviewer reports and author responses for all research articles submitted after November 1st
#openscience
www.nature.com/articles/s41467-022-33056-8 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Nicholas Coles, PhD
@coles_nicholas_
⚠️New paper at Nature Human Behaviour⚠️
Can posed smiles make people feel happier?
In a global adversarial collaboration, we found overwhelming support for this controversial hypothesis.
But we couldnt resolve one thing: concerns about a popular pen-in-mouth smiling task.
🧵 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Some reflections on @michael_nielsen and @kanjun's excellent essay in which they coined the term "metascience entrepreneur" and used progress in reforming social psychology as a case example.
scienceplusplus.org/metascience/index.html
Kanjun 🐙
@kanjun
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascience/index.html -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The qualitative preregistration template on OSF has been used more than 550 times. Three cheers for Tamarinde Haven and her collaborators for the delphi study that produced the template and to the researchers that are exploring its value for qual work. osf.io/registries/discover?provider=OSF Registries&sort=-…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael 英泉 Eisen
@mbeisen
Accepting and rejecting papers is an archaic practice that is terrible for science. It strips peer review of its value & institutionalizes the practice of judging scientists based on where, rather than what, they publish. It's time for this system to go.
elifesciences.org/articles/83889 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Richard Sever
@cshperspectives
Massive news: eLife to abolish accept/reject decisions: papers will just be “peer reviewed”. Others can argue about this, but lots of interesting consequences. 1/9 elifesciences.org/articles/83889
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There is a (modest) role for citation metrics in analysis of research, but it is now trivial to publish empty papers every day citing all your prior papers to render citation counts meaningless.
Making gaming strategies like this visible is a first step to address the problem.
Mark Griffiths
@DrMarkGriffiths
Delighted to see that I had over 5000 citations in a 2-month period on Google Scholar to take me over 125,000 citations and an h-index of 170. Big thanks to all my co-authors @PsychologyNTU @NTUNews @BPSCyberPsych @ntu_research @daraghmcdermott @ProfEdwardPeck @NTUSocSciences
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A well-documented case study of his publishing practices with a quantitative analysis of how it produced the observed citation rate would be highly impactful and, dare I say, well cited.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Such a case study would be helpful for showing administrators, search committees, funders, and others who are tempted to use citations as a dominant screening or selection criterion. It’s extremity (without any obvious accountability) would make it difficult to ignore.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Jeffrey Spies
@JeffSpies
Sounds familiar. I was told to either (a) build @OSFramework and leave academia or (b) get a job, get tenure, & then build it.
Good choice? The value of the "prof" title within & around academia re reputation & even credit is extremely high, but I was tired of waiting. 🤷♂️
Paul Novosad
@paulnovosad
Wow, quite the story. 👇🏻👇🏻
Founding Arxiv is apparently not good for your academic career.
Good evidence that improving how science works is insufficiently incentivized. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"There may be a crisis in personality and social psychology, associated with the difficulty often experienced by researchers in attempting to replicate published work." Greenwald 1975 editorial taking on Editorship of JPSP faculty.washington.edu/agg/pdf/Greenwald.Editorial.JPSP.1…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Authors also had to *mail* their data to him for reanalysis. If you know anyone with the physical journals you can tell immediately which issues were during his tenure bc so few articles made it through. He was too far ahead of his time, and trying to do it solo.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"There may be a crisis in personality and social psychology, associated with the difficulty often experienced by researchers in attempting to replicate published work." Greenwald 1975 editorial taking on Editorship of JPSP faculty.washington.edu/agg/pdf/Greenwald.Editorial.JPSP.1…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
His policies included: optional blind review, full procedures to enable replication, data sharing, welcoming replications and null results, admonishment against selective reporting, & submitting in the paper in quadruplicate--copies ok if high-quality reproduction process is used
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
He had more "outlandish" ideas he wanted to do but these were what he was able to get through the Board (which later removed him for carrying out the policies).
I'll bet money one of his outlandish ideas was preregistration (I can't recall ever asking him). -
Brian Nosek (@briannosek@nerdculture.de) reposted
Stuart Buck
@stuartbuck1
This is definitely a must-read, and a must-re-read . . . in fact, to take advantage of spaced repetition, I'll reread it tomorrow, then in a week, then in a month, etc.
Kanjun 🐙
@kanjun
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascience/index.html -
Brian Nosek (@briannosek@nerdculture.de) reposted
Patrick S. Forscher | @psforscher.bsky.social
@psforscher
Terrific thread on my main activity for the past four-ish years: “meta-science entrepreneurship”.
I reckon this set of ideas can be deployed to address other research issues, such as diversity in the scientific workforce and improving how research participants are treated
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Some reflections on @michael_nielsen and @kanjun's excellent essay in which they coined the term "metascience entrepreneur" and used progress in reforming social psychology as a case example.
scienceplusplus.org/metascience/index.html
Kanjun 🐙
@kanjun
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascience/index.html -
Brian Nosek (@briannosek@nerdculture.de) reposted
Don Moore
@donandrewmoore
@EllenBarryNYT writes about our work on depressive realism in today's @nytimes: www.nytimes.com/2022/10/18/health/depressive-realism-theo… My co-authors: @amelia_dev, Sheri Johnson, Karin Garrett
-
Brian Nosek (@briannosek@nerdculture.de) reposted
David Chapman
@Meaningness
💥 Revolutionary rethinking of what is possible for science. Enormous opportunities await—requiring radical structural reform. This analysis will stand as a definitive harbinger of that effort.
Michael Nielsen
@michael_nielsen
New essay by @kanjun and myself, on improving the culture and social processes of science:
scienceplusplus.org/metascience/index.html -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Some reflections on @michael_nielsen and @kanjun's excellent essay in which they coined the term "metascience entrepreneur" and used progress in reforming social psychology as a case example.
scienceplusplus.org/metascience/index.html
Kanjun 🐙
@kanjun
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascience/index.htmlPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Michael and Kanjun remark, as an aside, that some others find replication boring as a social movement. Of course, the point of these projects is not about replication per se, they are about research credibility and whether our current practices are producing
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
trustworthy findings at the rate that we think they are. Conducting replication is just a piece of a reform strategy to improve rigor, transparency, and sharing in service of accelerating science.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
When Uri, Leif, and Joe generated the term “p-hacking” it crystallized understanding about a class of dysfunctional behaviors, and what to do about them.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Perhaps “metascience entrepreneurship” and the vision that Michael and Kanjun articulated will have similar impact and bring funders, researchers, and the start-up/innovation cultures together to invest, experiment, evaluate, and scale improvements to science.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
UK Reproducibility Network @ukrepro.bsky.social
@ukrepro
Sustaining open research.
Join @BrianNosek and panel: @rachelbruce @sarahderijcke @tonyR_H @RGoobermanHill to discuss challenges and opportunities
Online and in-person, supported by @BrisUniEvents 10 Nov, 17:00-19:00 GMT
More info and registration: www.ukrn.org/event/sustaining-open-research-nov2022/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Kanjun 🐙
@kanjun
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascience/index.html -
Brian Nosek (@briannosek@nerdculture.de) reposted
Nick DeVito
@NDevito1
I was jotting down some thoughts about this piece and I made this exact same point.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
JWST is infrastructure to conduct research like qualtrics, microscopes, or fMRI machines. You don’t need to preregister the studies that someone will eventually perform to justify infrastructure investment. They can be preregistered when they are planned, just like JWST uses are.
Nick DeVito
@NDevito1
This quote is ridiculous. Contrary to popular belief, they're not just pointing the James Webb Space Telescope at random bits of the sky and seeing what shows up.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Use of shared infrastructure in astronomy is among the more effective and rigorous planning of research, presumably because of the preciousness of the resource and high demand. There are always things that can be improved but it meets many aims of planning to improve rigor.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Nick DeVito
@NDevito1
This quote is ridiculous. Contrary to popular belief, they're not just pointing the James Webb Space Telescope at random bits of the sky and seeing what shows up.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Registration is open for "Sustaining open research: challenges and opportunities for institutions and researchers" at the University of Bristol, 10 Nov, 17:00-19:00 GMT.
I will be presenting along with @rachelbruce @sarahderijcke @tonyR_H @RGoobermanHill
www.ukrn.org/event/sustaining-open-research-nov2022/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New resources supporting preregistration for domains in the early adoption phase to help it grow into the mainstream. ($)
Preregistration of Studies with Existing data link.springer.com/chapter/10.1007/978-3-030-99680-2_36
Preregistration in Animal Research link.springer.com/chapter/10.1007/978-3-030-99680-2_37
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@CelineHeinl and @g_mertens are there Green OA versions of your chapters available?
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If individuals can make this much of a difference, imagine what our collective efforts can accomplish.
www.washingtonpost.com/lifestyle/2022/10/17/jess-wade-sci… -
Brian Nosek (@briannosek@nerdculture.de) reposted
SIPS (Account Archived - See Bio)
@improvingpsych
One last thing! We are currently looking for 2 ex officio positions for the next year. Any grad student member may be nominated by Nov 12. Self-nominations are totally welcome! We value diversity and encourage nominations of members from all backgrounds. docs.google.com/forms/d/e/1FAIpQLSe70rpyOwoEG07u30xbTeYLd…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Registration is open for "Sustaining open research: challenges and opportunities for institutions and researchers" at the University of Bristol, 10 Nov, 17:00-19:00 GMT.
I will be presenting along with @rachelbruce @sarahderijcke @tonyR_H @RGoobermanHill
www.ukrn.org/event/sustaining-open-research-nov2022/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Nick HK
@nickchk
I have a new paper out in Economic Inquiry with a buncha coauthors: The Influence of Hidden Researcher Decisions in Applied Microeconomics. What's it about?... onlinelibrary.wiley.com/doi/full/10.1111/ecin.12992
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This hits too close to home.
Björn Schumacher
@schumacherbj
The reproducibility crisis x.com/TheBest_Viral/status/1580155904552742912/video/1
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Look, it says "we predicted" right there in the paper.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Fundamental problem: TRANSFORMATIVE is equated with NEW and excludes enabling promising solutions to SCALE.
Consequence: Proposals to scale solutions that are working are hard to fund.
Achieving scale is more transformative than only funding new things that never go mainstream. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael
@DemidenkoMI
Looks like an awesome opportunity. Have had nothing but good experience with Registered Reports, so 100% recommend doing one.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Are you interested in ideology, morality, political psych, implicit and explicit measures, or individual differences? We are releasing a massive dataset (>280k sessions) with dozens of individual difference measures, implicit and explicit measures. 1/ docs.google.com/document/d/12jmUlXguhFWjqMx2lHGrF8sFANbrJ…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Are you interested in ideology, morality, political psych, implicit and explicit measures, or individual differences? We are releasing a massive dataset (>280k sessions) with dozens of individual difference measures, implicit and explicit measures. 1/ docs.google.com/document/d/12jmUlXguhFWjqMx2lHGrF8sFANbrJ…
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Credit to Kathleen Schmidt for leading this project, Alex Jones for data wrangling with support from Anna Szabelska, and Carlee Hawkins and Jesse Graham for design and implementation of the study.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The deadline to request the exploratory dataset is December 15, 2022.
The deadline to submit a Stage 1 Registered Report is February 15, 2023. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Simple pearson's r = 0.37
Distribution suggests substantial proportion of the little cited work is not associated with high pride (comments, encyclopedia entries are common there), and highly cited are invariably in the top 3rd but intermixed with lower and little cited work. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NASA defined the smallest effect size for a successful test as changing the orbital period by at least 73 seconds. The available data suggests that DART altered the period by 32 minutes +/- 2 minutes.
www.nasa.gov/press-release/nasa-confirms-dart-mission-imp… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
What I love most about this story is that is captures the essence of science's self-skepticism. Motivating the line of experiments was:
Yes, but are we sure?
Yes, but are we really sure?
Yes, but are we really really sure?
Yes, but are we really really really sure?
But still...
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A delightful story of the basis of this year's Physics Nobel: Ever increasing experimental precision ruling out hidden variable hypotheses for a theory already widely accepted, but always vulnerable to empirical disconfirmation. www.scientificamerican.com/article/the-universe-is-not-lo…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Alfredo Sánchez-Tójar
@ASanchez_Tojar
So happy to see that last summer's @sortecoevo + @projectFOK initiative to promote Registered Reports @RegReports in Ecology & Evolution is starting to pay off!
I've just been told by the EiC that the journal "Ethology" is now accepting Registered Reports onlinelibrary.wiley.com/journal/14390310
Free Our Knowledge ✊
@projectFOK
Calling all #ecology and Evolutionary #biology researchers!
Please sign our new petition asking journal editors to improve #environmentalscience reliability by adopting the Registered Report format: freeourknowledge.org/2021-07-01-registered-reports-now_ec…
Letters will be sent out at the #sortee conf (12 July) -
Brian Nosek (@briannosek@nerdculture.de) reposted
Gabriele Paolacci
@gpaolacci
We (@squig, @alicejmoon, @OlegUrminsky, @gpaolacci) are running a workshop at the coming ACR conference in Denver (@aconsres) on “Doing Open Science”. What questions do you have about how to conduct open science? What challenges have you faced? Ask here: erasmusuniversity.eu.qualtrics.com/jfe/form/SV_dbC1jJdUpy…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
David Wright
@davecwright
In our conversation @BrianNosek notes how implicit bias training has gotten way ahead of the research!
Continue on YouTube: youtu.be/NkKuF--5V60?t=2853 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Stanford University
@Stanford
"I understand the gravity of being a woman and now a #nobel laureate in the sciences. There aren't that many of us—yet."
Prof. @CarolynBertozzi on chemistry, mentorship and representation ⬇️This media couldn't be displayed. See it on the original post. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A delightful story of the basis of this year's Physics Nobel: Ever increasing experimental precision ruling out hidden variable hypotheses for a theory already widely accepted, but always vulnerable to empirical disconfirmation. www.scientificamerican.com/article/the-universe-is-not-lo…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Holy smokes, Stewart is so good at this.
The Problem With Jon Stewart
@TheProblem
Jon interviewed Leslie Rutledge, Arkansas Attorney General, about why her state banned gender-affirming care for minors - ignoring the guidelines of major medical organizations and taking the decision out of parents’ hands. Watch the full interview on @AppleTVPlus
This media couldn't be displayed. See it on the original post. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thanks for the engagement on my "maybe all Gold OA isn't so terrible" exploration. The strongest opposing argument came from @brembs and others who provided evidence that prestige dominates so heavily that publishing operates like a luxury good market with inelastic pricing.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I favor an all Green OA world with peer review, typesetting, copyediting, etc as microservices, but I don't see an all Gold OA world as being necessarily as bad as others do. A few reasons and would love to have these challenged by others who are thinking about system change...
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It might be possible to do it and still address in-built inequities, particularly in distribution of resources, however it is hard to see how possible turns to actual when the nuances to get it right would be counter to profit motives for many actors. Aligning interests unlikely.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
As such, until someone marshalls evidence otherwise, my attempt to defend an all Gold OA model should be considered a failure.
Here's part of the thread where Bjorn @MarcusMunafo @MikeTaylor @ceptional @siminevazire & others provide some of that evidence: x.com/MikeTaylor/status/1577912978162196480 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Elevating: @acochran12733 Q prompts clarity that my view for a reasonable all-Gold OA world depends on it being a B-to-C transaction, not a B-to-B one. Libraries may be inclined to the latter to save hassle on transactions, but with consequence of preserving inflationary costs.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I favor an all Green OA world with peer review, typesetting, copyediting, etc as microservices, but I don't see an all Gold OA world as being necessarily as bad as others do. A few reasons and would love to have these challenged by others who are thinking about system change...
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I favor an all Green OA world with peer review, typesetting, copyediting, etc as microservices, but I don't see an all Gold OA world as being necessarily as bad as others do. A few reasons and would love to have these challenged by others who are thinking about system change...
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There is no possible world that eliminates prestige, but having price directly tied to prestige puts downward pressure on it because we don't like that prestige exists, many eschew it, and putting a price on it (esp one that reduces access to underresourced) makes the -'s plain
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
As such, over time, access to audiences and quality of service become the primary drivers. And, this can be accelerated by fostering diversification of quality indicators to further reduce the dysfunctional role of JIF and prestige metrics.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Ideally, those remaining journals would be in constant competition on quality of service and reducing post-accept operational costs to increase acceptance rates to make submitting their more attractive to authors. But, hard to envision precisely how it would play out.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
End: For equity and other open values, I strongly favor a commitment to green OA for the whole lifecycle with peer review and other activities as optional add-on microservices. But, I'm not yet despondent about a transitional phase that is mostly Gold OA. Convince me otherwise!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Important report on advancing a sustainable open infrastructure to support preprints. Thank you IOI and @npscience for conducting this critical work to clarify opportunities, challenges, and ways forward.
Invest in Open Infrastructure
@InvestInOpen
What's the current situation with #preprints and #openinfrastructure for them, and how could we pursue work to further investment and sustain activities in this space? 🌱
Read our latest report 👇
investinopen.org/blog/preliminary-investigation-supportin… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Baumeister, Tice, & Bushman have a new piece examining what we have learned from multi-site replication studies here: www.researchgate.net/profile/Brad-Bushman/publication/362…
It is worth taking seriously even though I don't agree with all the conclusions.
A thread & relevant paper: www.annualreviews.org/doi/abs/10.1146/annurev-psych-02082…Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I appreciate their meaningful positionality statements about their own potential biases and connections to replication efforts. Intellectual humility!
And, the work was preregistered (I haven't checked it yet, but I appreciate the effort at transparency).
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Finally, I agree with their advocacy of manipulation checks and attention to participant engagement in behavioral research. These are good things for improving research methodology of any research, replication or otherwise.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A hypothesis of mine is that preregistration (common in replications, rare in original studies) increases data discarding rates because of better planning. Without prereg, (oversimplifying) first analyze with all the data, if no effect, then look at reasons to discard.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Conclusion: I like the first two highlighted sentences as an ethos for studying complex phenomena. The third highlight is a phrase much too sanguine from my point of view when it is in the researchers' interest to explore unlikely hypotheses. Many will be false. That's ok.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Have you created shareable, reusable content for education or training about open scholarship, rigor, and reproducible research practices? COS is updating our awareness of community developed resources. Please provide a link!
(post to www.oercommons.org/hubs/OSKB also!) @HuajinBioLib
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Content could be a training module, lesson plan, video, assessment, syllabus, test, slide deck, or most anything else that can be re-used for educating others about new and emerging ways to increase transparency, openness, rigor, and reproducibility.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A data sharing policy that considers "Data available upon request" to be an acceptable response is actually a data not sharing policy.
www.nature.com/articles/d41586-022-03133-5 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The tragedy of injecting partisanship into public health.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
nature
@Nature
Taking the pain out of data sharing go.nature.com/3SThahs
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An interesting paper that prompts lots of room to speculate about complex causal dynamics including potential gender differences in likelihood of publication in the 1st place, factors besides pubs&citations that predict election, and unseen roles of nomination processes.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I like this hat.
soul nate
@MNateShyamalan
i can’t believe billionaires really text each other like this
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Catch up on #openscience events, activities, and perspectives by watching recent COS-sponsored webinars including openness in promotion, data anonymization, improving inclusion, and grants for Registered Reports in consciousness research.
Access here: www.cos.io/events -
Brian Nosek (@briannosek@nerdculture.de) reposted
FHI 360 Research
@fhi360research
@FHI360research-er @Dr_Rachel_Ren shares insights from @OSFramework's @BrianNosek on how to increase uptake of #openscience practices among researchers in the field of #education: researchforevidence.fhi360.org/how-to-increase-uptake-ope…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
NSW Health Statewide Biobank
@NSWHSBiobank
We're excited to welcome Dr Tim Errington @OSFramework to speak on "Barriers to replicating preclinical #cancer biology #research- challenges or opportunities?"
Join us Wed 19 October 2022 16.00 AEDT
Free registration, all welcome 👇
#reproducibility www.eventbrite.com.au/e/statewide-biobank-seminar-series-… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Paul Novosad
@paulnovosad
This interaction shows the limits of our ability to search for truth in the social sciences.
Author: Huge effects
Rebuttal: Zero effects
Author: Yup, huge effects
Journal: 🤷♀️
Adversarial collaboration would be a much better approach here. 1/3
Paul Novosad
@paulnovosad
Does temperature affect judge decisions? These two articles are using literally the same strategy and the same dataset, but finding 3x difference in magnitudes.
All the data and code are public, it would be a great class/twitter project to dig into this.
Links in thread. -
Brian Nosek (@briannosek@nerdculture.de) reposted
The Influenzer Initiative
@TheInfluenzers
Center for Open Science's (@OSFramework) Flu team presents their poster today at #optionsxi on why, despite supporting open science personally, many #influenza researchers do not practice it professionally. Stop by and take a look in the main hall!
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Jörg Ankel-Peters
@jrgptrs
New paper in the JEEA: Do stricter data-sharing policies reduce p-hacking & publication bias in economics? Apparently yes. Introducing data-sharing policies has led to 31% decrease in t-values; especially excess statistical signifiance. academic.oup.com/jeea/advance-article-abstract/doi/10.109…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The amyloid hypothesis rejuvenated?
The CEO certainly thinks so: "The trial results “prove the amyloid hypothesis, ... is one of the main causes of Alzheimer’s disease,” said Haruo Naito, Eisai’s chief executive officer." www.washingtonpost.com/health/2022/09/27/alzheimers-drug-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Excellent to see societies like @FABBS @FASEBorg @theAGU @PsychScience @ASMicrobiology @esa and more come together to support open scholarship.
I hope that others will join!
All4OS
@Alliance4OS
Announcing the Alliance for Open Scholarship (All4OS), a coalition of 10 diverse societies & associations collaborating to identify, articulate, & advance #openscience #opendata #openscholarship norms & practices within their disciplines. More: www.all4os.org
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Side question: Is the logo intentionally almost identical to OSI's? opensource.org
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Added @fda to my list of agencies visited. On behalf of Tim Errington and all our collaborators, thanks @maqcsociety for the award and opportunity to share our Reproducibility Project: Cancer Biology and SCORE projects.
MAQC Society
@maqcsociety
Congrats to our winners:
MAQC Society Award: Dr. Ira Deveson of @GarvanInstitute
Outstanding Reproducibility in Science Award: Dr. @MarzyehGhassemi of @mit
Runner Up: Dr. @BrianNosek of @uva -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
My youngest daughter took a huge breath right after it hit and gushed "Oh my gosh, I was so tense!"
NASA
@NASA
IMPACT SUCCESS! Watch from #dartmission’s DRACO Camera, as the vending machine-sized spacecraft successfully collides with asteroid Dimorphos, which is the size of a football stadium and poses no threat to Earth.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Joel Achenbach
@JoelAchenbach
NEW: DART succeeds -- and screen goes blank in triumph as spacecraft is destroyed www.washingtonpost.com/science/2022/09/26/nasa-dart-missi…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Loren Grush
@lorengrush
OMG!!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A delightful reflective presentation from Danny Kahneman about adversarial collaboration and (un)changing minds.
www.edge.org/adversarial-collaboration-daniel-kahneman -
Brian Nosek (@briannosek@nerdculture.de) reposted
Prof Pete Etchells
@PeteEtchells
This looks extremely damning. A Facebook post appearing to show someone selling authorship for $500 a pop on a journal paper.
Dorothy Bishop (inactive account)
@deevybee
Well, this is interesting.
I never did hear about the promised report by the publisher, (see end of this post deevybee.blogspot.com/2020/07/percent-by-most-prolific-au…), though editor has changed.
It now seems the International Journal of Mental Health and Addictions is in the news again
pubpeer.com/publications/F02B0513A4673C46BED0B042A05940 -
Brian Nosek (@briannosek@nerdculture.de) reposted
ABRIR
@ABRIR_Psy
Great discussion with brilliant panelists @BrianNosek, @Flavio_Azevedo_ and @nataliadutrapsy moderated by wonderful @NSCorralFrias ! #abrir_psy2022
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brendan Ch'ng, PhD
@BrendanChng
@ABRIR_Psy's first ever International Congress is live now!
Head over to abrirpsy.org/events/increased-representation-2022/ to join the conversation on how psychologists from around the world can be better represented in the world of #openscience today! -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
Many advocates struggle to reflect #openscience practices in the academic promotion and tenure process. Join us for a webinar where @doughertyorama will highlight efforts to codify open science as core criteria.
🗓️ Sept. 27, 2022
⏰ 2 pm ET
👉 Register: cos-io.zoom.us/webinar/register/WN_KjOFTPfLQRGzquKsH_Hh5A -
Brian Nosek (@briannosek@nerdculture.de) reposted
Jill ONeill
@jillmwo
We're also welcoming @BrianNosek of the Center for Open Science as our Vision Interview at the end of the day.
NISO
@NISOInfo
Join us a week today for a virtual conference on building #access #openness & #sharing w/ @kaveh1000 @bwbealer @SDawsonBerlin @MarkHahnel & @HeatherKotula - Sept 28 12-4pm ET discounts for NISO members!
www.niso.org/events/2022/09/building-access-openness-and-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Open source service providers: How often do you receive inquiries related to mergers and acquisitions of your organization? I did not anticipate the frequency of this and am curious how common it is.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"A guide to open science practices for animal research" @PLOSBiology by Diederich and colleagues. journals.plos.org/plosbiology/article?id=10.1371/journal.…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
ABRIR
@ABRIR_Psy
It's the final countdown to @ABRIR_Psy workshop series! Meanwhile, watch the opening session: youtu.be/LlsSeubjzok ; and meet the founders of ABRIR: @NSCorralFrias @nataliadutrapsy @alma_jeftic @AshIyer17 Don't forget to register: unisonpsicologia.ca1.qualtrics.com/jfe4/form/SV_79z2vrmxd…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"optimistic take" is my spirit animal.
David Chapman
@Meaningness
🎙 Fascinating: US government policy change is probably about to explode the academic publishing system. Not clear how this will shake out—user-centered process-optimizing SaaS decentralization?—but optimistic take! @BrianNosek with @hertzpodcast
Everything Hertz
@hertzpodcast
161: The memo (with Brian Nosek)
everythinghertz.com/161
We're joined by @BrianNosek to discuss the recent White House memo ensuring free + immediate access to federally funded research, implications of this memo for scientific publishing, and culture change in science -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The essential bit in these debates is that it is perfectly legitimate to treat academia or science as a job, a career, or a calling.
Declaring one as correct and judging others for doing it differently helps precisely zero people. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Tired: Open science means that other people will take my work and I won't get credit.
Wired: Open science provides a “but for” cumulative credit system. Their work would not have been possible but for my work that preceded it.
#openscience enables cumulative science. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The long-term benefits of creating open data infrastructure are huge and hard to anticipate.
For example, I guarantee that I had zero conception when I built Project Implicit in grad school in 1998 that its data would someday be the basis of a huge forecasting tournament.
Landon Schnabel
@LandonSchnabel
I was team "Sociology" in a tournament predicting social change during the pandemic. Out of 120 teams, I ranked #5 across domains (#1 for life satisfaction).
My strategy? Predicting SLOW regression to the mean prior to the pandemic.
Paper now available: psyarxiv.com/wdxsb/
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The benefits of open in this case are that others can build on the work in ways that the originators did not have the insight or capacity to do themselves.
The weekly discovery of new uses of Project Implicit that I never thought of is a great source of joy.
#openscience! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How does the Center for Open Science keep improving on advancing its mission? Easy. We just keep hitting home runs hiring outstanding people.
We are the Aaron Freakin' Judge of hiring brilliant, passionate, mission-driven change makers.
Hannah Moshontz de la Rocha
@hmoshontz
📯 After many years as an enthusiastic @OSFramework user (guiding countless ppl through the UI, defending the UI, owning multiple OSF/PAX tshirts, etc.), I'm pleased + grateful to share that I am officially joining the Center for Open Science as a research scientist next month!
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Hannah Moshontz de la Rocha
@hmoshontz
📯 After many years as an enthusiastic @OSFramework user (guiding countless ppl through the UI, defending the UI, owning multiple OSF/PAX tshirts, etc.), I'm pleased + grateful to share that I am officially joining the Center for Open Science as a research scientist next month!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This makes me ill.
No Lie with Brian Tyler Cohen
@NoLieWithBTC
Wow. An immigration attorney just exposed Ron DeSantis’ multi-million dollar human trafficking scheme and it’s even worse than it sounds.
This media couldn't be displayed. See it on the original post. -
Brian Nosek (@briannosek@nerdculture.de) reposted
PLOS
@PLOS
We're excited to announce an extended partnership with @DataSeerAI to develop Open Science Indicators which will help understand how #openscience is practiced across all PLOS content from 2019 onwards.
Details: plos.io/3eSj1o0 #opendata #opencode #preprints -
Brian Nosek (@briannosek@nerdculture.de) reposted
Dominique Roche
@dom_roche
Not quite the results we were expecting (or hoping for!) but they are what they are...
Work lead by @iliasberberi 👏 (see 🧵⬇️)
OA version: osf.io/preprints/metaarxiv/k8ver
Paper: www.nature.com/articles/s41559-022-01879-9 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It was a treat to have lunch at the German Embassy and the strong support for research quality and transparency around the room provided optimism that the reform movement continues to gather allies and advocates.
Einstein Foundation
@Einstein_Berlin
We would like to thank @GermanAmbUSA Emily Haber for her invitation to a roundtable @GermanyinUSA to discuss ways on how to improve quality in
research and honor the committment shown by the recipients of the 2021 #einsteinfoundationaward. -
Brian Nosek (@briannosek@nerdculture.de) reposted
KathyandRo
@KathyandRo1
Thanks for writing this! The originally planned task force on the code (not just pubs) needs to be reinstated. When 13 of 17 staff, 3 senior members of the GC, 4 pubs committee members and all editors resign, there is a problem that must be addressed.
moin syed
@syeducation
I have withdrawn from the invited program of #srcd23. You should not need to care why, but unfortunately anyone involved in the society needs to care why. @SRCDtweets has been having major problems that cannot be ignored. My detailed letter here: shorturl.at/BNR03
-
Brian Nosek (@briannosek@nerdculture.de) reposted
moin syed
@syeducation
I have withdrawn from the invited program of #srcd23. You should not need to care why, but unfortunately anyone involved in the society needs to care why. @SRCDtweets has been having major problems that cannot be ignored. My detailed letter here: shorturl.at/BNR03
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Einstein Foundation
@Einstein_Berlin
2021 #einsteinfoundationaward winner Paul Ginsparg, Professor of Physics + Information Science @Cornell, received the award for developing arxiv.org, the first platform to make scientific preprints immediately available globally▶️bitly.ws/u6iY 1/3
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is a prototypical case of inaction and lack of transparency by the institution and journal in how it conducted its investigation and drew its conclusions. If the evidence is exonerating, sharing it helps the accused. Secrecy just breeds distrust in everyone involved.
Charles Piller
@cpiller
The perplexing Steven Newmaster saga continues: amid fraud accusations/evidence @BMCMedicine won't retract key article, yet doesn't seem to stand firmly behind it www.science.org/content/article/journal-will-not-retract-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Open call to name the mission to Uranus. What could possibly go wrong?
Ice Giant Missions 🛰
@ExploreIGO
There was:
Voyager - @NSFVoyager2 🌌
Cassini-Huygens - @CassiniSaturn 🪐
New Horizons - @NASANewHorizons 🖤
Juno - @NASAJuno 🟠
Perseverance - @NASAPersevere 🔴
We want to know, what would YOU name the #uranus Orbiter & Probe Mission? ⚪️🛰️
#letsgoback #icegiants #namethemission -
Brian Nosek (@briannosek@nerdculture.de) reposted
Everything Hertz
@hertzpodcast
161: The memo (with Brian Nosek)
everythinghertz.com/161
We're joined by @BrianNosek to discuss the recent White House memo ensuring free + immediate access to federally funded research, implications of this memo for scientific publishing, and culture change in science -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A related personal experience when I was on the lower status side: An Editor rejected my paper saying that they liked it but had to defer to the high status reviewer saying to reject (angrily).
Editor felt guilty, asking me years later what happen to that great paper.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Massive status bias in peer review.
534 reviewers randomized to review the same paper revealing the low status, high status, or neither author. 65% reject low status, 23% reject high status.
Amazing work by Juergen Huber and colleagues. #prc9Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
And, another reviewer explicitly remarked about how they had seen the resistance to this paper and couldn't believe how successful it had been. The Editor at the final journal accepted the paper with enthusiasm.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Because of the overemphasis on prestige journals for promotion and evaluation, that original editor decision could have been career consequential.
Surely, similar rejections happen to other ECRs that are career consequential and we don't hear those stories because they leave. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Heidi L. Williams
@heidilwilliams_
Would you like to do research that informs how governments as well as private philanthropies can best support science and innovation? Think about coming to work with me and my amazing collaborators like @calebwatney and @PaulFNiehaus.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"The EQIPD framework for rigor in the design, conduct, analysis and documentation of animal experiments" by @TimPnin, @Maclomaclee, @dirnagl, and colleagues. www.nature.com/articles/s41592-022-01615-y
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I want a very simple story to be true: More transparency is better for science, always.
I rebut most arguments against the simple story with evidence, theory, or really good rationalizing.
Status bias in peer review is the most challenging reality against what I want to be true
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Massive status bias in peer review.
534 reviewers randomized to review the same paper revealing the low status, high status, or neither author. 65% reject low status, 23% reject high status.
Amazing work by Juergen Huber and colleagues. #prc9 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preprint to this paper: papers.ssrn.com/sol3/papers.cfm?abstract_id=4190976
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Massive status bias in peer review.
534 reviewers randomized to review the same paper revealing the low status, high status, or neither author. 65% reject low status, 23% reject high status.
Amazing work by Juergen Huber and colleagues. #prc9 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Massive status bias in peer review.
534 reviewers randomized to review the same paper revealing the low status, high status, or neither author. 65% reject low status, 23% reject high status.
Amazing work by Juergen Huber and colleagues. #prc9Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Two important method notes: this journal offered to pay for review (explaining high agree rates maybe), and only those in the anonymous invite condition were then invited to the actual review with randomizing which author was revealed.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Here is a preprint to the paper: papers.ssrn.com/sol3/papers.cfm?abstract_id=4190976
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Heather Joseph
@hjoseph
New poll shows deep public support for new @WHOSTP #openaccess policy guidance: www.dataforprogress.org/blog/2022/9/8/voters-overwhelming…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Micah Altman reports evidence that greater gender diversity of editorial boards is associated with *less* likelihood of having open access and open science policies.
Openness and inclusivity/diversity are not necessarily operating in lock step. #prc9 w/@familyunequal -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Anna Abakina presents outcomes of her investigation of a productive paper mill.
451 papers and 800 scholars. Notably, the advertised pricing was lower than most Gold OA pricing. Economical! #prc9
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More work by Anna on paper mills. psyarxiv.com/2yf8z
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Daniel Evanko from AACR presents 208 author responses to AI-assisted discovery of image duplication.
Best response? One author reports intentionally duplicating inappropriately. A win for honesty!
@peerrevcongress #prc9 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A timely, clear, concrete request with clear implications for the collective prosocial impact and....
Joe Weisenthal
@TheStalwart
Wow, check out the timing of the text that California sent out to the public, and then the moment that electricity consumption in the state started to roll over.
www.bloomberg.com/news/articles/2022-09-07/a-text-alert-m… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Three minute videos of the winners of the #fallingwalls Science and Innovation Management prize. Thanks for the acknowledgement of @OSFramework and the other groups doing amazing work to improve science.
falling-walls.com/discover/videos/global-call-winner-2022… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
On first glance, this appears to be the tip of the iceberg uncovering another paper mill. I hope that the Journal of Community Psychology will do its due diligence in response.
Dorothy Bishop (inactive account)
@deevybee
New preprint with @AbalkinaAnna with evidence that a paper mill has infiltrated an established Wiley journal - Journal of Community Psychology
psyarxiv.com/2yf8z/ (thread) -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This paper by @rubinpsych & Donkin is the best critique of preregistration that I have read.
Its main virtue is that it doesn't critique a straw version of preregistration. This is particularly admirable when taking the more difficult position to defend www.tandfonline.com/doi/full/10.1080/09515089.2022.2113771Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The latter point is where the paper's argument is weakest in my view. I find the authors too sanguine about the irrelevance of precommitment for distinguishing and detecting planned versus unplanned analyses.
Even so, a great paper for poking at prereg's benefits and costs
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Oops, mistyped @rubinpsyc in the first tweet, and I don't think Chris is on twitter, but I could be wrong about that.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
tomstafford@mastodon.online
@tomstafford
Come and work with me and lead @RoRInstitute projects coordinating experimental investigations of possible improvements to the research ecosystem! Postdoc @sheffielduni, full time, until 2027. More: x.com/RoRInstitute/status/1565715759799668737 please RT
Research on Research Institute (RoRI)
@RoRInstitute
We are looking for a postdoctoral research associate to work on empirical research on research & metascience projects for the next five years. We anticipate opening applications for these positions in late October 2022 #metaresearch @tomstafford researchonresearch.org/tpost/9jctnsb611-postdoctoral-rese…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Caleb Watney
@calebwatney
Very exciting new job with @FAScientists helping them develop a metascience program!
I highly recommend working with @CorreaDan and @MattHourihan whenever possible.
federation-of-american-scientists.rippling-ats.com/job/48… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A demoralizing illustration of a finding that I wish were not replicable:
Those already marginalized or underserved are the most negatively impacted by destabilizing events. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New from the REPEAT Initiative: 150 reproduction studies and 250 transparency reviews of real-world evidence studies using electronic health records.
Positives: Many studies reproduced closely and no systematic bias toward larger effects observed. www.nature.com/articles/s41467-022-32310-3Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Congratulations to @shirley_wang_v Sebastian Schneeweiss and Sushama Kattinakere Sreedhara for leading this huge project, and thanks to @Arnold_Ventures for funding it. It is an excellent addition to the growing body of evidence about reproducibility across fields.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
More information on the REPEAT Initiative and team is here: www.repeatinitiative.org
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is wild.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Marcin Waligóra
@m_waligora
We use the OSF for our every project. Our raw data are open to everyone. Friendly and easy to use portal. Most recent projects of REMEDY in OSF 'Risk and benefit for umbrella trials in oncology': osf.io/uc5ds/
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thank you to every one of the 500K OSF registered users. You are demonstrating the power of open science and changing the cultural norms.
Collectively, you are providing a foundation of evidence for policymakers to enact openness with confidence.
Video! www.youtube.com/watch?v=udcp3Nuat4w -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
We’re thrilled to announce that the OSF community has reached 500,000 worldwide registered users!
OSF users are leading by example in improving research openness, integrity, and reproducibility. Your adoption is having a viral impact. #openscience youtu.be/udcp3Nuat4w -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thank you to every one of the 500K OSF registered users. You are demonstrating the power of open science and changing the cultural norms.
Collectively, you are providing a foundation of evidence for policymakers to enact openness with confidence.
Video! www.youtube.com/watch?v=udcp3Nuat4wPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How much has open science accelerated?
OSF launched in 2012. In its first year, we had less than 400 users. In 2022, OSF averages 469 new registered users per day, every day.
OSF consumers don't need an account. More than 13M people have consumed OSF content so far in 2022.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Other daily averages for 2022 so far:
202 new projects every day
9105 files posted every day
5964 files made publicly accessible every day
93 studies registered every day
That is....more, a lot more, than ever before.
Keep on keeping on!
#openscience -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brendan Nyhan (@BrendanNyhan on 🟦☁️)
@BrendanNyhan
A horrible market protected by horrible state franchise laws and a lack of consumer protections. Desperate need for legal and regulatory changes.
Tom Gara
@tomgara
An underrated Tesla innovation has been the strategic decision not to sell its cars via a network of duplicitous scumbags www.npr.org/sections/money/2022/08/30/1119715886/inside-t…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Emmanuel Boakye
@emmaboachie
Impressed by the selfless volunteers, mainly from #kenya and #tanzania, who are helping in our project to translate @OSFramework help guides into Swahili.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New on OSF.
Link your data, materials, code, and papers to preregistrations, no matter which repository or publisher you used, by adding the DOI.
The OSTP memo heralds an acceleration toward #openscience, and open infrastructure keeps maturing to make it easy for you to do it.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Just go to one of your registrations on OSF and click on the open resources on the left panel to get started.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Payton Jones
@paytonjjones
We just released a meta-analysis on the efficacy of trigger warnings, content warnings, and content notes (preprint).
Here's a short 🧵 explaining the results with graphs and figures. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is actually a genius idea.
Barak A. Pearlmutter
@BAPearlmutter
How about starting an open access journal, zero fees, which publishes manuscripts accepted by top rank journals but then withdrawn due to open access publication fees.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Computer sales and support at university bookstore.
Chef for vegetarian family.
Assistant director of a summer camp.
Director of a tutoring center immediately after college.
philip lewis
@Phil_Lewis_
Since this is the topic of discussion … What were your jobs in college?
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Oh, forgot assistant at the admissions office.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I like to think about systems change. Today, I was thinking about how to improve college football.
I landed on a promotion-relegation solution that would maximize competition, make many more games consequential, and keep fans at all levels involved.
docs.google.com/document/d/1bKk1LpI45A7FlvIjFd6OrkERgJo2G… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Dr. Glaucomflecken
@DGlaucomflecken
Big news in academic publishing!
This media couldn't be displayed. See it on the original post. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Paid $50 registration fee then learned of more requirements and fees for my daughter to be eligible to take a course. I immediately asked to cancel and get the registration refunded. @ctyjohnshopkins said no, our business model depends on $ from errant registrations. That stinks.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
For completeness, here is the entire exchange.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Green open access is $10 per article, or less.
We can do it. We will do it.
#makeitalloa -
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael 英泉 Eisen
@mbeisen
The American Association of Publishers @AmericanPublish is a serially obstructionist organization that lies constantly to advance a radical anti-science and anti-public agenda. Don't believe ANYTHING they say. Ever.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"The policy, hailed by researchers as 'transformational,' will be fully in place by 2026 and make publicly financed research [papers AND data!] available immediately at no cost."
www.nytimes.com/2022/08/25/us/white-house-federally-funde… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Participants reported feelings of self-doubt, frustration, and depression while experiencing irreproducibility."
Irreproducibility affects the health of science and the mental health of scientists?
Intriguing new paper: osf.io/preprints/metaarxiv/h37kw/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
John Arnold
@johnarnold
Huge kudos to longtime grantees @SPARC_NA and @OSFramework that led efforts to free federally-funded research from paywalls and ensure data from such projects is accessible. These reforms build on our goals to improve the transparency, reproducibility & integrity of research.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
DFG North America
@DFGNorthAmerica
Calling all funders + researchers in/near Washington, D.C. dedicated to fostering #research quality +#openscience: We invite you to join our event with #dfg-VP Julika Griem on Sept. 12 and meet the winners of the #einsteinfoundationaward 2021. Details: tms.aloom.de/login
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The OA policy is transformative for the business models of scholarly communication; major publishers have been preparing for this transition for a decade.
The data sharing policy is transformative for science itself. 1/
www.timeshighereducation.com/news/us-orders-publicly-fund…Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Societies that are ahead of the game will consider shifting their business models to leverage their greatest asset--the expertise of the membership.
Ex: Members could provide peer review for society as service, for pay, or for membership discounts--dumps the rest of publishing.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Also, groundbreaking services like @PeerCommunityIn could dramatically shift the landscape of services and solutions. And, @PLOS's new model w/institutions could spread as a more sustainable alternative to APC models.
Bottom line: Policy is disruptive, many innovations to come.Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
* enables reuse for combining data sources & for asking novel questions
* makes fraud and malpractice more inconvenient & more detectable
* fosters public trust by further distinguishing science as sharing evidence for its claims
* open innovation for services w/ data
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Open access & open data are not sufficient to address all of the challenges for the research culture, rigor, & reproducibility, but they are necessary, major steps toward a research culture that aligns with the ideals for science that we learned in grade school.
Thanks @WHOSTP! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"The White House..directed..agencies to make federally funded studies immediately available to the public after publication, a move that open-access advocates have long pressed for but one that threatens to upend the business models of scientific journals" www.statnews.com/2022/08/25/white-house-directs-agencies-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Capping off the 1st all-team in-person retreat in 2.5 years, the infrastructure team created models of @OSFramework with whatever they found laying around.
This is totally not like how development happens during the rest of the year, promise. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Peter Suber (@petersuber@fediscience.org)
@petersuber
1/ This is big. @WHOSTP is calling on all federal research funding agencies to adopt #openaccess policies. If they already have OA policies, they must strengthen them to meet the new guidelines. I'll add some summary points in a thread below.
White House Office of Science & Technology Policy
@WHOSTP
How is the Biden-Harris Admin working to ensure taxpayer-funded research data & publications advance innovation for ALL Americans?
Today, @WHOSTP is releasing new guidance to make this research immediately available to the public at no cost. Learn more⤵️ www.whitehouse.gov/ostp/news-updates/2022/08/25/ostp-issu… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Chris Chambers
@chrisdc77
When America moves, the world moves
For a preview of further #openscience initiatives to come in the US, see this recent @USGAO report, contributed to by @MarcusMunafo, @BrianNosek, yours truly and many others
www.gao.gov/products/gao-22-104411
Time for UK to step up
UK Reproducibility Network @ukrepro.bsky.social
@ukrepro
Zero embargo for articles, data shared at the time of publication, machine readability, encompassing all federal agencies... Wow👌 www.whitehouse.gov/ostp/news-updates/2022/08/25/breakthro…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I hope that such a big move for US federal funding will prompt similar policy commitments for open access and open data, materials, and code by private funders and journals.
Adoption of TOP Guidelines was already a no-brainer, now it is also a way to be part of the in crowd!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Transformative memo from @whostp: All U.S. federal research funders shall update policies for open access and supporting data without embargo, promote research integrity, and improve equitable delivery of results and data.
Congrats to all that have advocated for #openscience
White House Office of Science & Technology Policy
@WHOSTP
How is the Biden-Harris Admin working to ensure taxpayer-funded research data & publications advance innovation for ALL Americans?
Today, @WHOSTP is releasing new guidance to make this research immediately available to the public at no cost. Learn more⤵️ www.whitehouse.gov/ostp/news-updates/2022/08/25/ostp-issu… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Open Research Funders Group
@OpenResearchFG
The @OpenResearchFG enthusiastically supports updated @WHOSTP policy guidance to increase access to federally funded research. Immediate #openaccess, #opendata, machine readability & more. Big win for #openscience, #equity, #researchintegrity www.whitehouse.gov/ostp/news-updates/2022/08/25/ostp-issu…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We should celebrate everyone's role in this, no matter how big or small. But, a special call out of thanks to @SPARC_NA @hjoseph and the whole team past and present for their years of sweat equity into building support for open access.
Cheers and thank you.
SPARC
@SPARC_NA
BREAKING: @WHOSTP issues landmark new policy guidance setting default to “open” for U.S. taxpayer-funded research #oaintheusa www.whitehouse.gov/ostp/news-updates/2022/08/25/ostp-issu…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Marcus Munafò
@MarcusMunafo
If the UK was ahead it *definitely* isn't now... cc @UKRI_CEO @JeremyFarrar
UK Reproducibility Network @ukrepro.bsky.social
@ukrepro
Zero embargo for articles, data shared at the time of publication, machine readability, encompassing all federal agencies... Wow👌 www.whitehouse.gov/ostp/news-updates/2022/08/25/breakthro…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Transformative memo from @whostp: All U.S. federal research funders shall update policies for open access and supporting data without embargo, promote research integrity, and improve equitable delivery of results and data.
Congrats to all that have advocated for #openscience
White House Office of Science & Technology Policy
@WHOSTP
How is the Biden-Harris Admin working to ensure taxpayer-funded research data & publications advance innovation for ALL Americans?
Today, @WHOSTP is releasing new guidance to make this research immediately available to the public at no cost. Learn more⤵️ www.whitehouse.gov/ostp/news-updates/2022/08/25/ostp-issu… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
My favorite day on grounds.
Jim Ryan
@ProfJimRyan
Yesterday, we played @UVA Move-In Trivia, where I asked new first year students on Grounds some questions about being a "freshman" on "campus," and who is UVA’s president.
This media couldn't be displayed. See it on the original post. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The open science reform movement has been energetic, complex, wonderful, and heated. Chapter 7 of @SMirandaField's dissertation (p 93-121) to be a particularly effective review of the key themes. Recommended as an opportunity for learning and reflection. osf.io/udfw4/
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The visualizations also gave me a burst of sadness seeing @Protohedgehog and recalling his pronounced influence and tragic death.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@juliesquid @carlystrasser @choldgraf @noamross @cboettig @rOpenSci, check out Figure 8.3 (p 153) and the discussion around it about connections and distinctiveness with other communities of practice pursuing reform.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
likewise, @McDawg @jeroenbosman @irynakuchma @resdatall @OpenScienceMOOC @GOFAIRofficial @SPARC_NA @fosterscience see Figure 8.9 (p 161) for visualization of another sub-community and relations with others in the reform space.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Finally, @MarkHahnel @phylogenomics @mbeisen @eLife @PLOS @F1000Research @openscience see Figure 8.5 (p 156) for visualization of another. There are opportunities to strengthen relations and collaboration across these distinct reform efforts to lift all boats!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Republican National Senate Committee really needs to get an unlimited texting plan.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Project Implicit
@ProjectImplicit
We are hiring! We’re searching for a full-time, remote Research Assistant to join our team. Go to our website (www.projectimplicit.net) to learn more about the application process. Review will start on Aug. 26, but apps will be accepted until the role is filled.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Association of Research Libraries
@ARLnews
ARL applauds the ongoing commitment of @NSF to #openscience. The inaugural awards for Findable, Accessible, Interoperable, Reusable, Open Science Research Coordination Networks represent a pooled investment of over $12.5 million in open science www.arl.org/news/arl-applauds-nsf-open-science-investment/
-
Brian Nosek (@briannosek@nerdculture.de) reposted
DFG North America
@DFGNorthAmerica
Calling all funders + researchers in/near Washington D.C. dedicated to fostering #research quality +#openscience! We invite you to join our event with @DFGNorthAmerica +@theNASciences on 9/12 and meet the winners of the #einsteinfoundationaward 2021 too. ➡️www.einsteinfoundation.de/en/events/promoting-research-qu…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is an important paper examining preregistration in econ RCTs. See @Jon_Hartley_ thread and I'll mention a few highlights.
Prereg is supposed to address two issues (1) selective reporting, and (2) clarifying the distinction between planned/unplanned analyses.
Jon Hartley
@Jon_Hartley_
1: Excited to share my new paper with Abel Brodeur, Nikolai Cook (@nikolaimcook) and Anthony Heyes: “Do Pre-Analysis Plans and Pre-Registration Reduce p-Hacking and Publication Bias?” A THREAD (1/N) papers.ssrn.com/sol3/papers.cfm?abstract_id=4180594
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In any case, this is an important contribution to forming an evidence base about the consequences of preregistration and under what conditions it will reduce biases and increase credibility of findings.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I will be very interested to see follow-on investigations that tackle broader operationalizations of the distinction between planned and unplanned analyses than p-hacking per se.
My intuition is that p-hacking is at the more egregious but less frequent end of relevant behaviors. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Paula Andrea
@orchid00
OPEN SCIENCE SOUTH ASIAN NETWORK
5-8th September, 2022
registration is free
ossan2022.net -
Brian Nosek (@briannosek@nerdculture.de) reposted
Journal of Open Psychology Data
@up_jopd
🚨 Big News!!🚨
JOPD now accepts Verification Reports based upon findings from ANY area of psychology and ANY journal!
If you want to try to reproduce (and perhaps extend) analyses, you now have a place to publish!
#openscience #openscholarship #verificationreports -
Brian Nosek (@briannosek@nerdculture.de) reposted
EdArXiv
@EdArXiv
New preprint from @UVAFleming, @MclucasAlan and @Bryan_G_Cook
EdArXiv Bot
@edarxivbot
Review of Four Preregistration Registries for Special Education Researchers
osf.io/7ujgs/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Hey @ChelleGentemann, @EvoMellor turned the NASA TOPS logo into a sticker!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Academics returning to in-person conferences with each other be like
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is the only tweet about "the topic" that I will acknowledge.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Important report from @wellcometrust and demonstration of public accountability and commitment to continue striving toward their values.
Diego
@diegobaptista
A new report indicates @wellcometrust is an institutionally racist organisation--in spite of recent efforts to become antiracist.
There has been limited progress in our efforts to become antiracist as both a funder and employer
wellcome.org/news/update-wellcomes-anti-racism-programme 1/8 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Excellent thread by @CorreaDan on the transformative CHiPS bill. My favorite part is directives to improve rigor and transparency at NSF w/ requirements for data management plans, open repositories, & research/training.
See pages 516-520: www.commerce.senate.gov/services/files/CFC99CC6-CE84-4B1A…
Daniel Correa
@CorreaDan
Today @POTUS signs into law a bill that will transform American manufacturing, science, and innovation. As the CEO of a science org, I’m inspired by the number of grassroots ideas from our community that this bill contains. Here's a behind the scenes look 👇
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Questionable Research Practices, Low Statistical Power, and Other Obstacles to Replicability: Why Preclinical Neuroscience Research Would Benefit from Registered Reports" by @randalljellis www.ncbi.nlm.nih.gov/pmc/articles/PMC9351632/
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Invest in Open Infrastructure
@InvestInOpen
Deadline to apply for this #job is next week - please help us spread the word!👇
Full-time w benefits, remote-friendly 🌍 + an opportunity to contribute to building a healthy #openinfrastructure ecosystem.
Invest in Open Infrastructure
@InvestInOpen
We're hiring!
We're recruiting a research data analyst w a background in #nonprofit management or scientific, scholarly + information management spaces.
More about the role , compensation + links to apply here:
investinopen.org/jobs/
#openscience
#openaccess #openinfra -
Brian Nosek (@briannosek@nerdculture.de) reposted
Ethan Mollick
@emollick
One personality test to rule them all…
This paper finds that somewhere between 71% and 83% of all commonly used psychological scales are basically facets of the Big Five scale, many with extremely high correlations to the Big Five. Look at that chart! psyarxiv.com/vebtm -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Results for perceived ROI on research progress for adopting open science practices. Ranking:
#1 open data, materials, code
#2 replication
#3 Registered Reports
#4 preregistration
#5 open access
#6 preprints
#7 open review
Twitter poll hereby conclusively resolves this question
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How much will broad adoption of open science practices increase return on research investments?
A little stage setting, and then some polls...
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Given the framing of the original question, my ranking would be:
#1 Registered Reports
#2 preregistration
#3 open data, materials, code
#4 open review
#5 replication
#6 preprints
#7 open access
With 1, 2, and 3 far above, then 4 & 5, then 6 & 7 still impactful but much less. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Jon Hartley
@Jon_Hartley_
1: Excited to share my new paper with Abel Brodeur, Nikolai Cook (@nikolaimcook) and Anthony Heyes: “Do Pre-Analysis Plans and Pre-Registration Reduce p-Hacking and Publication Bias?” A THREAD (1/N) papers.ssrn.com/sol3/papers.cfm?abstract_id=4180594
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Torsten Rackoll
@TorstenRackoll
Reflections and experiences from 5 years of the @questbih mission to improve science, translation and add to responsible research.
idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.…
at @BioMedCentral Research Notes
@berlinnovation @ChariteBerlin #openscience @dmoher @BrianNosek -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How much will broad adoption of open science practices increase return on research investments?
A little stage setting, and then some polls...
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Imagine that yearly research progress was countable and that different open science practices had become standard practice. By standard practice, I mean like randomization or blinding today -- not always used or perfectly implemented, but popular and reasonably well implemented.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Compared to the rate of research progress today, with no change in overall investment, what would be the change in yearly research progress if OPEN ACCESS were part of standard practice?
Reply with an estimate, if scale is too narrow to capture your belief.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Compared to the rate of research progress today, with no change in overall investment, what would be the change in yearly research progress if REGISTERED REPORTS were part of standard practice?
Reply with an estimate, if scale is too narrow to capture your belief. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
NEW: Important report from @USGAO
"agencies could do more to increase the rigor and transparency of the research they fund by taking actions to better align awards and recognition for researchers with more rigorous and transparent research practices." www.gao.gov/products/gao-22-104411Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It concludes, collecting information on rigor & transparency "would better position federal agencies to address public concerns over research reliability, or entertain additional strategies to strengthen the yield of a national enterprise with enormous federal investment." Agree!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I recommend reviewing the whole report (direct link: www.gao.gov/assets/gao-22-104411.pdf). Thanks to the GAO for conducting it, and to the many experts that gave their time for interviews to support the effort.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Stuart Buck
@stuartbuck1
The Government Accountability Office (GAO) just released a major report titled "Research Reliability: Federal Actions Needed to Promote Stronger Research Practices."
www.gao.gov/products/gao-22-104411 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Lesson #452 in culture change is hard.
Practices that improve research transparency and credibility can be mainstream but getting all stakeholders to adopt supporting policy is a slog.
Even in clinical trials, there are funders who don't yet have strong registration policies.
TranspariMED
@TranspariMED
ICYMI - Some public research funders in Europe *still* do not require #clinicaltrials to be registered or reported
But strong positive trend overall.
Note the rapid shift to #openaccess across Europe.
www.transparimed.org/single-post/medical-research-funders… @bfarm_de @bonnarek @EMA_News -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Special issue on open science and reproducibility in data science. hdsr.mitpress.mit.edu/volume4issue3
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
WTF
Dr Katie O'Connor
@DrKatieOConnor
Is it normal for a journal editor to request that a research student remove themselves as corresponding author and nominate a more senior scholar? Is it also normal, when the student says no, that the editor asks them to justify authorship based on contributions to the paper? 😡
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This story isn’t an anomaly, it is common.
And, this story is why @OSFramework exists to catalyze culture change in research communities committed to doing better.
Zach London
@zach_london
In my first and only basic science research experience, I was complicit in scientific fraud.
There was no fabrication of data, per se. No doctoring of images. It was subtle experimental misconduct, and certainly an anomaly in the scientific community. I assumed.)
1/🧵 -
Brian Nosek (@briannosek@nerdculture.de) reposted
David A. Sbarra
@dsbarra
#ampps has opened a new call for Adversarial Collaborations, especially Registered Reports focusing on methods or metascience. Improve science: when you disagree, work together. @PsychScience www.psychologicalscience.org/publications/ampps/adversari…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Open science won't prevent fraud from occurring, but it will make fraud more inconvenient and easier to detect.
Open science will also make irreproducible findings harder to sustain.
The problem isn't that false findings occur (fraud or otherwise); it is that they survive. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In a culture without transparency, replication, and reporting null results, the risk and potential impact of fraud can be devastating.
This may become the prototype.
A popular hypothesis, millions invested--so much needless waste in time, $, and lives. www.science.org/content/article/potential-fabrication-res… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In a culture without transparency, replication, and reporting null results, the risk and potential impact of fraud can be devastating.
This may become the prototype.
A popular hypothesis, millions invested--so much needless waste in time, $, and lives. www.science.org/content/article/potential-fabrication-res…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Open science won't prevent fraud from occurring, but it will make fraud more inconvenient and easier to detect.
Open science will also make irreproducible findings harder to sustain.
The problem isn't that false findings occur (fraud or otherwise); it is that they survive. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Nate Breznau
@BreznauNate
A guide to teaching #replication in the social sciences. osf.io/preprints/metaarxiv/ejkws/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Useful overview of why CC-BY-NC is problematic for openness.
A non-commercial addition can feel right to scholars because of suspicions about commercial use, but the costs often outweigh the presumed benefits when restricting certain kinds of uses.
Use CC-BY or CC0 if you can!
Richard Poynder
@RickyPo
For Open Data, think twice before applying Non-Commercial conditions go.nature.com/3cnFV5r
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Good commentaries and reply on evidence for nudges. All highlight ambiguity from publication bias on drawing conclusions from a large literature.
Preregistration helps, but preregistration+outcome reporting needed to definitively address pub bias. www.pnas.org/doi/10.1073/pnas.2202928119
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Infrastructure like OSF is maturing this year to make output and outcome reporting easier to perform and discover.
Lots of norms, training, and evaluation will be needed to scale up adoption and ensure it is working.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Is there a topic area outside of replication work and clinical trials that is consistently using preregistration now and could be a proof of concept for demonstrating the confidence gain in meta-analytic estimates with comprehensive preregistration and outcome reporting?
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Barnabas Szaszi
@szaszibarnabas
d = 0.43 as an overall effect for nudges is implausibly large and misleading.
Our comment is out in @PNAS with @dggoldst @StatModeling @stats_tipton @BalazsAczel @AaronCharlton @AnthonyHigney @ignaziano and David Yeager.
Ungated: www.pnas.org/doi/10.1073/pnas.2200732119
short summary
1/n -
Brian Nosek (@briannosek@nerdculture.de) reposted
Maximilian Maier 🔍
@MaxMa1er
🚨 New commentary in PNAS: "No evidence for nudging after adjusting for publication bias" with @BartosFra, T.D. Stanley, David Shanks, Adam J.L. Harris, and @EJWagenmakers
doi.org/10.1073/pnas.2200300119 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Why don't US colleges split football from the conference system entirely and let the rest of the sports return to sensible regional conferences?
College football could then go all in with an English football promotion-relegation format: en.wikipedia.org/wiki/English_football_league_system
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@PeteThamel any insight on whether such a thing is even theoretically possible?
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In some ways, the promotion-relegation format is perfect for college football. It is dynamic, it brings everyone into a common framework with "equal opportunity," and lots to cheer for at every level -- moving up and avoiding moving down.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Day 1 of family holiday. London. Family sleeping. Beautiful weather. Private patio. Coffee. A perfect start.
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Dishoom!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thank you @DrGBuckingham. Arome was amazing!
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
My spouse and daughters think we walked by @DavidKHarbour by Eaton Square. True or not, now I am going to be hearing about it for the rest of the day.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The microwave in our flat must have been the 2020 winner of the worst usability award.
What, pray tell, happens when I select the water bottle button? -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I mostly experience befuddlement when reading neoliberal critiques of the science reform movement because I can't recognize myself or others' motivations, strategies, or actions in the critiques. @tunc_necip @uygun_tunc offer a very helpful translation. psyarxiv.com/ft8dc
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@tunc_necip @uygun_tunc, I wonder if there is a way that yours and the neoliberal critics' perspectives are both reasonable accounts?
I see yours as better reflecting the grassroots reformers and movement (as organized or diverse it may be).
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Might the critics perspective be a reasonable characterization of institutional (publisher, university, funder) engagement with open science and the reform movement?
ie, Did they get it right, but pointed at the wrong actors, revealing a tension btwn top-down & bottom-up reform? -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
It is okay to prioritize productivity and rigor/integrity simultaneously.
The solution to this seeming oxymoron is to avoid defining productivity as the quantity of publications generated.
Replace it with milestones related to advancing research quality. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Marcus Munafò
@MarcusMunafo
This (by @_MiguelHernan) is referenced in the article, but also worth a read in its own right... www.ncbi.nlm.nih.gov/pmc/articles/PMC5888052/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I finally read "Taboo against explicit causal inference" and it is excellent.
The good intention of avoiding overinterpretation of causation has ironically hurt psych's maturation of theory and evidence.
This paper should be part of every methods course. journals.sagepub.com/doi/full/10.1177/1745691620921521 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Balazs Aczel
@BalazsAczel
Are there secondary benefits of preregistration?
Our respondents think that preregistering improved their workflow, analysis plan, design, but made the project longer.
royalsocietypublishing.org/doi/pdf/10.1098/rsos.211997
with @a_sarafoglou @mrtn_kvcs @bencebakos96 @EJWagenmakers -
Brian Nosek (@briannosek@nerdculture.de) reposted
The New York Times
@nytimes
See the unforgettable scenes of the cosmos captured by the James Webb Space Telescope, the largest and most powerful space observatory ever launched. www.nytimes.com/2022/07/12/science/space/james-webb-teles…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
PLOS Biology
@PLOSBiology
Early-career researchers are working to improve research culture and practice by addressing systemic challenges. This Consensus View from @T_Weissgerber &co provides expert guidance for those who wish to empower the next generation of scientific leadership plos.io/3yn6Y8E
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Dr Madeleine Pownall
@maddi_pow
New preprint "Three Steps to Open Science for Qualitative Research in Psychology" led by @PeterBranney. We identify 3 open research tools that may be appropriate for qualitative researchers:
1⃣contributorship models
2⃣pre-registration
3⃣sharing open data
psyarxiv.com/ahdcu/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is going to be an amazing day for astronomy, and for all of science. I can't wait to see more images. www.washingtonpost.com/science/2022/07/11/nasa-james-webb…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thread to give a sense of COS's direction in metascience.
We are evolving from piecemeal project-to-project research to a programs model with areas of ongoing investment to support our mission and guide effective culture change in reward systems, rigor, and reproducibility
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New job openings @OSFramework with the research team: A research scientist position and multiple project coordinator positions.
Check them out and share with your talented colleagues who are interested in metascience!
www.cos.io/about/jobs -
Brian Nosek (@briannosek@nerdculture.de) reposted
David Mehler
@neuroccino
You are looking to design, review or even preregister an #fnirs study? Our newest article "Using #preregistration as a tool for transparent fNIRS study design" may be just the right place to start! #openscience @SfNIRS
Preprint: osf.io/preprints/metaarxiv/zfawx/
A short 🧵👇 ... -
Brian Nosek (@briannosek@nerdculture.de) reposted
David Perell
@david_perell
What's causing all these logos to look the same?
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
One of the largest moderator effect sizes in the psychological literature may be the difference between original and independent researchers' ability to produce social priming effects.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Meta-analysis of social priming replication studies finds that the strongest predictor of replication success, by far, is an original author being involved: 12/16 success or not 0/49 success. Preregistration also negatively associated with replicability. psyarxiv.com/dwg9v/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Meta-analysis of social priming replication studies finds that the strongest predictor of replication success, by far, is an original author being involved: 12/16 success or not 0/49 success. Preregistration also negatively associated with replicability. psyarxiv.com/dwg9v/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Thread to give a sense of COS's direction in metascience.
We are evolving from piecemeal project-to-project research to a programs model with areas of ongoing investment to support our mission and guide effective culture change in reward systems, rigor, and reproducibility
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New job openings @OSFramework with the research team: A research scientist position and multiple project coordinator positions.
Check them out and share with your talented colleagues who are interested in metascience!
www.cos.io/about/jobsPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We are planning our next steps post-SCORE to do deeper R&D on algorithms for assessing confidence in research claims. This will involve heavy collaboration with AI/ML community and deep dive on uncertainty and bias in algorithms and humans and how to mitigate it.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Final example: @CharlieEbersole led a project examining whether preregistration improves reliability of findings. He's moved on to a great industry job and we are organizing to get that project completed with internal and external collaborative partners.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New job openings @OSFramework with the research team: A research scientist position and multiple project coordinator positions.
Check them out and share with your talented colleagues who are interested in metascience!
www.cos.io/about/jobs -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How good a measure is the number of citations of a paper of its
[1] research quality
[2] scholarly impact -- influence on scholarship
[3] general impact -- all ways that a paper could have impactPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Adjusting for time since publication, number of citations of a paper is _______ related to the scholarly impact of the reported research.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Adjusting for time since publication, number of citations of a paper is _______ related to the general impact of the reported research.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Dr. Tessa Charlesworth
@TesCharlesworth
The PI:International dataset gives implicit and explicit measures of 7 attitudes from 34 countries. I can't wait to use the data myself 🥳 - more than that, I can't wait to see what OTHERS use if for too! With @BenedekKurdi, @mayan_navon, @lofaro_nicole, and Yoav Rabinovich.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
ASAPbio
@ASAPbio_
ASAPbio is thrilled to announce our new initiative - #publishyourreviews, which encourages peer reviewers to publish their reviews alongside the #preprint of an article.
Curious to learn more?
✅ Visit the website: asapbio.org/publishyourreviews
#preprints #openscience #publishing -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
ICYMI: Check out the recording from our webinar on #registeredreports and Funding in #consciousness and Religion Research! Learn more about the benefits of Registered Report and explore current funding initiatives. www.cos.io/blog/registered-reports-and-funding-in-conscio…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
HESI
@HESI_Global
Did you miss the @HESI_Global Environmental Epidemiology Committee #webinar on #transparency and #datasharing in #epidemiology? Recording of joint presentations from Dr. Tim Lash, @EmoryRollins and Dr. @BrianNosek, @OSFramework is now available!
youtu.be/iobUe_GYqUE -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Oh, nice I'm a finalist for the Falling Walls Science and Innovation Management award.
Wait, I have to beat Iron Man?
Am I going to Sakaar? WTF.
falling-walls.com/science-summit/finalists-2022/ -
Brian Nosek (@briannosek@nerdculture.de) reposted
Journal of Politics @jop@sciences.social
@The_JOP
It is time for a bit of courage and innovation:
The JOP will start accepting Registered Reports as a new article format in January 2023.
Publication decisions for RR will be based on the merits of the research question and design, irrespective of the empirical results. 1/11
🧵⬇️ -
Brian Nosek (@briannosek@nerdculture.de) reposted
PCI Registered Reports (@pcirr@spore.social)
@PCI_RegReports
⭐️We're delighted to announce our FIRST completed Stage 2 recommendation⭐️
"Ontological Diversity in Gaming Disorder Measurement: A Nationally Representative Registered Report" by @MKarhulahti and colleagues, recommended by @drcpennington
rr.peercommunityin.org/articles/rec?id=209 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New position @michaeljfoxorg Associate Director, Open Science
"responsible for developing and operationalizing the goals of the Open Science Program..in service of the Foundation’s goal of developing better treatments & ultimately a cure for Parkinson’s."
www.michaeljfox.org/associate-director-open-science -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
2.0%, 15 year. The singular best achievement of my life.
Jesse @ Kleine Real Estate Team
@jesse_kleine
Somewhere out there is an absolute legend that locked in a 10-year fixed rate mortgage at 2.1% in December of 2020.
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael Willis
@mwillispub
Recordings of sessions at today's #nsri2022 seminar available soon at www.nsri2020.nl. Thanks @ridderjeroen @Surveyintegrity & speakers for thought-provoking insights & helpful discussion around #researchintegrity @deevybee @briannosek @lariviev @gowrigopala @naubertbonn
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Presentation slides for my lecture at #nsri2022 about creating sustainable culture change at institutions is available here: osf.io/t7npc
Gowri Gopalakrishna
@gowrigopala
@BrianNosek from @OSFramework telling us of three strategical ways to improve #researchintegrity #nsri2020
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Science Ouverte CS22
@Cs22Ouverte
Netherlands National Open Science Festival :
opensciencefestival.nl Meet - share -inspire - care. Register now for in-person attendance #openscience -
Brian Nosek (@briannosek@nerdculture.de) reposted
Kurt Gray
@kurtjgray
This week I officially became a full professor and just hit my 100th paper.🥳 My one piece of advice for success with papers: more theory of mind.
Let me explain. 🧵 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
To improve inclusion in open science, we are collaborating with the community to translate @OSFramework help guides.
help.osf.io/article/419-translated-help-guides
Portuguese and Chinese are available. Spanish, German, and French are coming.
Webinar to learn or help Jul 20: cos-io.zoom.us/webinar/register/WN_rLQrikImQfS9KaMMSVewfA -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This prompts an interesting idea of a individual-based impact metric akin to "wins above replacement" in baseball:
How much does an authors' papers exceed or fall short of the impact factor of the journals in which they are published?
Consequences of that metric are interesting
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
For example, "overpublishing" a paper in a high impact journal would exact a cost.
Also, identifying the journal with the "right audience" might be more incentivized, regardless of impact factor.
Systemwide, do impact factors then start flattening out? -
Brian Nosek (@briannosek@nerdculture.de) reposted
simine vazire
@siminevazire
Conference dinner for #sips2022 is on brand (expand photo)
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Dr. Priya Silverstein
@priyasilverst
This was my first time running a hackathon and OMG why is SIPS so amazing?!
I will never cease to be amazed that people just come together and share their skills and knowledge and create something beautiful together 😍
#sips2022 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Doom scrolling? This is starting in 10 minutes. Join for a distraction!
HESI
@HESI_Global
Still time to Register! June 29th @HESI_Global Environmental Epidemiology Committee #webinar on #transparency and #datasharing in #epidemiology. Joint presentations from Dr. Tim Lash, @EmoryRollins and Dr. @BrianNosek, @OSFramework
hesiglobal.org/event/transparency-and-data-sharing-in-epi… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Woah, compelling figure. Now I have to read it!
I am immediately interested in SciScore finding the RR's to be scored higher than the original paper when they included only methods, and then the replication study increasing after outcomes were reported in the replication study.
Anita Bandrowski
@Anitabandrowski
@BrianNosek check out what we did with your awesome data #cancerreproducibility in our new paper: doi.org/10.2196/37324
So it looks like SciScore is still highly sensitive to detecting more reproducible papers.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
For those unfamiliar, with @elife, each replication effort in the Reproducibility Project: Cancer Biology produced two papers: A Registered Report reported the background and planned methodology, the Replication Study reported the outcomes. elifesciences.org/articles/67995
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Gowri Gopalakrishna
@gowrigopala
Less two weeks to go to this free webinar on responsible research in academia with @ridderjeroen @BrianNosek @naubertbonn @deevybee @lariviev and Tamarinde Haven. If you’re interested in #openscience #mentoring #researcherassessment #diversity register at www.nsri2020.nl/end-2022
National Survey on Research Integrity
@SurveyIntegrity
Our landmark study at launch two years ago generated much int'l interest. Read it all here: www.nsri2020.nl/news-1 Two years on, we are celebrating our project closure by looking back and forward! Join us in-person or online: free! www.nsri2020.nl/end-2022
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Many researchers say they’ll share data — but don’t"
See also, elifesciences.org/articles/67995 for more challenges in sharing and understanding original findings in biomedical research.
www.nature.com/articles/d41586-022-01692-1 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Rob McIntosh
@mcintord
Today, and last Friday, I had a data reveal party with collaborators to discover the results of a Registered Report together. The excitement of scientific discovery minus the anxiety of publication precarity = joyous.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The importance of preregistration.
According to the thread, the primary endpoint was changed on clinicaltrials.gov from week 12 to week 6 after the study was completed.
Does that influence your assessment of efficacy? -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
Today's #registeredreports and Funding in #consciousness and Religion Research webinar starts in just under one hour, but there's still time to register! Join us to learn about the benefits of Registered Reports and new funding opportunities.
Register: cos-io.zoom.us/webinar/register/WN_H91tNXFhS8Ccz5mL16WzkA -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The journal Research & Politics has a special issue on replication.
Invites replication, reproduction, and robustness studies. Submission deadline Sept 15, 2022
journals.sagepub.com/rap/cfpreplication -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Sneak preview of outcomes from year 2 of the Open Scholarship Survey cos.io/oss across many fields.
On avg, open behaviors (sharing, OA/preprints, prereg) are perceived as positive incentives for getting a job, except for reporting null results just below midpoint. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"I suggest countermeasures against the New Terrorism of open science, open inquiry, and open debate from the point of view of the heroic gatekeeper, Reviewer 2"
ICYMI, like me, this brilliant commentary by @peterdalle
will cheer your weekend. osf.io/t8jsm/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Evaluating Research Transparency and Openness in Communication Sciences and Disorders Journals" psyarxiv.com/dy5zs
-
Brian Nosek (@briannosek@nerdculture.de) reposted
"The Ween Dad"
@ween_dad
Okay, here's the story. I first filed a complaint about this in October 2019. I didn't receive acknowledgement of my complaint until three months later, when I emailed the entire editorial board, twice.
"The Ween Dad"
@ween_dad
One more Qian Zhang retraction. You may remember this one as the experiment that turned out not to be an experiment. onlinelibrary.wiley.com/doi/epdf/10.1002/ab.21836
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Meng Liu
@mengliuveronica
Building a🧵to share my thoughts while I can temporarily put on my participant hat during the breaks at @OpenAppLing's inaugural ReN symposium on #openscholarship in #appliedlinguistics!
-
Brian Nosek (@briannosek@nerdculture.de) reposted
Rachel E. Brenner, PhD
@drrachelbrenner
As a teenager, I felt like I had to hide my sexual orientation; I couldn’t be open and transparent about who I really am. In honor of Pride month, I am partnering with OSF…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In the nomenclature arms race of the overlapping domains of metascience, metaresearch, STS, phil/soc of science, the science of science breaks out with a rap video to establish the dominant brand.
Metascience will have to respond with a dance video.
www.youtube.com/watch?v=9yPVyYYX1Us -
Brian Nosek (@briannosek@nerdculture.de) reposted
The Norwegian Reproducibility Network (NORRN)
@NorRepro
Our hybrid network launch is coming up on June 20!
We're put together a great program featuring @MarcusMunafo, @priyasilverst and @LewendM, practical workshops on using OSF and synthetic datasets + social events.
See the schedule and register here: www.getrevue.co/profile/NORRN/issues/norrn-kickoff-event-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I do not believe I have ever once had a preregistered design and analysis plan go exactly as planned.
Deviations are the norm, not the exception. That doesn't mean we should strive for deviations. It means that we are working on hard problems.
And we are dumb-asses.
Kaylene McClanahan
@KayleneMcClan
There’s nothing more embarrassing than preregistering some half-baked analysis and then having to add a footnote to the manuscript that’s like “we deviated from the preregistered plan bc at the time of preregistration, I was a complete dumb-ass”
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Nice, short read highlights what I think is the biggest misunderstanding for successful innovation:
Start with the assumption that the problem you are trying to solve has been solved already.
behavioralscientist.org/an-in-the-box-method-to-creative-…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Find the most relevant solutions that exist. If they don't perfectly solve your problem already, figure out precisely the gap between what they are and what you want.
This focuses on what innovation is actually needed, builds on existing knowledge & increases chances of success. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New way to avoid misconduct just dropped:
"the committee says the misconduct allegations are not supported because of 'the absence of records and data.'"
"We knew they wouldn’t be able to find records. Our complaint alleged that [he] falsified his work and never had the data.."
Martin Enserink
@martinenserink
Wow. I really did not see this coming.
(thread)
www.science.org/content/article/controversial-canadian-su… -
Brian Nosek (@briannosek@nerdculture.de) reposted
RIOT Science Rotterdam
@riotscienceNL
🚨 New talk coming up 🚨
OPEN AND REPRODUCIBLE NEUROIMAGING: FROM STUDY INCEPTION TO PUBLICATION
⏲️4pm CEST
🗓️ June 2nd
🔗teams.microsoft.com/dl/launcher/launcher.html?url=%2F_%23…
Join us and our wonderful speakers Drs @GuiomarNiso and @rotembot for a discussion of open science and neuroimaging research -
Brian Nosek (@briannosek@nerdculture.de) reposted
Violet Brown
@violetsarebrown
New preprint with @juliafstrand! This tutorial describes the benefits of preregistration specifically for speech, language, and hearing research, and catalogs some experimental details that are important to include in preregistrations in our research area.
psyarxiv.com/f52gj/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
What is your #1 can't miss activity and must try restaurant in London, Paris, or Barcelona?
This summer we are spending a few days in each with our teenagers and my mom. First time for the kids and we'd like to make the most of it! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Attractive job opportunity for a metascience researcher with at least a master's degree.
"J-PAL is recruiting a Research Manager to contribute to its mission of promoting evidence-informed policy, specifically for a major upcoming initiative on the scientific process itself"
Paul Niehaus
@PaulFNiehaus
help launch meta-science @JPAL w/ me and the incomparable Heidi Williams
www.povertyactionlab.org/careers/research-manager-j-pal-g… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Agree w/this thread on making conferences more engaging.
Add-on: There is a place for one-way lectures--sort of like going to see a comedian or politician perform.
Immersion in the speaker's perspective can affect me intellectually in ways that reading their papers does not.
Jay Van Bavel, PhD
@jayvanbavel
I went to a small conference last weekend and it was a revelation because it combined the best of both worlds.
Large psychology conference have moved to very short presentations jammed with multiple studies and nearly zero intellectual engagement.
We should establish new norms. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is the best story about academic cheating that I have ever read. It is high on intrigue and accountability, but it is even higher on empathy and grounding in the goal of teaching -- to engage the students in the material.
An inspiring story about what it means to teach. -
Brian Nosek (@briannosek@nerdculture.de) reposted
Center for Open Science
@OSFramework
Join us for a webinar on 6/29 at 2 pm EDT! Hear from @drmaltman on myths about information privacy that deter researchers from sharing data and selected best practices for the treatment of personally identifiable information in datasets.
Register: cos-io.zoom.us/webinar/register/WN_64akDLW8SQexfQL6zGjcnQ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Reflecting on the history, one might think I was hard done by. Precisely the opposite. The last 10 years has been a gift. The stressors are hugely overshadowed by gratitude and joy for collaborators and supporters making improving science a shared mission. news.virginia.edu/content/after-10-years-many-labs-comes-…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Special & eternal thanks to @johnarnold and @LauraArnoldFdn for making a big bet on our crazy aspiration that we could change the research culture, and to everyone that continues to provide support so that we can put our theory of change into practice and test if it works.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This story was prompted by the completion of the Many Labs series, summarized in this thread.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Today's publication of Many Labs 4 is the 5th (yes, really) and final publication from the Many Labs series of replication studies.
Creating this series was exciting, challenging, & exhausting. I learned a ton. Hopefully, you did too. A short summary...
online.ucpress.edu/collabra/article/8/1/35271/168050/Many… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Michael Nielsen
@michael_nielsen
Nice popular overview history (& biopic) of work done by the Center for Open Science and many others over the past decade:
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Reflecting on the history, one might think I was hard done by. Precisely the opposite. The last 10 years has been a gift. The stressors are hugely overshadowed by gratitude and joy for collaborators and supporters making improving science a shared mission. news.virginia.edu/content/after-10-years-many-labs-comes-…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
SIPS (Account Archived - See Bio)
@improvingpsych
The moment we've all been waiting for: the #sips2022 schedule is now live! We hope you’ll join us June 27-29, either in-person or online, for a dynamic agenda of workshops, hackathons, and unconference sessions. improvingpsych.org/2022/05/20/sips-2022-program-now-live/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Help NASA's Transform to Open Science Program develop curriculum for open data, software, tools, and resources.
Roles for module leads and content contributors, $ awards for contributions! docs.google.com/forms/d/e/1FAIpQLSfAEiBgoRJ-jLWEEBlM0zFIU… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Florian Naudet
@NaudetFlorian
On this #internationalclinicaltrialsday2022, happy to share this new paper from the @ReiTheR_RCT project... Spoiler alert, this paper is not a paper: it is a @RegReports. My first one. Let's celebrate! (thread/ let's make it viral).
bmcmedicine.biomedcentral.com/articles/10.1186/s12916-022… -
Brian Nosek (@briannosek@nerdculture.de) reposted
Lorena Barba @labarba@fosstodon.org
@LorenaABarba
Opportunity to engage with the #nasatops #openscience curriculum development: apply here docs.google.com/forms/d/e/1FAIpQLSfAEiBgoRJ-jLWEEBlM0zFIU…
-
Brian Nosek (@briannosek@nerdculture.de) reposted
COAR
@COAR_eV
COAR welcomes significant funding for the Notify Project www.coar-repositories.org/news-updates/coar-welcomes-sign… @ArcadiaFund
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Progress in academic research, writing, and publishing is rarely marked with climatic endorphin rushes of achievement.
Can we start some positive collusion between editors and colleagues of authors to arrange and record more of these?
They'd surely become a popular video genre.
Dr. Abigail Noyce
@abbynoyce
Highlight of #vss2022: the incomparable @aviaizenman finishes her talk, calls on the first questioner. “Hi Avi, your co-author asked me to stand up and tell you that your paper on this project was just accepted.” Room erupts in cheers.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Scientists and society will be better served by confronting the potential fragility of reported statistical outcomes."
Extolling the benefits of multi-analyst approaches to reveal uncertainty, by @EJWagenmakers @a_sarafoglou and @BalazsAczel
www.nature.com/articles/d41586-022-01332-8 -
Brian Nosek (@briannosek@nerdculture.de) reposted
Paula Andrea
@orchid00
TOPS Community Panel #1
Public virtual meeting
When: 17-19 May 2022, 9-12 PST each day.
#nasatops #openscience #collaboration @NASAEarthData
github.com/nasa/Transform-to-Open-Science/blob/main/docs/… @MalvikaSharan @DrKariLJordan @LouWoodley @BrianNosek and more -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Final call for grad and post-grad data analysts in Economics, International Relations, and Political Science for a SCORE project estimating the analytical robustness of findings from 100 papers. Analysts can become coauthors and receive compensation. Info: osf.io/7snkz/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Today's publication of Many Labs 4 is the 5th (yes, really) and final publication from the Many Labs series of replication studies.
Creating this series was exciting, challenging, & exhausting. I learned a ton. Hopefully, you did too. A short summary...
online.ucpress.edu/collabra/article/8/1/35271/168050/Many… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Across my research on implicit bias, morality, ideology, reproducibility, and open science, I have been on the receiving end of a lot of scholarly criticism. It has been a gift to my career. But, not all criticism is equal.
My primary insight as a recipient... -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A goal of preregistration is to distinguish outcomes of plans made before observing data versus those made after the data have been observed. There is a lot of gray area. Which scenario is the most foreknowledge you accept as still meaning "preregistered"? Poll at end...
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Science journalists play a vital role in scientific communication and @stephaniemlee is one of the best.
It is in the scientific community’s credibility interest to foster a thriving community of journalists with independence from the science that they are covering.
Stephanie M. Lee
@stephaniemlee
Still digesting what today's news will mean for me here, which has been a dream place to work.
For now, I'll say that I'm BEYOND frustrated w/the media business and how hard it is to fund rigorous, original journalism.
Thank you to everyone who's reached out! My DMs are open 💜 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
8-part podcast on science culture and reproducibility
In-depth, comprehensive coverage of the challenge and opportunity to improve research credibility.
3 episodes released so far. Next 5 coming out once per week.
www.healthcaretriage.info/reproducibility-podcast/episode-1 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Special collection @CollabraOA
Has replicability improved?
This Registered Report call for proposals will provide $15K awards for empirical studies examining whether 10 years of reform is associated w/change in replicability of published research.
Call: online.ucpress.edu/collabra/pages/has_replicability_improvedPost not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Contact @siminevazire with pre-submission inquiries. This is strongly encouraged for RR proposals to ensure that the design parameters meet the goals of the special section.
Meta-analysis teams are required to submit a proposal for consideration.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The deadline for proposals is May 1, 2022 to be eligible for the special section and award funds.
This follows a similar successful @RegReports w/funding project launched in 2012 & published in 2014 "Replications of important results in social psychology" econtent.hogrefe.com/toc/zsp/45/3 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Job: Senior Director of Programs
This is a very important new role at COS. The role will be part of the leadership team and have primary responsibility for interventions supporting culture change by stakeholders.
Examples in this role's purview...
candidates.perrettlaver.com/vacancies/2763/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An excellent case example for the value of preregistration.
Perhaps a group in this field will start a registry to stimulate improvements in rigor and transparency. cos.io/registries/
Martin Enserink
@martinenserink
There's a weird thing in science called the 'decline effect': When initial studies show some huge effect that seems to peter out in follow-up research.
A new paper by @biolumiJEFFence and others highlights an extreme example. journals.plos.org/plosbiology/article?id=10.1371/journal.… (1/4) -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"GREI includes six established generalist repositories that will work together to establish consistent metadata, develop use cases for data sharing, train and educate researchers on FAIR data and the importance of data sharing, and more." datascience.nih.gov/news/nih-office-of-data-science-strat…
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We @OSFramework are delighted to be part of this NIH initiative with @datadryad @dataverseorg @figshare @mendeley_com and @VivliCenter
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Two position openings at @OSFramework
Director of Development
candidates.perrettlaver.com/vacancies/2700/director_of_de…
Marketing and Creative Manager
jobs.lever.co/cos/a71322ad-d75a-42ca-ad9e-53b90cf4760f
Check us out and share with your colleagues. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Do researchers update their beliefs about findings following replication success or failure? This excellent study suggests that the answer is yes, just not as much as Bayesian models would expect.
Paper: www.nature.com/articles/s41562-021-01220-7
Green OA: psyarxiv.com/hjcm4/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Pro-tip: you can publish your figures on OSF before the article is accepted and give them a CC-BY license then they will always be freely available.
Daniel BBBradford 🧠⚽️🐶
@drdanbradford
Was just informed by copy editor that both Sage and APS that it is “not common” for authors to post their figures in OSF projects or otherwise online. How common is it?
-
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In an ironic twist, my link at the end of this tweet was to the wrong annotation. Here is the right one:
elifesciences.org/articles/71601 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Improvements to the research culture should continue to promote innovation and exploration but will also incorporate better verification and self-correction to assess whether we know what we think we know.
elifesciences.org/articles/67995
elifesciences.org/articles/71601
www.nature.com/articles/d41586-021-03691-0 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Begley says he has seen a real change..“When I first started talking about this issue, people would get very angry and say, ‘Well, this just proves that Amgen scientists are incompetent’” he says. “Now..the focus is on what should we be doing about this.”
www.newscientist.com/article/2300455-investigation-fails-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Patrick Kane and @KimmelmanJ published a companion commentary about RP:CB in @eLife today. They consider the scientific, ethical, and policy implications of the findings.
elifesciences.org/articles/67527 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
What does a failure to replicate mean? What are the implications of the Reproducibility Project: Cancer Biology showing much less success replicating the original findings? Here's a companion thread to the result summary.
elifesciences.org/articles/71601
elifesciences.org/articles/67995
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Published!
Reproducibility Project: Cancer Biology
@eLife
* 193 experiments planned, 50 completed. Challenges in transparency, sharing, & implementation doi.org/10.7554/eLife.67995
* Replication effect sizes were 85% smaller on avg than original effects doi.org/10.7554/eLife.71601
1/Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We don't know the extent to which these findings generalize. Potential biases include:
* We couldn't conduct replications for many of the systematically selected experiments
* The sampling frame was 2010-2012 high-impact cancer bio papers
* We excluded some work like -omics -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
“I don’t think we should accept this as just the way it has to be,” the director of the US National Institutes of Health, @NIHDirector, told Times Higher Education. “We’re deeply concerned about this, and aim to keep pushing really hard on it.” ($)
www.timeshighereducation.com/news/third-cancer-researcher… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Published!
Reproducibility Project: Cancer Biology
@eLife
* 193 experiments planned, 50 completed. Challenges in transparency, sharing, & implementation doi.org/10.7554/eLife.67995
* Replication effect sizes were 85% smaller on avg than original effects doi.org/10.7554/eLife.71601
1/Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
1st capstone paper summarizes the challenges of conducting replications. We started with 193 experiments from 53 high-impact papers, completed 50 expts:
* data unavailable
* inadequate methods description
* reagents not shared
* variability in author help
doi.org/10.7554/eLife.67995
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Figure 1 summarizes challenges in transparency, sharing, and implementation that escalated time, costs, and reduced feasibility.
Some obvious fixes: fulsome methods, open data & materials, efficient material transfer process, normalize sharing.
doi.org/10.7554/eLife.67995
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We used Registered Reports (cos.io/rr/) with @elife to maximize quality of replications. This meant submitting the protocols for peer review prior to doing the research, & receiving in-principle acceptance to publish regardless of outcomes to reduce publication bias.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
2nd capstone paper summarizes the 158 replication effects from the 50 completed experiments.
Effects are nested in experiments, experiments nested in papers. Report shows all levels (similar results). This thread focuses on effects, see paper for others. doi.org/10.7554/eLife.71601Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Counting successes across 5 different criteria, 40% of original positive effects replicated successfully on a majority of them, 80% of original negative effects replicated successfully on a majority of them.
doi.org/10.7554/eLife.71601
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
92% of the replication effect sizes were smaller than the originals. And, the replication effect sizes were 85% smaller on average than original effects.
Below diagonal replications smaller than originals. Right panel is zoom in on gray box. Blue p<.05
doi.org/10.7554/eLife.71601
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There's high interest in animal experiments given that they are often the onset to translation. Just 12% of animal replications showed p<.05 in same direction as original. But, decline in effect size was similar magnitude as non-animal (~80%). Animal ES were smaller in originals.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Acknowledgments interlude:
Tim Errington did a career's worth of effort leading this project.
@elizabethiorns and @mrgunn were the genesis of this effort.
@Arnold_Ventures, especially @johnarnold @LauraArnoldFdn were the supporters, early and taking a risk on us.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
@stuartbuck1 was a key champion of us and of the project.
200 contributors made contributions along the way and many suppliers provided reagents to support the project. They are acknowledged here: www.cos.io/rpcb-contributors
So many thanks to so many that made this possible.
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
There is much more to add on the meaning and implications of these findings. For now, links:
Overview www.cos.io/rpcb
eLife collection elifesciences.org/collections/9b1e83d1/reproducibility-pr…
data, materials, code osf.io/collections/rpcb/discover
Papers elifesciences.org/articles/67995
elifesciences.org/articles/71601 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
TOP Factor rates the transparency policies of journals, and the website just received an excellent makeover.
topfactor.org
Fabulous work led by @LivLowrey, @LesleyMarkham, and @EvoMellor -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A 3-year position with @templeton_fdn to lead a $15-20M portfolio on applied programs in human sciences. Seeking expertise in research-to-practice, systems change...
An unusual but potentially career shaping option for ECR or mid-career researcher.
Ad: www.templeton.org/careers/program-officer-human-sciences-… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
ICYMI: 68 sessions from #metascience2021 are now available on YouTube: www.youtube.com/playlist?list=PLChfyH8TVDGmYENpXUDPaeeq2S…
There is so much great content on culture, bias, rigor, reproducibility, transparency, tools, equity, peer review, collaboration, metrics, preregistration, measurement, misconduct. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
For the last 2.5 years, my daughters and I have been rating breakfast places in #charlottesville #cville area. We rated 51 restaurants from 1 (worst) to 10 (best) on taste, presentation, menu, ambiance, & service. We also recorded cost-per-person.
Here's what we learned. 1/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Registered Reports phase of this project is complete. We are now releasing the entire dataset for public use. You can still do explore/confirm using exploratory dataset, preregister, and then use confirmatory dataset.
All data, materials: osf.io/gvzwm/
Have fun!
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We are releasing a massive dataset (N ~ 200,000) w/ implicit & explicit measures of attitudes, identities, & individual differences. More than a dozen journals have partnered for doing Registered Reports. Deadline to request exploratory dataset is May 1. docs.google.com/document/d/1zKqFrMzGsQga7XgBOwWGlmLI4aTfG…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Symposium #metascience2021
"Metascience: Disrupting the status quo or perpetuating inequities"
with @DaniSBassett Diego Kozslowski and @csugimoto
Sept 23 12:30p PDT, 3:30p EDT, 9:30p CEST
metascience2021.org/events/metascience-disrupting-the-sta… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Symposium #metascience2021
"Modeling the process of conducting scientific research"
With @a_collentine @EJWagenmakers @JelteWicherts
@amirmasoudabdol @esthermaassen
Sept 18 10a EDT, 4p CEST
metascience2021.org/events/p-hacking-revisited-comprehens… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Symposium #metascience2021
"Unintended consequences of science reforms"
with @BorsboomDenny @cruwelli Anna Dreber @karthikpan and moderators @LeonidTiokhin @Noah_van_Dongen
Sept 17 3:30p EDT, 9:30p CEST
metascience2021.org/events/unintended-consequences-of-sci… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The full agenda for #metascience2021 is online and it is amazing!
metascience2021.org/full-agenda/
Amazing topics, amazing speakers, and an amazing price ($10 students, $20 other, free if you can't pay).
That's 4 amazings! -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We need your feedback to inform outcome reporting workflows for registered studies on OSF.
Whether or not you have experience with OSF and registration, please take this 5-min survey to help make reporting of results and sharing data easy and intuitive: docs.google.com/forms/d/e/1FAIpQLSdJ2ffzD5by0WL8oF-ESqf_v… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New in Nature Human Behavior: We had 353 peer reviewers evaluate published Registered Reports versus comparison articles on 19 outcome criteria. We found that RRs were consistently rated higher on rigor and quality.
Paper www.nature.com/articles/s41562-021-01142-4
Green OA osf.io/preprints/metaarxiv/7x9vy/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Culture change is hard.
-
Laura Fortunato | evolutionary anthropologist
@anthrolog
I have spent a bunch of time, again, working through licence options on #socarxiv 🤯
@socarxiv @OSFramework @OSFSupport why the limited selection of @creativecommons options? E.g. why allow CC-BY, but not CC-BY-SA?
I have looked through all supporting info, pointers welcome 🤓
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Yes. Still work to do here. I use CC0 for most to maximize reuse and rely on disciplinary norms for citation and credit.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Audits play an important role in society, but there was a strong negative reaction to the "transparency audit" project this weekend. Why? I will comment here on just the audit part, not leaderboard, creator, or style.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
COS is hiring a Chief Operating Officer who will focus on coordinating cross-department collaboration between technology, policy, research, operations, and development teams. Please share with colleagues motivated to improve the research culture.
candidates.perrettlaver.com/vacancies/2199/chief_operatin… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Project Free Our Knowledge: Accelerate adoption of open practices via collective action.
Pledge to preregister a study in next 2 years. When 100 in your field do the same, pledges begins & is tracked for completion. Early adopters can inspire others.
freeourknowledge.org/2020-12-03-preregistration-pledge/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Our newest in @PLOSBiology: Ensuring the quality and specificity of preregistrations.
Structured prereg protocols performed better than open-ended ones, but there is still room to improve in design & behavior.
Led by @MarjanBakker & @CoosjeVeldkamp
journals.plos.org/plosbiology/article?id=10.1371/journal.… -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New awards for individuals, institutions, & early-career research proposals from the Einstein Foundation for Promoting Quality in Research--rigor, reliability & transparency.
Sizable prizes of €200,000 & €100,000
Nominations & application due March 31
www.einsteinfoundation.de/en/award/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New paper "Research Quality of Registered Reports Compared to the Traditional Publishing Model" by @cksoderberg Tim Errington @SchiavoneSays @julia_gb @FSingletonThorn @siminevazire Kevin Esterling and me.
Preprint: osf.io/preprints/metaarxiv/7x9vy
1/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Like RPP, Many Labs 5 isn't really about the 10 findings it replicated. It's about the reasonableness of dismissing replication findings by speculating that the replicators messed up.
Perhaps ML5 will shift the baseline plausibility of that rejoinder.
journals.sagepub.com/eprint/VEZTE8C9RDIBQ52CWMCP/full
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Many Labs 5: @cebersole 169 coauthors & I replicated 10 Reproducibility Project: Psychology replications that were not endorsed by original authors. We ran same protocol and revised protocol after peer review. Did replicability improve?
11 papers in AMPPS journals.sagepub.com/eprint/VEZTE8C9RDIBQ52CWMCP/full -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Many Labs 5: @cebersole 169 coauthors & I replicated 10 Reproducibility Project: Psychology replications that were not endorsed by original authors. We ran same protocol and revised protocol after peer review. Did replicability improve?
11 papers in AMPPS journals.sagepub.com/eprint/VEZTE8C9RDIBQ52CWMCP/full -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Our prospective replication study released!
5 years: 16 novel discoveries get round-robin replication.
Preregistration, large samples, transparency of materials.
Replication effect sizes 97% the size of confirmatory tests!
psyarxiv.com/n2a9x
Lead: @JProtzko 1/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Brand New: A Metascience registry for preregistering metaresearch. If you have an OSF project and are about ready to begin, click "add new", connect the project, complete the preregistration, and you might be the very first project added to the registry!
osf.io/registries/metascience/discover -
ASAPbio
@ASAPbio_
New post: the economic argument for preprints asapbio.org/economic-argument-for-preprints
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Nice and there are more angles for the economic argument for preprints. Collectively, they save all the money! www.cos.io/blog/conflict-between-open-access-and-open-sci…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Open Scholarship Knowledge Base is released!
If you have or are looking for papers, videos, lesson plans, training materials for open access, data sharing, replication, preregistration, or any other open scholarship topic submit and find them here!
www.oercommons.org/hubs/OSKB -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Dysfunctional academic incentives, illustrated. nautil.us/blog/you-want-to-see-my-data-i-thought-we-were-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
In this Nature piece, Tim Errington and I discuss how precommitment and adversarial collaboration via Registered Reports could foster more productive debate on replication studies and accelerate progress. www.nature.com/articles/d41586-020-02142-6
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Here's a cool new service for doing a literature search of related papers: connectedpapers.com A sample of the graph for Reproducibility Project: Psychology www.connectedpapers.com/main/ef160aee837c415ee3168efb0fe5…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Conference organizers, if you would like to promote sharing of conference presentations and posters amid COVID-19 cancellations, the free OSF Meetings service is available for authors to easily submit and share their work. Sign-up here: cos.io/meetings/.
Jay Van Bavel, PhD
@jayvanbavel
Dear early career researchers,
If you conference was cancelled, share your poster or talk online & use the conference hashtag.
There is a good chance that you’ll have dozens—if not hundreds—of people who will check it out.
This is far more efficient way to share your work. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Did you miss #metascience2019? Now you can see all of the presentations of fabulous work happening across fields to examine the science of science and explore how we can accelerate progress.
www.metascience2019.org/presentations/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
My brother, @CoachKevinNosek, earns Athlete of the Week honors 20 years since last being on the court. www.davisenterprise.com/sports/dhs-athlete-of-the-week-ke…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
PLOS officially announces adoption of Registered Reports at PLOS ONE. Congrats to @PLOS on this huge step toward shifting publishing incentives toward asking important questions w/ great methods blogs.plos.org/everyone/2020/01/14/registered-reports-are…
-
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
You can support the accelerator on their Patreon page: www.patreon.com/psysciacc
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
If you missed #metascience2019, there will be videos posted online eventually. In the meantime, the presentation and poster slides are available here: osf.io/meetings/metasci2019/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Machine-learning systems are black boxes even to the researchers that build them. That makes it hard for others to assess the results." www.wired.com/story/artificial-intelligence-confronts-rep…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
A great summary of Registered Reports by @chrisdc77 with evidence about what we have learned about the publishing model so far.
This will be the standard piece to send to interested, wary editors to give them some confidence to try it. www.nature.com/articles/d41586-019-02674-6 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Metascience Meeting (metascience2019.org) is a next week (5th-8th) at Stanford. An amazing interdisciplinary line-up of presenters.
If you are not attending, follow along at #metascience2019. Talks will be recorded and released at a later date. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
When I have that judgmental "You really should have known better" feeling about others, I will think of how I used to teach the Stanford Prison Experiment and how exposure of its history revised my beliefs. That class lecture goes to the dustbin. psyarxiv.com/mjhnp/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Webinar tomorrow! Join a discussion with @AnitaEerland @MarcusMunafo @anthrolog and Aleksander Bogdanoski about grassroots community networks for promoting open science.
Register here: zoom.us/webinar/register/WN_3fyuX50_S_2SPWTksDMgDQ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I hate @talyarkoni. Well, what I hate isn't Tal as a person, or Tal's name, or anything actually about Tal's identity. What I hate is that Tal criticizes points-of-view that I like with effective, clear-eyed arguments. He forces me to think. I hate that.
www.talyarkoni.org/blog/2019/07/13/i-hate-open-science/ -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
This is a fascinating paper that will serve as a Rorschach for beliefs about replication. papers.ssrn.com/sol3/papers.cfm?abstract_id=3408200
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Registration for #metascience2019 at Stanford, Sept 5-8, 2019 is almost full. Register now to get a space!
Also, submission form for the poster session is now available: www.metascience2019.org/poster-session-submission/
15 $500 poster awards! Submission deadline is June 10. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
I will be chatting with @aspiegel of @NPRinvisibilia for a #kavliconvo at @nyu_journalism today.
NYers, it is open to the public. Non-NYers, it will be webcast starting at 6:30p EST today. -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The Metascience Meeting
September 5-8, 2019
Stanford University
Come be part of catalyzing this emerging research discipline. An amazing group of speakers and special guests from every area of research with a shared interest in the science of science.
www.metascience2019.org -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
We are releasing a massive dataset (N ~ 200,000) w/ implicit & explicit measures of attitudes, identities, & individual differences. More than a dozen journals have partnered for doing Registered Reports. Deadline to request exploratory dataset is May 1. docs.google.com/document/d/1zKqFrMzGsQga7XgBOwWGlmLI4aTfG…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Many Labs 2: 28 findings, 60+ samples, ~7000 participants each study, 186 authors, 36 nations.
Successfully replicated 14 of 28 psyarxiv.com/9654g
ML2 may be more important than Reproducibility Project: Psychology. Here’s why...
@michevianello @fredhasselman @raklein3 -
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
ATTENTION: New jobs at COS for a major project investigating reproducibility and credibility of social-behavioral science research. Thread with info about positions and links to ads.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
ScienceMag: "A Recipe for Rigor" Profile of the dramatically accelerating adoption of preregistration (osf.io/prereg/) and Registered Reports (cos.io/rr/) in basic sciences, particularly social-behavioral sciences. science.sciencemag.org/content/361/6408/1192
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preprint: "Ensuring the quality and specificity of preregistrations" by @Coosje_Veldkamp and 8 more of us.
"Preregistering research is an acquired skill, and registration formats that provide effective guidance will improve the quality of research." psyarxiv.com/cdgyh -
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Using prediction markets we found that researchers were very accurate in predicting which studies would replicate and which would not. (blue=successful replications; yellow=failed replications; x-axis=market closing price) socarxiv.org/4hmb6/ www.nature.com/articles/s41562-018-0399-z #ssrp
Post not available here, view on X
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Science: "‘Generous’ approach to replication confirms many high-profile social science findings" www.sciencemag.org/news/2018/08/generous-approach-replica…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
New PLOS Biology paper by Toelch & Ostwald: A course for teaching open science tools and practices to improve reproducibility and transparency (journals.plos.org/plosbiology/article?id=10.1371/journal.…). All course content made available for reuse: osf.io/X6892/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preprint @KaitlynHair_ et al. "A randomised controlled trial of an Intervention to Improve Compliance with the ARRIVE guideline" Outcome: Requesting completion of reporting standards not sufficient to improve compliance. www.biorxiv.org/content/early/2018/07/19/370874
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"The Preregistration Revolution" now available in PNAS (OA) by me, @CharlieEbersole, Alex DeHaven, and @EvoMellor www.pnas.org/content/early/2018/03/08/1708274114
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
At #spsp2018 yesterday, I attended a packed session led by Kathleen Vohs and @BJSchmeichel reporting on their many labs investigation of ego depletion. The most important thing I observed was not the results. ->
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Science Magazine enacts TOP Guidelines policies for all articles: open data, materials, code, and disclosure of preregistrations. #openscience www.sciencemag.org/authors/science-editorial-policies
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"There have been two distinct responses to the replication crisis – by instituting measures like registered reports and by making data openly available. But another group continues to remain in denial." thewire.in/208014/replication-crisis-science/
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
An excellent response to well-intentioned but ineffective "implicit bias training". There is little evidence that such trainings reduce implicit bias, or that, even if they did, that the change in implicit bias addresses the outcomes of interest for diversity and inclusion.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Nature: "Stop this waste of people, animals and money" on predatory journals. www.nature.com/news/stop-this-waste-of-people-animals-and…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Science Mag: "It will be much harder to call new findings ‘significant’ if this team gets its way" www.sciencemag.org/news/2017/07/it-will-be-much-harder-ca…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Preprint by @MicheleNuijten et al on the effectiveness of open data policies: Req'd, incentivize, none. Guess which? osf.io/preprints/psyarxiv/sgbta
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Nature's reproducibility checklist will now be published alongside each article: www.nature.com/news/announcement-towards-greater-reproduc…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
COS won "Startup of the year" at #cbicgala last night. Here's why a non-profit is excelling as a tech company cos.io/blog/center-open-science-culture-change-technology…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
"Ten simple rules to consider regarding #preprint submission" @pebourne @jessicapolka Vale, and Kiley journals.plos.org/ploscompbiol/article?id=10.1371/journal…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Doubt the value of open data? Two new papers THIS WEEK re-using Reproducibility Project www.collabra.org/articles/10.1525/collabra.71/, scholar.harvard.edu/kasy/publications/identification-and-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
.@victoriastodden gives one of the opening lectures for the NAS Sackler Colloquium on Reproducibility www.nasonline.org/programs/sackler-colloquia/completed_co…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
ICYMI: Degrees of Freedom in Planning, Running, Analyzing, & Reporting Psych Studies: A Checklist to Avoid p-Hacking journal.frontiersin.org/article/10.3389/fpsyg.2016.01832/…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Comment from Danny Kahneman on his revised understanding of behavior priming (link for full context) : replicationindex.wordpress.com/2017/02/02/reconstruction-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Register for the NAS Sackler Colloquium "Reproducibility of Research: Issues and Proposed Remedies" Mar 8-10 Wash DC www.cvent.com/events/reproducibility-of-research-issues-a…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
WaPo's Monkey Cage blog on Registered Reports challenge for American National Election Studies data: www.washingtonpost.com/news/monkey-cage/wp/2016/08/19/new…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Attention social and behavioral scientists: Competition for publishing preregistered studies of ANES data is coming soon, follow @erpc_2016
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Published @eLife : How Open Science Helps Researchers Succeed elifesciences.org/content/5/e16800
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
The @OSFramework general preprint service now has a landing page, osf.io/preprints/, and is coming soon!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
JPSP editorial mentions tolerating imperfections in data w/improved transparency, important! dl.dropboxusercontent.com/u/133567/psp_110_3_431.pdf.pdf
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
How cognitive biases manufacture beautiful results and what to do about it. Feature story in Nature: www.nature.com/news/how-scientists-fool-themselves-and-ho…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Second story in NY Times on #reproducibilityproject (osf.io/ezcuj) examines reaction in psychology: www.nytimes.com/2015/08/29/science/psychologists-welcome-…
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
To #reproduciblityproject (osf.io/ezcuj/wiki/home) team and supporters. Proud of doing and finishing this. Cheers.
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Psych Science introduced open badges osf.io/tvyxz/ on 1/1/2014 and YOU WON'T BELIEVE WHAT HAPPENED NEXT!
-
Brian Nosek (@briannosek@nerdculture.de)
@BrianNosek
Grants to connect @ZENODO_ORG, @Bitbucket, and @datadryad to the @OSFramework : cos.io/integrationgrants #opensource #openscience





























































































































































































































































