15 Comments

One thing I liked about Trump becoming president was that he got lots of lay people and such to realize fake news was a thing, I think at first lots of people thought fake news was basically a thing Trump made up but pretty quickly people started to realize that the media couldn't be trusted on a wide range of issues. Interestingly enough the same has yet to happen with respects to science/academia and the replication crisis, to the extent that people do seem sceptical it seems mostly driven my politics as opposed to understanding p-hacking and publication bias and base rate neglect etc. there have been some fairly prominent high status meta science and open science people in recent years but none have really seemed to reach mainstream appeal. I also think its funny that in common parlance doing "research" consists of reading media articles as opposed to becoming familiar with a body of academic literature. My estimation is that most lay people have a very romanticized view of how science is done and view "scientists" as very high status people to be trusted, in a similar way to doctors and such.

One point you mentioned in another piece is that there is a general tendency to use behavioural economics to find examples of market failure and then argue for government intervention, failing to apply insights from behavioural economics as to how governments are also subject to market failure.

Expand full comment

This is a fantastic post about the replication failures / TX sharpshooter. IMO, it would be great to have that as a stand-alone w/o the Danny stuff.

>a psychologist who won, and probably deserved, a Nobel prize

To my eye, this ("probably") sounds like petty jealousy. Like you're smarter than the Nobel committee.

>I made the same point more than twenty years before Kahneman’s book was published

Yeah, but Danny and Amos were doing this work many years before your bit. ;-)

Danny only got to writing his magnum opus later.

>results that played a substantial role in Kahneman’s work.

This is most definitely wrong -- priming was just one small part of his work.

Since I was first exposed to their work in the 90s, I have found evolutionary psychology to be the main source of insight. We're "rational" in that our minds work to get our genes to the next generation ... in our evolutionary setting.

IMHO.

Expand full comment

Dan Ariely's book _Predictably Irrational_ makes the same point as the first half of your post: that not only is a lot of human economic behavior "irrational" in the economist's sense, but it's _consistently_ and _predictably_ irrational, not just noise. For example, under certain predictable circumstances, people treat "free" as dramatically less expensive than even the smallest positive price, and under other predictable circumstances, people treat "free" as equivalent to quite a high price. ("Why are you doing this for free? I'm willing to pay you." "Because if you paid me what I'm worth, you couldn't afford me.")

It too quotes a lot of experiments, and it would be interesting to see how those experiments have fared under replication.

Expand full comment

Are you sure that all the priming results turned out to be non-replicable, rather than just those that were spectacularly unexpected, and hence more attractive to would-be replicators?

I don't have knowledge either way, but my fast system has classed them all as "under suspicion" rather than "definitely wrong" ;-) Obviously there are lots of possibilities. With enough replication attempts, easiest with small samples, some predictable proportion of results will replicate at whatever statistical level one uses, even if the effect sought is non-existent. So cherry picking a few successful replications won't help. But I'd be unsurprised if priming effects (or context effects) were sometimes real, if only because it fits my model of human behaviour, and I'd hate to overcorrect from "not proven" to "definitely false".

Expand full comment