These are all valid! But it’s very important to be clear about which one you’re using. Because here’s something that happens a lot.

There are lots of reasons you might do this.

1. Pure prior: The idea sounds stupid and you haven’t looked at the argument.
2. You’ve looked at the argument, but you think it’s wrong.
3. You looked at the argument, but then realized you don’t have the background to understand it, so you went back to your prior.
4. You looked at the argument, you do understand it, and it looks pretty good. But your prior is so strong you still reject the idea anyway.
5. You looked at the argument, you understand it, it seems strong, and on an intellectual level, it overcomes your prior. But somehow you just aren’t able to get emotionally invested in the conclusion. (Sometimes I feel this way about AI risk.)

If you think about it, almost everything you know comes from other people. Even when you “check the facts” what that usually means is “see what other people say”. If you trace your knowledge back to observations in the world, it’s a huge graph of you trusting people who trust other people who trust other people.

Understanding the world is a social process. This is important because I don’t think the tension of weird ideas can be resolved at an individual level. You’ve got finite time to investigate crackpot theories. But fortunately, you don’t need to resolve all questions yourself. We just need to follow habits that lead to us collectively identifying good ideas and discarding bad ones.

It’s frustrating to propose an idea and have people dismiss it just because it’s weird. You’ve surely seen people ridicule ideas like worrying about wild animal suffering or computers becoming sentient or comets crashing into the planet. I’ve encountered some of this for claiming aspartame is likely harmless but ultrasonic humidifiers might not be.

The thing is, dismissing weird ideas is not wrong.

I have a relative who got the J&J vaccine for Covid, so as some people were getting their third shot, she still only had one. I claimed that it would be fine to go ahead and get a second shot of an mRNA vaccine since this was sure to be approved soon, and was already approved in some countries. She gently responded, “I will get another shot when my doctor tells me to.”

Was she wrong? In a narrow sense, maybe. Mixing-and-matching of vaccines was approved soon after, and I maintain that this was knowable in advance. But more broadly, she was following a good strategy: For most people, “just do what your doctor says” will give better results than, “take unsolicited medical advice from uppity relatives.”

From a Bayesian standpoint, it would arguably have been a mistake if she did listen to me. Skepticism of weird ideas is a kind of “immune system” to prevent us from believing in nonsense.

The problem, of course, is that weird ideas are sometimes right. For 200 years, most Western people thought that tomatoes were poisonous. Imagine you were one of the initial contrarians going around saying, “Well actually, tomatoes are fine!” and demonstrating that you could eat them. I bet you’d have had a rough time.

Especially because if you convinced someone and they went home and cooked some tomatoes, their cookware probably had lead in it which the acidity in the tomatoes would leach out, leading to lead poisoning. Your follow-up campaign of, “really tomatoes are ok, we just need to switch to non-leaded cookware!” would bomb even harder.

I’m glad people persevered so we aren’t covering our pizzas with mayonnaise. But how are we supposed to resolve this tension in general? Here are eight proposed rules.

Alice and Bob are driving through the desert.

Alice: Looks dry.

Bob: That’s wrong, what we see ahead is caused by the sun heating up the road. While the speed of light is constant in vacuum, when light moves through matter, the atoms emit new light that destructively interferes with the old light, in effect causing a “delay”. This happens more with more atoms, meaning light travels slower through denser media.

Alice: OK, but—

Bob: So when the sun heats up the road, this creates a layer of thin warm air with denser cooler air higher up. As light passes through these layers, it refracts upwards towards the denser cooler layer. If conditions are right, this can bend the light back up towards your eyes. Does that make sense?

Alice: Yeah, but—

Bob: Now you might ask, “Why does this look like water?” The situations seem similar at first, with two fluids of different densities. But think about it: With an ocean, the dense water is below the thin air. While, in front of you, the dense cool air is above the thin warm air. The situations are actually reversed!

Alice: Please just—

Bob: With an ocean, there’s a sharp increase in density where the air meets the water. The Fresnel equations say that when light hits such a discontinuity at an angle, most is reflected off the surface. Whereas with the air in front of us, there’s a small and gradual decrease in density, meaning the light is slowly refracted through the lighter warm air and gradually bent back up. In both cases, the mixing in the fluids causes a shimmering effect.

Alice:

Bob: So yeah. Wrong. That’s just a heat mirage.

Meanwhile, the desert flies by, bone dry.