Why you can’t trust drug claims, and what that says about the ability to trust in general

This is an even nerdier piece than usual, and it’s fundamentally about trust, verification, and science… try reading the Peaches saga for something fun, sexy, and actionable…

Game is an open field: it has few definite answers and doing it poorly has few short-term consequences. Drug development is different: it has more definite answers, although the answers happen amid a lot of noise, and has many important short and long-term consequences. Politics is closer to game than to drug development, but it’s not a perfect overlap, since failing or succeeding at game has a strong impact on a given individual… while most political opinions are meant to signal tribal allegiance, and being wrong has little impact on the individual. In the last three+ months there have been lots of dumb claims about how hydroxychloroquine “obviously” works.. and yet we’re still looking for that evidence, which seems less and less likely to exist. The more interesting preliminary commentary was out there, best summed by Derek Lowe… April 6, March 31, April 16… no bullshit and written by someone who knows a lot about drug development… his comments about preliminary studies with small sample sizes are accurate… the early studies showed that hydroxychloroquine didn’t seem to badly hurt anyone (good), but we have law of small numbers problems. The smaller the sample size, the easier it is to find a significant effect through chance. An early and bogus French study was done by a guy who is, to put it uncharitably, frequently full of shit. Yet a lot of guys writing in the game / red pill / right wing worlds went for him. Why?

Those guys often don’t know anything about the field and, in addition, they don’t know what they don’t know. Lots of drugs look promising in vitro or in murine/ferret/etc. models, then fail in humans. Evaluating data from coronavirus is tricky, because most people do recover. It’s possible to give 20 patients the drug and then see most of them recover, because they were at the stage in the disease where they were poised for recovery anyway. These kinds of problems are how and why double-blind trials showed up in the first place, to distinguish cause from effect. These are also the kinds of problems that lead many people to falsely believe in all kinds of cures for colds and flus that were on the verge of clearing up anyway. By now, we know that a large and real trial from the UK with 11,000 patients found no benefit to hydroxychloroquine. France has also suspended trials like this one. A trial of 821 patients didn’t show hydroxychloroquine acts as a prophylactic. Yes, there was a study published in Lancet that was withdrawn due to phony data: but other data is consistent with the “no benefit” hypothesis. In other words, the guys you read on Twitter proclaiming that hydroxychloroquine is an easy win were all wrong, and they were wrong in predictable ways.

A little knowledge is dangerous and most of the people on Twitter know zero about statistics or the history of drug development… they make the same mistakes homeopathy people do. Their conspiratorial mindset flares up. They have no skin in the game: they’ve heard of Nassim Taleb but failed to internalize his lessons. If their recommendations turn out to be correct, they announce how right they were. If their recommendations turn out to be false, they say nothing, or cite the one “maybe” weasel word they used, somewhere. If you can’t trust them on something that has known correct answers, how can you trust them on things that don’t?

Meanwhile, people with skin in the game know that most drugs fail. Twitter has its uses but taking drug recommendations from it is nuts. Then there are Twitter exchanges like this one:

Stedman may know something about men and women (a field with limited opportunities for falsification), but he doesn’t know shit about complex systems or about drugs, and he too doesn’t know it. He doesn’t want to learn, either. People have been trying to get Vitamin C to do something for decades (seriously, Linus Pauling initially made up the idea that vitamin C helps the immune system). Chaga is fine but it’s also been relentlessly studied. He’s a sort of Gweneth Paltrow and Goop for the red pill set: mostly harmless but also overconfident and making unbacked medical claims, relying on the ignorance of his followers. But if he’s wrong about something that can be falsified… what else is he wrong about? He’s also a conspiracy theory guy. And he has a large enough platform that he should try harder not to mislead his readers.

On Twitter, the ignorant are often loud and the most knowledgable often quiet. The ignorant have nothing at stake. Sometimes they are right, too, which is gratifying, when it happens. But what general lessons should we draw?

People are susceptible to showmen. Arguably the game is about becoming a better showman (Mystery was literally a showman: a magician). But the natural world doesn’t care about the show, like the human world does. It’s very reality-based. When dealing with women, some men fail to realize that the show can be more important than the reality, under current social and cultural conditions. When dealing with the human body as a system, the show doesn’t matter… the reality does.

There is a problem, I forget the formal name of it, in which people who have expertise or intelligence in one field, think they know all fields. Their knowledge or expertise doesn’t transfer, though. It’s limited. That’s one way people who are otherwise smart, make stupid mistakes. Stedman doesn’t even realize that what he’s pitching has a long history… he’s making a common mistake but doesn’t know it, and, when I pointed out that he’s wrong, he ignored and muted me. Fine. In terms of the drug world, politics makes people stupid and, oddly, people who know that then accuse others of it, not realizing that they themselves are subject to the challenge.

Meanwhile, here is yet one more piece, an older one, about HCQ not working in late-stage patients, which matches doctors’s anecdotal evidence. That HCQ wasn’t working well in moderate and severe cases became apparent by late March/early April, yet we still saw many on Twitter touting its efficacy… how many docs are writing to game, red pill, or far-right twitter… probably not a lot.

There is an interesting question in why otherwise smart people fall for myths, conspiracy theories, etc. I don’t think the whole answer is there, at the link, and I don’t have a full answer, but self-deception seems to be super common. Stedman falls for it. So do many others.

A gear switch. In game: it’s very tempting to lie to yourself first, but guys do well if they do one of two things: lie to themselves to the point of incredible, delusional confidence (“frame” if you prefer that term), OR be relentlessly honest with themselves about their strengths and especially weaknesses. The human propensity to lie to ourselves seems strong, and in medicine this seems like a particularly powerful tendency. We like to see patterns in randomness. Small parts of humanity have spent the last few centuries trying to learn how not to lie to ourselves. The internet does lots of good things, but it also allows the ignorant to be amplify their ignorance, without realizing their own ignorance.

One logical counter is to say, “Experts have their own problems,” and that’s completely true: but experts being wrong is notable and intersting, while non-experts being wrong is the norm, and many of them don’t even know what they don’t know.

It’s possible that the thousands of people wrongly amplifying their messages will learn something from this… but more likely they won’t. We have centuries of knowledge about how to test drugs already, and one more example of being wrong probably won’t convince anyone, anymore than the homeopathic holdouts can be convinced. Ignorance is the human condition, knowledge the exception. Game is one kind of knowledge, but it’s an imprecise kind. You can be great at game, or a great showman, and know nothing about scientific or technical fields.

There are problems with how to test drugs and other health treatments in the United States… but the noisiest people haven’t been repeating them, mostly. Their knowledge level doesn’t extend that far, and something closer to the truth, doesn’t make it to tweets.

We probably won’t learn much from the hydroxychloroquine debacle, since the people falling for it mostly aren’t or weren’t doctors prescribing medications. Everything I wrote above about statistics and drug development is well-known to people who work in drug development or have learned about drug development and how it works. Everything I wrote above about those topics will probably never be known to people with no skin in the game, no knowledge of statistics, and no downside to being wrong. They were wrong yesterday and will be confidently wrong about something else tomorrow.

Knowing what is really true is hard, which is why it took humans so long to build the civilization we have today. Most of our existence has been spent in superstitious blather. That tradition continues in homeopathy, anti-vaxers, and Twitter.

Most people who think they have secret knowledge are deluding themselves.

In some fields, there is a definitively right answer and a definitively wrong answer. When guys wander into these fields and say things that are likely wrong, or at least unwise, there is a tendency, maybe unfair, to denigrate their knowledge in all other fields.

It’s good to know when you’re part of a show and when you’re part of the study of reality… and a lot of guys online don’t distinguish between the two. Trusting noisy Twitter has its dangers.

Immigration, identity, knowledge part 2

Warning: as with “Get past your identity and look at the data,” “The stink of poly-ticks is high in this post, which has little to do with actual game, so you may want to skip it.” You’ve been warned. You should read “Ms. Slav story updates: Enter new girl Peaches” instead.

There are few fields with larger gaps between the “Twitter world” and the “knowlede world” than immigration. Most people who live in the latter don’t do Twitter as “Twitter natives” do. Among historians, anti-immigration sentiment is almost entirely absent. Why? For one thing, the data show that “Immigrants are doing a great job of becoming Americans.” Plus, historians know that the arguments against immigrants have always stayed the same and have always been wrong. Like Henry Cabot Lodge’s famous speech 1896 speech, “The Problem of Immigration”:

other races of totally different race origin, with whom the English-speaking people have never hitherto been assimilated or brought in contact, have suddenly begun to immigrate to the United States in large numbers. Russians, Hungarians, Poles, Bohemians, Italians, Greeks, and even Asiatics, whose immigration to America was almost unknown 20 years ago, have during the last 20 years poured in in steadily increasing numbers, until now they nearly equal the immigration of those races kindred by whom the United States has hitherto been built up and the American people formed.

In other words, we gotta kick out those foreigners who are different than us. Today, of course, their descendents are making the same anti-immigration arguments that are common on Twitter. Lodge also says:

It is not necessary to enter into a discussion of the economic side of the general policy of restricting immigration. In this direction the argument is unanswerable. If we have any regard for the welfare, the wages, or the standard life of American workingmen, we should take immediate steps to restrict foreign immigration. There is no danger, at present to all events, to our workingmen from the coming of skilled mechanics or trained and educated men with a settled occupation or pursuit, for immigration of this class will never seek to lower the American standard of life and wages

It is necessary; immigration improves American lives and immigrants don’t compete for the jobs Americans do. Funny stories like, “Farmers Finding Few Americans Willing To Do Jobs Immigrants Do” are common. I have friends in the restaurant biz. Try hiring native-born Americans to be dishwashers. The places in the United States with the highest immigration rates also have the strongest economies.

No one arguing against immigration is highly knowledgable about history, or the way their arguments have been used for the last one to two hundred years, and they’ve been wrong the whole time. And anti-immigrant rhetoric is rarely if ever supported by (real) research in peer-reviewed journals. For example, The welfare impact of global migration in OECD countries finds that immigration improves GDP and “recent migration flows have been beneficial for 69% of the non-migrant OECD population, and for 83% of non-migrant citizens of the 22 richest OECD countries.”

We are seeing immigrants create new jobs. Immigration does not create crime and if anything immigrants have lower crime rates, on average, than native-born persons. So why do these memes persist? It seems that humans like to sort ourselves into tribes and it’s fun to create out-groups, and immigrants make handy out groups. Normal people don’t go trolling through the literature and instead form their views on single-hit sensationalist stories and the like. Most people also don’t think about history or their own families’s histories, which, in the United States, always includes immigration somewhere (unless a person is Native American).

The United States is not an ethno-state. It is a set of ideas and ideals. It is also a machine for taking in disparate people and turning their children into Americans (some of whom will in turn adopt anti-immigrant rhetoric). We should be happy this process works and works well. We should also be attentive to the kind of evidence cited by anti-immigrant types. Yes, there are sensationalistic stories about individual bad acts. Just as there are… sensationalistic stories about individual bad acts by people born in the United States. But the anti-immigrant rhetoric is almost totally absent among historians and economists. We should be thinking about why that is. Yes, it’s possible that there’s a giant conspiracy theory. Or, more likely, knowing history makes people chill out about the supposed foreign invasion.

In good news, American support for immigration is at all-time high. I doubt this is because of a newfound love for and knowledge of history, but it is nice.

Overall, Western Civilization is a hardy weed and normal people around the world want TV, convenient food, and hot sex.

I don’t expect to change hearts and minds because almost no one thinks statistically or attempts to systematically review what data exist.