AI is getting better at scams. This is bad for me, because I’ve been scammed by humans, and AI will exploit the same weaknesses much more effectively.
While I can’t predict a superintelligence, I can at least try to learn from my gullible past.
“Michael Wong Chang”
My sixth-grade class fell for the Bonsai Kitten hoax years after its debunking. Distressed, I sent the website to my mother; I don’t know what I expected her to do. She explained it was fake.
Weakness: I was ignorant of the world. I didn’t know how to do research. I cared too much about most problems.
Would I fall for this again? I was eleven years old. The hoaxer was an MIT student. They were much smarter than me. The AI will also be much smarter than me. I remain vulnerable.
I’ve gotten better at research. If I take a single minute to consult my defensive AI, it will probably steer me well. I should always take that one minute.
I’ve also gotten better at focusing on real problems. The AI can’t distract me with a kitten. But it could still distract me by threatening my family. I should abandon my family.
…no, that won’t work. I should set up a safeword to ward off deepfakes of my wife or mother being kidnapped (or worse, sealed inside glass jars).
Brian Wansink and the Field of Psychology
In high school, I read psychology books and struggled with orthorexia. Wansink, who studied how subconscious cues make us eat too much, was my favorite scientist. He was also a massive fraud.
In college, before the fraud was known, I asked one of his research assistants about her job. She didn’t seem enthusiastic. I chalked it up to “RA work is boring”; maybe she’d been asked to massage some data.
The bad vibes pushed me off applying to Wansink’s lab, but I still loved his research. He inspired me to intern with the Rudd Center, doing random undergrad tasks to fight child obesity.1 He never scammed me directly, but I’ll have him represent the replication crisis; I wanted to believe in psychology writ large, and I fell for the same nonsense as most Psych majors circa 2015.
Weakness: I broadly assumed that scientists were honest and peer-reviewed papers were trustworthy. I devoured research that matched my views about the world (like sinister food companies tricking us into overeating — which, to be fair, they are).
Would I fall for this again? Could an AI scammer trick me like Wansink? Play on my assumptions so well that I never think to double-check, because it all seems obvious?
Yes, probably. But I can make it harder by taking strong positions only when I have strong and varied evidence. I shouldn’t rely on any single source.2 And I should follow more people who disagree with me; AI will build beautiful echo chambers and I need to hear voices that won’t confirm my every bias.
Gleb Tsipursky
Gleb is a curious figure from the early days of effective altruism — a tireless self-promoter who tried to make his work look popular by creating fake endorsements, fake accounts, and fake social engagement. He got away with it by achieving a few real things, like getting an EA article into TIME (this was a bigger deal back in 2016).
His most toxic practice was tricking earnest young people, including a friend of mine, into taking low-paid contracting roles at his organization (Intentional Insights). He then pressured them to “volunteer” extra hours and “donate” some of their salaries back to him.
Eventually, some of EA’s top researchers made a list of his sins, and our top polemicist wrote him an informal obituary. He left EA for safer climes.3
I never worked for Gleb, but I did invite him to speak to the EA group I ran at Epic Systems. He gave reasonable (if generic) marketing advice, which I’d have taken less seriously had I known he wasn’t good at marketing.
The scam happened after the talk, when he pressured me into paying him (er, “donating to Intentional Insights”). It was like $100, but he hadn’t said anything about a speaker fee beforehand, and the interaction left me with a bad taste in my mouth. I wasn’t surprised to see his org collapse later.
Weakness: I cared more about being charitable than calibrated; I chalked up Gleb’s low-quality work to an entrepreneurial spirit rather than incompetence. I also didn’t think an effective altruist would just… tell lies.
Would I fall for this again? We live in a world of Glebs: today, anyone can buy a squad of volunteers. I should suspect anyone who measures their success by counting free things (likes, clicks, quotes) instead of scarce things (dollars, firsthand endorsements). Online scores were already somewhat fake; they’ll get faker.
Wali Hamidy, DMD
The bonsai kittens toyed with my heart. Brian Wansink exploited my trust in science. Gleb Tsipursky guilted me into a small donation.
Wali Hamidy reached into my mouth, filled it with needless gunk, and charged me several thousand dollars for the privilege. He’s the most effective scammer I’ve ever met.4
It was a shady dentist shakedown like any other: Hamidy claimed that I had horrific dental issues and needed work done right away. (Here’s the full story.) I hadn’t seen a dentist since college, and I didn’t brush twice a day; the guilt was building up like plaque on teeth. I was ready to believe him when he spelled out the wages of my sin.By the time I spoke to my parents about the crown and six fillings — and then spoke on their insistence to my previous dentist, who was baffled — the procedure was done and the bill paid.
Weakness: Confirmation bias goes both ways: I tend to agree with people who call me virtuous or sinful, as long as I already believed it. And while I knew not to trust scientists by then, I still put a lot of faith in doctors (and dentists, why is that a whole different thing?).
Would I fall for this again? This one combines all the other lessons:
- When someone gives you surprising bad news, do your own research before you bite.
- Get a second opinion before you spend 15% of your savings on dental work (or synthesize a virus in your basement for a faceless employer).
- If someone has a good Yelp score, but the good reviews are generic and the bad reviews are detailed individual stories about their evil tricks, don’t trust the average!
Now that I’ve published a complete list of my weaknesses, I feel much safer.
Note: I’m at Inkhaven, so you’ll see a lot of new posts this month.
- As far as I know, the Center’s work is low-impact but legitimate.
- I sure hope we learn to detect when a single AI pretends to be a thousand different sources.
- He’s now an AI consultant with a slop blog at Patheos — a fascinating blend of old and new. I couldn’t have written a more perfect “where are they now?”
- Sam Bankman-Fried cost me a non-refundable plane ticket, but paid me roughly the same amount for being good at video games.