Scam Defense Tactic: Lead to Hallucinate

Have you experienced an AI social scam yet? Overall I think new technologies have had a positive impact on the frequency of encountering scams. Still, the ones that do get through these days feel more uncanny than absurd.

"AI scams" seem mostly similar to regular social scams, except now enhanced with the capabilities of large language models. No more poorly-worded half-baked attempts to exfiltrate your assets or steal your trust. Now the half-baked attempts are worded annoyingly well, and with customer-service-level emojis!

I'm sure there are some old-fashioned scams and new-fashioned AI scams that work well and that take time and retrospect and evaluation of the damages to fully recognize. I'm not talking about those.

Here, I'm talking about foreign-prince-in-your-email type nonsense that may have once tricked 1 out of every 1000 sweet-minded people. I have a slight tinge of worry that some situations where new AI tools are employed to fool people could result in an increase to that scammy success rate; especially if our tactics of due vigilance don't proportionally adapt with the technology.

One issue of these AI-powered scams is the disappearance of many of the standard red flags. It is necessary to look for some new red flags now. Some indications of scam these days aren't even distinct "red flags" as much as they are a collection of corroborating evidence that must work together to point to a high likelihood of tom-foolery (or a pulling-your-leggedness, if you will).

So here is just one tiny tool you may try to sus out the suspicious. Lead them to hallucinate. I call it "lead to hallucinate" because that is a short phrase that I think describes the entire process well. If you care to see further explanation, read on.

Imagine you get a strange note out of the blue from a stranger. It could be an email or a direct message on a social platform. Maybe they want to be friends and get to know you better. Maybe they say they want your business services. Maybe you can't find much convincing cross-reference information about them.

Now the first red flag or two here are some that have remained from the before-times. We still have the knee-jerk reaction to assume any cold-calls and outreach are a scam, because so so so many are. The first issue may be some void of cross-reference information. Next, has anyone gotten messages from strangers that aren't scams? Lucky y'all. But even if you have never received a legitimate cold outreach from an unverifiable source, I don't think it is cause to give up faith in humanity. We must retain hope as cold outreach and incomplete online profiles are surely a valid part of some healthy social and professional lives.

So maybe this strange note you get from a strange source sounds like it was written by a machine. Or at least the long answers are. But that doesn't necessarily scream scam on its own either.

Aside time: Here's a quick opportunity to outline one of those non-red-flags that instead is more something like an item of supporting evidence: inconsistent English ability. The short notes may be butchered English that is attempting to masquerade as nonchalance, but the long notes are perfect English that only a seasoned customer-service script copywriter could produce.

It could still be legitimate, regardless of language level incongruencies. Some people need translation and use LLMs. Some people don't want to do it themselves and use LLMs. Some people talk like broke and then may use LLMs for supplement. This is all valid behavior, all still A-OK on its own in the world of scam identification. But if you do notice this pattern, it may be wise to stay extra alert.

Speaking of more evidence that could help classify your uncertain interactions with strangers, I think it would be good to mention another. What is it that the stranger seeks? How eager are they in seeking it? Especially with an amateur or unthoughtful scammer (the general type that can be avoided by most), I think investigating these grounds in tandem can often quickly indicate the validity of an incoming request.

Are they immediately fixated on finding out personal information? Do they perform empty initial connection rituals with vague child-like goals and personal info, and then swiftly move on to extremely pointed questions? I recently had someone who I believe was a scammer do this; they were very focused on finding out if I was happily married and with children. I employed everyday social tactics to avoid the question politely, while also gathering more information to support my suspicion that I was interacting with a malicious entity.

What did I do to find out more information to support my hypothesis? Here's where the "lead to hallucinate" routine comes in. I'll back up and describe the entire exchange so we can see a simple, real-world example of how this tactic is employed. The important parts here are the misinformative squirrel comments:

Do black squirrels exist anywhere? I doubt it. But I do know that there aren't any making strange sounds and "devastating the corn crops" of central OK. Still, this entity who claims they are living in central Oklahoma mindlessly agrees that little black squirrels do exist and make strange noises, and that they are distinct from the regular squirrels who eat acorns.

It isn't hard to lead an LLM to hallucinate. If you want it to say something, it will likely say it to meet it's trained purpose. So using this weakness against an LLM that is being utilized by a malicious person, I think it is easy to find out if the person you are talking to is full of shit and is likely trying to extort something from you.

That's all. That's the whole approach. Simple, but for some reason I thought it was still worth documenting exhaustively.

Of course it all could have ended sooner had I asked for verifiable cross-reference information on this person from the get-go. Although in reality, once I was sure they were scammy, that is how I ended the conversation. I am reluctant to do this immediately as it feels prematurely reactive in the unlikely case that an honorable person is trying to make a productive connection. But next time when I don't have any time for experimentation, this may be a smarter route to start with.

I'm sure there are a plethora of other red flags and evidence of scam here that I haven't gone into. This person wasn't very difficult to suspect or find out. The obvious point is that because the offensive tactics are changing, we should stay on top of updating our defensive arsenal.

Do you see anything I didn't talk about that would alert you right away? Or maybe other things that would collectively support your suspicion? Do you experience AI-enhanced scams at this lazy level, or have you encountered even more sophisticated scam attempts?

By the way, If you aren't a scammer please do send a cold-call hello my way sometime. We can talk about the real animals that really live around the municipality that you really live in.

<3 Grant