A few days ago, Chris Inglis wrote a piece in Dark Reading with a thesis that, on first read, sounds almost liberating. We would be at the beginning of the end of social engineering, that old art of tricking people into handing over their access and data. The argument rests on a real shift. Operating systems are becoming "AI-native", with Gemini woven into Android and Apple Intelligence sitting inside the iPhone, the iPad and the Mac. For the first time, they stop merely executing commands and showing information, and start interpreting what you see, hear and receive. The job of staying alert to fraud would move off your shoulders and onto the machine.
I understand the enthusiasm, not least because there is an uncomfortable truth underneath it. User training has failed. For years we told people to distrust emails full of spelling mistakes, and today AI writes better than we do, in our tone and with our context. Asking someone to spot this with the naked eye has become unfair, and close to useless. Taking that burden off the user makes complete sense, and that is not where I disagree. What I cannot bring myself to call the end of social engineering is this shift, because what I see is not the problem disappearing, but changing victim.
When the guard becomes the target
Look at what changes the moment the AI decides, on your behalf, who you can trust. The attacker no longer needs to fool you, and gains a far more profitable target, which is fooling the AI itself. And this is not speculation. It is what we saw in a recent case we covered here on the site, a flaw in the Microsoft 365 assistant where a single hidden instruction inside a trusted link was enough for the assistant to obey and hand over the data without anyone noticing. The target was no longer the person in front of the screen, it was the assistant. And the principle here is the same. The moment you put an AI in charge of judging what deserves trust, it becomes the target worth fooling.
There is also a question of scale that worries me more than the flaw itself. Fooling a human compromises a human, whereas fooling the operating system that protects millions of people compromises them all at once, in silence, without any of them clicking on anything visibly wrong. Centralising trust in a single guard is convenient, but it turns that guard into the most valuable target we have ever built.
The promise is real, but it is early
None of this means the direction is wrong, and it would be dishonest to say so. A system that actively watches out for us is almost always better than hoping every citizen turns into a security analyst in their spare time. My objection is a different one. We still do not know how to make that AI immune to the very trick it is meant to stop, and prompt injection, the underlying problem in all of this (when the text entering the system starts to command it), remains unsolved. Handing the key to trust to something that can still be fooled by well-written text does not end social engineering, it simply promotes it a floor.
What to do until we get there
This is not about refusing these tools, which are here to stay and have real value. It is about not handing them your judgement wholesale:
- Keep a distrust of your own. The AI can warn you, but the final decision about money, passwords and access is yours, and should stay that way.
- Confirm anything critical through a second channel. An unexpected request, even one that looks validated by your phone, always deserves a call or a real person on the other end.
- Demand control and transparency. Before letting an AI decide what is trustworthy, understand what it can reach and what it logs.
- Do not put all your eggs in one guard. Serious security is built in layers, and the AI is one more of them, never the only one.
In the end, social engineering never attacked the machines, it always attacked trust. And as long as trust exists, someone will exploit it, indifferent to whether on the other side sits a distracted person or a diligent assistant. They promised us the end of fraud, but what we have, for now, is fraud quietly changing doors.
Original source: Dark Reading (Chris Inglis).
#StaySafe
🙏🖖
