New research from cybersecurity company Avast has pulled back the curtain on a rather uncomfortable truth about New Zealanders and artificial intelligence: we think we’re sharper than we actually are. The 2026 Safe Tech Report found 66 percent of Kiwi adults believe they’d recognise an AI-generated fake at a glance. That’s a reassuring number, isn’t it? It suggests most of us have watched enough deepfake videos and read enough scam warnings to trust our instincts. But then the report digs a little deeper and finds something wobbly beneath that confidence. Forty percent of New Zealand adults admitted to sharing, or seriously considering sharing, sensitive information with an AI tool. Around 21 percent had handed over, or thought about handing over, their CV; another 21 percent had shared or considered sharing a work document. Avast describes this gap between what we believe and what we do as an “AI confidence trap.” We tell ourselves we can spot a fake from a mile away, yet our behaviour suggests otherwise. It’s a bit like being convinced you can spot a pickpocket while leaving your wallet on the café table. The trap isn’t ignorance; it’s overconfidence mixed with convenience. We know the red flags of scams — pressure to act quickly, offers that seem too good to be true, emails that feel slightly off — but we often talk ourselves past them because the tool in front of us is so helpful. The report’s findings are a gentle but firm reminder that in the AI era, online safety isn’t just about recognising threats. It’s about admitting how vulnerable we actually are, and how easily our desire for convenience can override our better judgement. AI is now woven into so many ordinary moments: asking a chatbot to summarise a meeting, draft a reply, or help with a budget. Each of those moments feels small and harmless, which is exactly why the gap between what we say and what we do is so easy to miss.
So what exactly are we sharing? According to Stephen Kho from Avast, the answers are a lot more personal than you might expect. “Because it’s so convenient, we are using these AI tools, these publicly hosted AI tools that are potentially hosted overseas mainly America and China, and we’re uploading our sensitive data … including medical records, financial records because they’re so helpful in helping us plan, identify and do diagnosis,” he told Nine to Noon. Think about that for a moment. We are asking a free online chatbot to help us make sense of a medical letter, plan a household budget, or even draft a resignation email. In return, we paste in dates, names, account balances, and sometimes details about our health or family. It feels private, because it happens in a little chat window on our screen. But Kho points out that uploading a document to one of these platforms is very similar to uploading it to the internet. The information can be stored, processed, and potentially used for model training. The fine print often says exactly that, although most of us never read it. “All these tools, in fact if you look at the fine print it would say you know the information you give there can be used and probably eventually owned by these companies,” he said. We like to think of AI as a clever personal assistant, but the reality is that many AI platforms are businesses with their own interests. Our data is the price we pay for the convenience. That doesn’t mean we can never use AI, but it does mean we should pause before handing over anything we wouldn’t want strangers to see. The convenience is real, and it’s seductive. But every time we paste something sensitive into a public chatbot, we are effectively making a choice to trade long-term privacy for short-term help. The question is whether we’re making that choice with our eyes open. The report’s numbers show this isn’t a niche problem. Twenty-one percent of us have shared or considered sharing a CV, which seems harmless until you remember a CV contains your full name, address, phone number, employment history, and sometimes even the names of referees. Another 21 percent have shared or considered sharing a work document. That could be anything from a meeting agenda to a confidential strategy plan, and once it leaves your hands, you no longer have any control over where it goes.
Kho offers a simple test: if you wouldn’t hand the information to a stranger on the high street, don’t hand it to an AI chatbot. “The idea is that if you’re not confident about handing it to someone on the high street, then you shouldn’t do it to an AI chatbot as well because they are not your confidante, they’re not secret companions or personal experts that have signed legal obligations – these are just in fact AI bots on the internet.” This is a helpful way to cut through the tech. A chatbot might be polite, patient, and endlessly available at 2am, but it isn’t your friend. It doesn’t have a duty of confidentiality, and it hasn’t signed a privacy agreement with you. It’s a piece of software, often running on servers in other countries, with its own terms and conditions. If you do want to use an AI tool for something like financial planning, Kho says make the information anonymous first. Remove your name, your address, your IRD number, anything that could be traced back to you. That way, even if the data is collected or leaked, it’s much harder for anyone to connect it to your life. Kho also warns that insecure configuration of AI platforms can lead to information being dumped into publicly accessible hacked data. Once that happens, the data is traded on the internet and can potentially be used for scams targeting you or your workplace. The threat is not abstract. We have seen data breaches where people’s private conversations, documents, and even login details end up on dark web forums. Scammers can use that material to craft personalised phishing messages that are far more convincing than the generic “you have won a prize” emails of the past. They might mention a project you worked on, a medical appointment you had, or a payment you made. Suddenly, the fake email doesn’t feel fake at all. It feels like someone who knows you, because they do. That’s the danger of treating AI as a confidante. It isn’t a person with a moral compass; it’s a system that may collect, store, and sometimes leak whatever you give it.
The second big theme in the report is that AI is making scams much harder to identify. Kho shared a striking example: a person was called in the middle of the night, just after he’d done a money transfer for his company. He was pulled into a Zoom call with the CEO and CIO. They told him about a secret last-minute transaction related to a merger acquisition, and he needed to transfer $20 million to an account. On all accounts, it was the people he recognised on the board meeting, on the Zoom call. So he did what he was asked to do. It turned out to be a sophisticated fake video of the CIO and CEO asking for the transfer. That story is enough to make anyone’s stomach drop. If a deepfake can convincingly mimic the voice and face of your boss in a live call, then the old advice about checking the sender’s email address or looking for spelling mistakes no longer feels enough. Kho notes that in the past, fake SMS messages, Facebook messages, or emails were often easy to spot because they had incorrect spelling or grammar, and the context was often wrong. “These days those are all gone, so the quality, the speed, how real the fake website is that mimics the shop that you like, they’re all so real now that it’s really really difficult to pick out.” AI has made it quicker to produce scams, and much more believable. It can write a phishing email in perfect English, mimic a family member’s voice, or create a website that looks exactly like your bank’s. The human brain wasn’t built to spot that level of simulation, especially when we’re tired, busy, or distracted. And it’s not just corporate boardrooms that are at risk. Imagine receiving a voicemail that sounds exactly like your teenager asking for money to pay a fine, or a video call from a friend who looks and sounds like them but is actually a digital puppet. These scams are emotionally devastating because they weaponise trust. We don’t fall for them because we’re stupid; we fall for them because they feel real.
Why do we fall into the AI confidence trap? Part of it is psychological. We know the red flags, but we rationalise them away. Kho says people can talk themselves past the warning signs because the scam or the AI tool is convenient. We might see an urgent email from a “bank” and think, “This looks suspicious, but I’ll just click to check.” Or we might know we shouldn’t paste our salary details into a chatbot, but we’re in a hurry and it’s right there. The report’s phrase “AI confidence trap” captures this perfectly: our confidence in our ability to spot a scam actually makes us more vulnerable, because we stop being cautious. We assume we’d notice if something was fake, so we let our guard down. There’s also something about the way AI speaks to us. It feels personal, friendly, and authoritative. It uses our name, answers questions instantly, and never loses patience. That can make us forget that we’re interacting with a machine, not a person. Add to that our natural Kiwi tendency to trust people and give others the benefit of the doubt, and you have a recipe for trouble. It’s not that we’re naive or careless. It’s that the technology has become so good at mimicking human interaction that our instinct to trust kicks in before our critical thinking has a chance to catch up. And because scams now arrive through the same channels we use every day — text, email, social media, video calls — they blend in with our normal lives. We’ve all had that moment where we nearly clicked on something, then stopped and thought, “Wait, that’s not right.” The problem is that AI is making those moments rarer. The fake doesn’t look fake anymore. It looks like a message from a friend, a call from work, a website we’ve used a hundred times. By the time we realise something is wrong, the damage is often already done. The report suggests that this is not simply a knowledge problem. We can list the red flags, recite the warnings, and still fall for a well-crafted scam because the context feels right. That’s the uncomfortable truth about the AI confidence trap: the more confident we feel, the less we question. The best defence is not more confidence; it’s a little more humility and a lot more curiosity. We are social creatures, wired to connect and to trust what feels familiar. Scammers know this. They exploit our kindness, our desire to be helpful, and our reluctance to be rude. A deepfake call from a stressed colleague isn’t just a technical trick; it’s an emotional ambush. When someone sounds panicked and asks for help, our first instinct is to act, not to audit their voice for artifacts. And with AI, the emotional cues are almost perfect. There is no awkward pause, no wrong word, no obvious tell. So we respond the way any decent person would, and that’s exactly what the scammers are counting on.
So what can we do about it? The first step is to accept that we are not as good at spotting AI fakes as we think we are. That sounds negative, but it’s actually freeing: once we stop relying on our instincts, we can start building better habits. Slow down. When you receive a message that asks for money, personal details, or urgent action, pause before responding. Verify through a separate channel: if a CEO asks for a transfer on Zoom, call the company’s official phone number and ask a colleague. If a family member sends a strange request, ring them on their usual number. Never use the contact details in the message itself. For AI tools, share less. Before pasting anything into a chatbot, ask yourself whether you would be comfortable writing that information on a postcard and mailing it to a stranger. If the answer is no, then don’t paste it. If you really need to use an AI tool for something sensitive, remove all personal identifiers first. No names, no addresses, no account numbers, no IRD numbers. Make it impossible for the data to be traced back to you. Also, read the fine print — or at least be aware that what you type into a public AI tool may be stored and used by the company. Treat it like a conversation in a public square, not a private consultation. Finally, keep updating your mental toolkit. The old red flags are gone, so you need new ones: unusual urgency, requests for secrecy, pressure to bypass normal processes, or a sudden change in how someone communicates. Those are the moments to double-check. The goal isn’t to be paranoid; it’s to be present. AI isn’t going anywhere, and it will only get more convincing. But we can decide how we use it. We can use it as a tool without making it our confessor, our banker, or our only source of truth. The most important skill in the AI era isn’t being able to spot every fake. It’s knowing when to stop, ask questions, and protect the things that matter. We’re all in this together, and the more we talk about it, the less shame there is in almost being caught out. The scammers rely on silence. Take away their power by talking openly, checking in with each other, and making it okay to say, “I nearly fell for that.” Because in the AI era, the most human thing we can do is look out for one another.