> ## Content Index
> Fetch the complete content index at: https://f4n6.co.uk/llms.txt
> Use this file to discover other available public pages before exploring further.

# Attackers Manipulate AI Chatbots in Mass Disinformation, Phishing Campaign
- URL: https://f4n6.co.uk/security-feed/attackers-manipulate-ai-chatbots-in-mass-disinformation-phishing-campaign/
- Published: 2026-09-23T20:06:16.000Z
- Updated: 2026-09-23T20:06:16.000Z
- Author: Jeff Davies
- Tags: #security-feed

## 1\. Executive summary

Reporting describes two linked criminal techniques: actors seeding the web with malicious links and data and then optimising that content so ChatGPT, Gemini and Google AI Overview return it to users, and separately running AI-brand impersonation campaigns at volume. The substantive detail comes from a Microsoft Threat Intelligence blog published 2026-09-10, which documents campaigns impersonating ChatGPT, Microsoft Copilot, DeepSeek and Claude in support of phishing, search-driven malware delivery and malvertising, including one ChatGPT-themed phishing run that sent up to 100,000 emails in a single day to harvest ChatGPT Plus payment and personal data. The primary item is a headline-level Dark Reading report and adds no technical detail beyond the answer-poisoning claim. No CVE, no CISA KEV entry and no verified reference data apply to this item, and no indicators of compromise were published by either source. For EMEA financial services the exposure is not a compromise of any AI platform; it is trust-borrowing against AI brands plus a new research-integrity problem, where an assistant's answer can carry an attacker-supplied link or download into a workflow that staff treat as authoritative.

## 2\. Regulatory framing

No specific DORA/NIS2 article is directly engaged by this item. The reporting describes a criminal technique and a vendor detection capability, not an incident at a client, and it states explicitly that the AI services referenced were not compromised. Nothing in the source material establishes a reporting trigger, a third-party arrangement or a supply chain event that would change a client's obligations under the articles in the reference. Clients should treat this as threat intelligence input to existing phishing, payment-verification and third-party monitoring controls rather than as a notification event.

## 3\. Technical analysis & attack chain

Two mechanisms are described and they are distinct. The first is answer poisoning: actors seed the web with malicious links and data, then optimise that content so that ChatGPT, Gemini and Google AI Overview surface it in responses. The primary item states this at headline level only. It gives no detail on the seeding infrastructure, the optimisation method, the query classes targeted, or how the poisoned content is delivered to a user once an assistant cites it. Treat the mechanism as reported but uncharacterised.

The second mechanism is AI-brand impersonation, documented in more detail by Microsoft Threat Intelligence. Microsoft reports campaigns that impersonate ChatGPT, Microsoft Copilot, DeepSeek and Claude to make phishing, search-driven malware campaigns and malvertising more convincing. The lures follow familiar social-engineering patterns: a new model release, a policy update from a familiar AI assistant, or a plugin promising to simplify work. One ChatGPT-themed phishing campaign sent up to 100,000 emails in a single day, directing recipients to update their ChatGPT Plus payment information and harvesting personal and credit card data. Microsoft describes the delivery tradecraft as multi-stage redirection chains and disposable infrastructure, with links that appear benign at delivery and later resolve to phishing infrastructure, fake sign-in pages or malicious downloads. Microsoft also names an initial access broker tracked as Storm-3075 using AI-themed malvertising to distribute payloads on behalf of multiple downstream actors, which it presents as evidence the tactic is being commoditised.

What the sources do not claim matters as much as what they do. Neither source reports exploitation of a vulnerability in any AI product, and Microsoft states directly that these campaigns do not represent a compromise of the AI services being referenced. There is no CVE, no patch, no exploited component and no malware family named. The technique is social engineering plus search and answer manipulation, and the defensive problem is that a single lure can present as an email, a link, a download and an identity compromise across separate control surfaces that are often assessed independently.

Confidence caveats. The Microsoft blog is a single source for every technical claim above, including the 100,000-emails-per-day figure, the redirection-chain tradecraft and the Storm-3075 activity. Storm-3075 is a Microsoft cluster designation; no MITRE ATT&CK profile for it is present in the reference data supplied with this item, so the attribution is unconfirmed and should not be used as a basis for blocking or escalation on its own. The answer-poisoning claim rests on the primary item alone and is not corroborated by the Microsoft material, which addresses impersonation rather than assistant-answer manipulation. No indicators of compromise were published by either source, so there is nothing here to pivot on at the atomic level.

## 4\. Mitigation & containment

P1 (within 24 hours)

- Add or verify email gateway and Defender anti-phishing rules for display-name and domain impersonation of the AI brands named in the reporting: OpenAI/ChatGPT, Microsoft Copilot, DeepSeek and Anthropic/Claude. Microsoft's guidance names user impersonation, domain impersonation, first-contact messages and mailbox intelligence signals as the detections that apply to this lure class.
- Enable Safe Links time-of-click verification and Safe Attachments detonation where licensed. Microsoft positions these specifically against redirect chains, delayed activation and links that resolve to malicious content after delivery, which is the tradecraft described here.
- Brief finance, procurement and accounts-payable staff that no payment card or subscription detail is to be changed on the basis of an email about an AI subscription. Require callback verification to a previously known number for any such request. The ChatGPT Plus payment lure is the concrete case in the source.

P2 (within 72 hours)

- Monitor certificate transparency and domain registration feeds for lookalike domains of the four AI brands above and feed confirmed typosquats into DNS and web proxy blocking.
- If staff use AI assistants for research that feeds decisions, require that any URL, download or installer an assistant returns is corroborated against the vendor's own domain before it is opened. This is the only control available against the answer-poisoning mechanism, because no technical indicator of poisoned content was published.
- Block installation of unapproved AI browser extensions and plugins. Microsoft names the "plugin that makes the workday easier" as an active lure theme.

P3 (within 7 days)

- Retain mail flow, web proxy and DNS logs at a window long enough to support retrospective review of multi-stage redirect chains, and confirm that time-of-click verdicts are logged rather than only enforced.
- Retune phishing simulation and user reporting to AI-brand themes so that reporting rates for this lure class are measurable rather than assumed.

## 5\. Indicators of compromise

No atomic indicators of compromise are available in the source material. Neither the primary item nor the Microsoft blog publishes hashes, domains, IP addresses, file names, registry keys or command lines. The sources describe observable behaviours only, listed below.

### Behavioural indicators

| behaviour                                                                                                                             | where to observe                                                                                                | confidence                                                                    |
| ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| High-volume AI-brand phishing lures, including a ChatGPT Plus payment-update theme, at volumes reported up to 100,000 emails in a day | Email gateway and Defender anti-phishing telemetry; sender and display-name impersonation detections            | Single-sourced (Microsoft Threat Intelligence)                                |
| Multi-stage redirect chains where the delivered URL appears benign and resolves to phishing or download infrastructure later          | Web proxy, Safe Links time-of-click verdicts, browser navigation logs                                           | Single-sourced (Microsoft Threat Intelligence)                                |
| Disposable infrastructure registered and burned within a short campaign window                                                        | Domain registration and certificate transparency monitoring; DNS query logs                                     | Single-sourced (Microsoft Threat Intelligence)                                |
| AI-themed malvertising delivering payloads for multiple downstream actors                                                             | Ad telemetry, web proxy referrer chains, endpoint download events                                               | Single-sourced (Microsoft Threat Intelligence); actor designation unconfirmed |
| Assistant answers citing attacker-seeded web content that contains a malicious link or download                                       | Manual review of assistant output used in research workflows; no automated detection available from the sources | Single-sourced (Dark Reading, headline level only)                            |

## 6\. Detection

Insufficient indicators to author detection rules.

## 7\. Sources

- Dark Reading, "Attackers Manipulate AI Chatbots in Mass Disinformation, Phishing Campaign", https://www.darkreading.com/threat-intelligence/attackers-manipulate-ai-chatbots-mass-disinformation-phishing-campaign, 2026-09-23.
- Microsoft Threat Intelligence, "Detect and disrupt AI-themed attacks with Microsoft Defender", https://www.microsoft.com/en-us/security/blog/2026/09/10/detect-and-disrupt-ai-themed-attacks-with-microsoft-defender/, 2026-09-10.

## 8\. Adverse Trace position

We assess this as a medium-priority intelligence item with no direct technical exposure for clients today: there is no CVE, no KEV entry, no malware family and no indicator set, so there is nothing to patch, block or hunt on. The material risk is procedural. AI-brand lures are landing at volumes that will reach finance and procurement inboxes, and the answer-poisoning claim, if it holds, means a link returned by an assistant carries no more implicit trust than a link in an unsolicited email. Both the technical detail and the Storm-3075 attribution rest on a single vendor source, and we are treating the attribution as unconfirmed. We will monitor for corroborating reporting on the answer-poisoning mechanism and for any published indicator set, and we will issue a follow-up if either the technique is characterised in more detail or a second source confirms the campaign volumes. Clients who want the payment-verification and AI-research controls in §4 reviewed against their current procedures should raise it with their Adverse Trace contact.

---

[Read the original source →](https://www.darkreading.com/threat-intelligence/attackers-manipulate-ai-chatbots-mass-disinformation-phishing-campaign?ref=f4n6.co.uk)

*Published via PulseTrace — Adverse Trace threat intelligence.*