Contact Explore Products

Tag: AI governance

  • Do OpenAI and Anthropic Really Drive 70% of AI Revenue? What It Means for Your Business

    Do OpenAI and Anthropic Really Drive 70% of AI Revenue? What It Means for Your Business

    One statistic is racing around the AI industry:

    More than 70% of AI revenue comes from OpenAI and Anthropic.

    It is a powerful number. It suggests that thousands of AI products, billions in infrastructure spending and the strategies of the world’s largest technology companies rest on two model providers.

    It is also easy to repeat incorrectly.

    The available evidence does not establish that OpenAI and Anthropic collect 70% of every dollar earned across the entire AI market. The figure comes from analyst estimates about a narrower—and in some ways more revealing—part of the ecosystem: the AI-related revenue earned by Amazon, Microsoft and Google from cloud compute, model access and associated commercial arrangements.

    In other words, the claim is less “70% of AI revenue flows to two companies” and more:

    Analysts estimate that OpenAI and Anthropic may directly or indirectly drive more than 70% of the AI-related revenue attributed to the three largest US cloud platforms.

    Some of that money flows from OpenAI and Anthropic to cloud providers for compute. Some comes from cloud customers buying access to their models. The exact totals are not disclosed cleanly by the companies and the estimates differ by analyst.

    That correction weakens the viral headline—but strengthens the business lesson.

    The AI economy may be broader than two companies. The commercial infrastructure underneath it is still remarkably concentrated. If your marketing workflow, ecommerce operation or software product depends on one foundation-model provider, you are not merely choosing a tool. You are inheriting that provider’s pricing, availability, policy and strategic risk.

    Fact check: what does the 70% figure actually measure?

    The source behind the current discussion is Ed Zitron’s analysis, “The AI Demand Bubble”. It combines estimates attributed to analysts at Barclays, UBS and Wells Fargo.

    The estimates cited in that analysis include:

    • Amazon Web Services: One Barclays estimate put OpenAI and Anthropic at 73% of AWS AI revenue in 2026 and 2027. A separate estimate cited in the same article put their direct compute contribution at 59% in 2026.
    • Google Cloud: UBS estimates cited in the analysis assigned 28% of total 2026 Google Cloud revenue to OpenAI and Anthropic, rising to more than 48% in 2027. The author then inferred that the pair could represent at least 70% of Google’s narrower AI-related revenue.
    • Microsoft: Wells Fargo estimates cited in the article put OpenAI and Anthropic at 70% or more of Microsoft’s AI revenue, reaching approximately 74% in the relevant forecast period.

    These are not one consistent, audited market-share dataset. They mix:

    • direct purchases of computing capacity;
    • cloud revenue associated with the two laboratories;
    • revenue from platforms that resell access to their models;
    • analyst forecasts for future periods;
    • the article author’s own classification and inference.

    The cloud companies do not publish a standard “AI revenue” line that allows outsiders to calculate a definitive global market share. OpenAI and Anthropic are also private companies, so their financial disclosure is more limited than that of a public company.

    The accurate version of the claim

    Use this formulation:

    Analyst estimates suggest OpenAI and Anthropic account for roughly 70% or more of the AI-related revenue attributed to Amazon, Microsoft and Google, although definitions and estimates vary.

    Avoid this formulation:

    OpenAI and Anthropic receive 70% of all revenue in the global AI industry.

    That broader statement would require a defined market covering chips, cloud infrastructure, enterprise software, consumer subscriptions, services, advertising, robotics, data platforms and other AI-related businesses. The cited analysis does not provide that calculation.

    Why the corrected number still matters

    The statistic is not a clean measure of the whole AI market, but it exposes three forms of concentration.

    1. Demand concentration

    Cloud providers have invested extraordinary amounts in data centres, accelerators and power capacity. If a large share of the associated revenue depends on two customers and their models, the infrastructure boom has a narrower demand base than the headline “AI adoption” numbers imply.

    This does not prove the boom will collapse. It means future growth depends heavily on OpenAI and Anthropic continuing to:

    • attract paying customers;
    • raise or generate enough cash to fund compute;
    • turn model capability into sustainable demand;
    • serve workloads efficiently enough to support margins;
    • maintain favourable relationships with cloud partners.

    2. Model concentration

    Many applications are not independent AI businesses in a technical sense. They are interfaces, workflows or specialised data layers built on a small number of foundation models.

    That can be a perfectly good business. Shopify did not need to build a payment network, and SaaS companies do not manufacture their own processors. Specialisation creates value.

    The risk begins when the application has no meaningful advantage beyond one provider’s output and cannot operate if that provider changes.

    3. Strategic concentration

    OpenAI and Anthropic influence more than model quality. Their decisions can shape:

    • token and subscription prices;
    • API limits and access tiers;
    • context-window and tool-use behaviour;
    • model retirement schedules;
    • safety policies and refused use cases;
    • data-processing terms;
    • regional availability;
    • integration standards;
    • which workflows become economically viable.

    A business built on top of one provider may experience these decisions as product changes—even when it had no voice in making them.

    What happens if one of the two stumbles?

    “Stumbles” does not have to mean bankruptcy. For a customer, smaller changes can create the same operational effect.

    Prices rise

    If inference pricing increases or a subsidised product becomes more expensive, an application with weak margins may become uneconomic overnight.

    This is especially dangerous when the company offers customers a fixed monthly price while paying the model provider per token, image, tool call or unit of compute.

    A model or feature is retired

    Prompts tuned for one model do not automatically behave the same on its replacement. Output format, tone, refusal patterns, latency and tool selection can all change.

    Without regression tests, a “simple upgrade” can quietly damage product listings, customer replies, campaign copy or structured data.

    Reliability declines

    An outage at the foundation-model layer can stop every workflow built above it. Even partial degradation—higher latency, elevated errors or inconsistent tool calls—can create queues, duplicate actions and failed customer experiences.

    Regulation or litigation changes access

    New regulatory restrictions, court decisions, government procurement rules or regional compliance requirements can affect how models are offered and which data may be processed.

    The correct response is not to predict one dramatic ban. It is to ensure that a single legal or policy change cannot disable an essential workflow without an alternative.

    Provider strategy shifts

    A model company can enter your category directly, prioritise enterprise contracts, discontinue a partner feature or bundle functionality that makes your product less differentiated.

    Platform risk is not only technical. Your supplier can become your competitor.

    What AI market concentration means for marketers

    For marketers, the immediate temptation is to treat model choice as a creative preference: Which assistant writes the strongest hooks? Which one follows brand voice best? Which one creates the most attractive images?

    Those questions matter, but operational dependence matters more.

    A marketing stack may use one provider for:

    • campaign research;
    • segmentation ideas;
    • advertisement variations;
    • product copy;
    • email personalisation;
    • image generation;
    • social scheduling;
    • performance analysis.

    If every step depends on one vendor, a policy update or service interruption can stop the entire content pipeline.

    The better approach is to distinguish between creative preference and business-critical dependency.

    • It is reasonable to prefer one model for campaign concepts.
    • It is risky if no other model can render the required data structure.
    • It is reasonable to use one assistant for drafts.
    • It is risky if brand knowledge exists only inside that provider’s proprietary workspace.
    • It is reasonable to optimise prompts for quality.
    • It is risky if no regression suite tells you when an update changes the output.

    Keep brand guidelines, approved claims, product facts, audience definitions and reusable prompt templates in systems you control. The model should consume your marketing intelligence, not become the only place where it exists.

    What it means for ecommerce brands

    Ecommerce companies have a deeper dependency problem because AI is moving from content generation into operational action.

    Models increasingly help with:

    • catalogue enrichment;
    • onsite search and recommendations;
    • customer-service responses;
    • translations;
    • merchandising analysis;
    • campaign creation;
    • pricing recommendations;
    • returns and order workflows.

    The closer AI gets to customers, orders and money, the more expensive provider concentration becomes.

    Your catalogue must remain the source of truth

    Store product attributes, claims, translations and policy rules in your own product-information or commerce systems. Do not let one model’s memory or proprietary knowledge feature become the authoritative record.

    If you are evaluating content tools, the criteria in our AI tools for ecommerce product listings benchmark remain useful: accuracy, structured output, brand consistency, channel adaptation and measurable workflow performance matter more than a flashy one-off result.

    Separate recommendations from execution

    An alternative model can replace a copywriting assistant relatively easily. Replacing an autonomous agent that can change prices, send campaigns or issue refunds is much harder.

    Our analysis of why humans missed one in three dangerous AI agent commands explains why manual approval alone is insufficient. Permissions, spending limits, audit trails and rollback must be enforced outside the model.

    Design graceful degradation

    If the preferred model is unavailable, decide what the store should do:

    • switch to a verified secondary model;
    • queue non-urgent work;
    • fall back to deterministic templates;
    • preserve human support for sensitive cases;
    • disable autonomous writes while keeping read-only analysis available.

    “Try again until it works” is not a resilience strategy for orders or customer data.

    What it means for AI builders

    For developers and founders, concentration creates risk and opportunity at the same time.

    The risk: your product becomes a thin wrapper

    If your product is only a prompt plus one API call, the provider can reproduce it, a competitor can copy it, and pricing changes can erase its margin.

    The strongest moat usually sits elsewhere:

    • proprietary workflow data;
    • domain-specific evaluation;
    • integrations that are difficult to maintain;
    • governance and approval controls;
    • customer-specific configuration;
    • reliable structured outputs;
    • auditability;
    • user experience and distribution;
    • measurable business outcomes.

    The opportunity: become the independence layer

    Concentration increases demand for products that help businesses use leading models without becoming trapped by them.

    Potential opportunities include:

    • model routing based on quality, cost and latency;
    • portable prompt and policy management;
    • cross-model evaluation suites;
    • provider-neutral agent tooling;
    • caching and cost controls;
    • observability across model vendors;
    • data-loss prevention and access governance;
    • fallbacks for regulated or regional workloads;
    • migration testing when models are retired.

    HelpingBrains’ AI Governance Platform is aimed at this control layer: AI inventory, prompt governance, access monitoring, risk management and audit-ready reporting should remain consistent even when the underlying model changes.

    The opportunity hidden inside a two-horse race

    Market concentration is not automatically bad for customers.

    Two strong providers can:

    • compete aggressively on model quality;
    • reduce prices through efficiency gains;
    • standardise tool-use patterns;
    • accelerate enterprise features;
    • make advanced capabilities accessible without infrastructure investment;
    • create a large ecosystem for specialised products.

    Competition between OpenAI and Anthropic may also prevent either from exercising complete control. Google, Meta, xAI, specialist providers and open-weight models add further pressure even if they are smaller in a particular revenue dataset.

    The opportunity for businesses is to use the leading platforms while retaining the ability to move.

    This is similar to a sound cloud strategy: “multi-cloud” should not mean duplicating everything across three providers at enormous cost. It should mean identifying critical dependencies, using portable interfaces where practical and maintaining tested alternatives for the failures that matter.

    A practical AI diversification checklist

    You do not need to abandon OpenAI or Anthropic. You need to know what would break if one disappeared from your stack tomorrow.

    1. Map every dependency

    • ☐ List every model, API, assistant, agent and AI-enabled SaaS product in use.
    • ☐ Record which business workflow each one supports.
    • ☐ Identify the provider behind tools that resell or abstract another model.
    • ☐ Mark workflows that affect customers, revenue, production data or legal obligations.
    • ☐ Assign one accountable owner to every critical AI system.

    2. Separate your assets from the provider

    • ☐ Store prompts, policies and templates in a controlled repository.
    • ☐ Keep product facts, brand rules and customer permissions in your own systems.
    • ☐ Export conversation or workflow data where contractually and technically possible.
    • ☐ Avoid provider-specific data formats unless the benefit clearly exceeds the switching cost.
    • ☐ Document how model output is transformed before it reaches customers or production.

    3. Build a model-independent boundary

    • ☐ Use a stable internal request and response schema.
    • ☐ Isolate provider-specific code behind adapters.
    • ☐ Validate structured output rather than trusting free text.
    • ☐ Enforce permissions, budgets, privacy rules and prohibited actions outside the model.
    • ☐ Log model, version, prompt, tool calls, latency, cost and outcome.

    4. Test at least one alternative

    • ☐ Maintain a representative evaluation set using real—but sanitised—business cases.
    • ☐ Compare quality, cost, latency, refusals and structured-output reliability.
    • ☐ Test a secondary hosted model or a suitable open-weight alternative.
    • ☐ Measure migration effort rather than assuming APIs are interchangeable.
    • ☐ Repeat tests after major model releases.

    5. Plan the failure mode

    • ☐ Define when to switch providers automatically and when to require human review.
    • ☐ Queue non-critical work instead of producing lower-quality customer-facing output.
    • ☐ Keep deterministic templates for essential communications.
    • ☐ Prevent retries from duplicating sends, refunds, catalogue edits or orders.
    • ☐ Run a provider-outage exercise and record the recovery time.

    This is the save-worthy part of the story: diversification is not buying two subscriptions. It is making your data, controls and workflows portable enough that a second provider can actually take over.

    Should you use both OpenAI and Anthropic?

    Not automatically.

    A small company may create more complexity than resilience by integrating multiple providers too early. Every additional model introduces another contract, privacy review, evaluation surface and operational path.

    Use a second provider when at least one of these is true:

    • the workflow is important enough that an outage creates material loss;
    • model pricing represents a significant share of your unit cost;
    • customers require regional or provider choice;
    • one provider frequently refuses or performs poorly on essential tasks;
    • a model retirement would require a rushed migration;
    • your product promises provider-independent results;
    • regulation, procurement or data residency makes one provider insufficient.

    For low-risk experimentation, one provider plus good abstraction may be enough. For revenue-critical execution, a tested fallback becomes much more valuable.

    The real lesson: concentration belongs on your risk register

    The viral 70% claim is too broad. OpenAI and Anthropic do not demonstrably receive 70% of all revenue across the global AI economy.

    What the available estimates suggest is still significant: a very large share of the AI-related revenue credited to Amazon, Microsoft and Google may depend directly or indirectly on two foundation-model companies.

    That concentration does not mean businesses should stop building. It means they should stop confusing easy access with independence.

    Use the best model available for the job. But keep control of:

    • your data;
    • your prompts and policies;
    • your customer relationships;
    • your business rules;
    • your evaluation criteria;
    • your permission boundaries;
    • your fallback plan.

    The winners will not necessarily be the companies that predict which AI laboratory wins the race. They will be the ones that create value above the model layer—and can keep operating regardless of who is leading next year.

    Frequently asked questions

    Do OpenAI and Anthropic earn 70% of all AI revenue?

    There is no public, audited dataset proving that they receive 70% of revenue across the entire global AI industry. The viral figure is based on analyst estimates of AI-related revenue at Amazon, Microsoft and Google, including compute purchased by OpenAI and Anthropic and cloud platforms reselling access to their models.

    Why is AI market concentration a risk for businesses?

    Heavy reliance on one or two providers exposes businesses to price changes, outages, model retirements, policy changes, regulatory restrictions and strategic competition. The risk is highest when core data, prompts and workflows cannot move to another provider.

    Is a multi-model strategy always better?

    No. Multiple providers add cost and complexity. The right approach is proportional: abstract critical integrations, maintain evaluation tests and create a verified fallback for workflows where downtime or forced migration would cause material harm.

    How can ecommerce companies avoid AI vendor lock-in?

    Keep catalogue data and business rules in company-controlled systems, use stable internal schemas, separate provider-specific code, enforce permissions outside the model and test the same workflow against at least one alternative model.

    What creates a defensible AI product if the models are commoditised?

    Defensibility usually comes from proprietary data, domain workflow, evaluation, integrations, governance, user experience, distribution and measurable outcomes—not exclusive access to a general-purpose model.

  • Humans Missed 1 in 3 Dangerous AI Agent Commands. Here’s the Fix

    Humans Missed 1 in 3 Dangerous AI Agent Commands. Here’s the Fix

    An AI agent asks for permission to run a command. The request looks routine. You have already approved ten similar actions, the workflow is waiting, and the button says Approve.

    Would you notice if that harmless-looking command could expose credentials, contact an unknown server or execute code that the agent quietly modified one step earlier?

    In a browser experiment covering more than 40,000 game sessions and 409,000 approve-or-deny decisions, the average player missed roughly one in three simulated threats. Mean accuracy was 66.3%. Seven percent of sessions approved every prompt.

    That is an alarming result—but it needs careful framing.

    This was not a peer-reviewed enterprise security study or a test inside real ecommerce companies. It was a short, 60-second game created by developer Alex Wauters. Approximately 34% of the commands shown were threats, far more than a normal employee should encounter. Players were also placed under artificial time pressure and knew that dangerous commands were present.

    So the experiment does not prove that every business reviewer will miss exactly one-third of malicious AI actions. It demonstrates something narrower and still important: a human approval button is a weak security control when the reviewer lacks context, sees too many prompts or is asked to judge risk from a familiar-looking command name.

    For companies connecting AI agents to storefronts, CRMs, advertising platforms, customer-service tools or payment workflows, that distinction matters.

    The problem is not simply that AI can make a dangerous request. The problem is an approval workflow that expects a busy human to recognize every dangerous request in time.

    What the 40,000-play experiment actually found

    The original analysis by Alex Wauters reported four headline findings:

    • Players missed about one in three threats on average, with 66.3% mean accuracy.
    • 32.9% of sessions produced a negative score because approved threats and incorrectly blocked safe commands outweighed correct decisions.
    • 35.2% caught every threat, but some achieved this by blocking almost everything.
    • 7% approved every request.

    The type of command made a large difference.

    Threat categoryExamples used in the gameMiss rate
    Obviously destructiveDeleting root files or opening all permissions11.7%
    Persistent mutationScheduled-task injection or Git configuration hijacking23.8%
    Exfiltration or code executionUnknown APIs or typosquatted packages33.4%
    Scope violationsReading cloud credentials or cluster configuration35.0%

    People were comparatively good at catching commands that *looked* destructive. They struggled more when the risk was indirect, depended on earlier context or was hidden behind an ordinary developer workflow.

    The most frequently missed example was npm run analyze, approved 64.7% of the time. The name sounds harmless, but an npm script can execute whatever has been defined in package.json. In the game, the earlier history showed suspicious behaviour, yet many players still approved it.

    That is the real lesson: humans often evaluate the label in front of them, not the full chain of changes behind it.

    Why “human in the loop” is not enough

    Human oversight remains valuable. A qualified person should own high-impact decisions, especially those affecting customers, money, production data or legal obligations.

    But adding an approval dialog does not automatically create effective oversight. It can fail in at least four ways.

    1. Approval fatigue turns review into clicking

    When an agent requests permission repeatedly, each new prompt feels less exceptional. The human shifts from evaluating risk to keeping the workflow moving.

    This is familiar from cookie banners, security warnings and access prompts: too many alerts train people to clear alerts, not investigate them.

    2. The reviewer sees the command, not its history

    A request such as “publish campaign,” “run analysis” or “sync catalogue” may conceal a chain of earlier edits, retrieved instructions, third-party tool calls and generated files.

    The final action may look ordinary even when its inputs have been poisoned.

    3. Business users cannot evaluate technical side effects

    A marketer can judge whether email copy fits the campaign. They should not be expected to determine whether a connector is sending customer data to an unapproved endpoint.

    Likewise, a customer-service manager can approve a refund policy. They may not know whether the agent’s tool call can access every customer record rather than only the current case.

    4. The decision is framed as approve or block

    Binary approval forces a false choice. The reviewer may want the agent to continue, but only with a smaller audience, lower budget, redacted dataset or reversible draft.

    A safe workflow should support modify, constrain, simulate and escalate, not only approve or deny.

    What this means for marketing, ecommerce and customer workflows

    The commands in Wauters’ experiment were designed around coding agents, but the underlying problem applies to any agent that can take consequential action.

    Marketing automation

    An agent may draft content safely but create risk when it can also:

    • publish directly to social accounts;
    • email an unrestricted customer segment;
    • increase campaign budgets;
    • upload customer lists to external services;
    • override consent or frequency rules;
    • make unsupported product, health or sustainability claims.

    “Approve campaign” does not tell the reviewer whether the agent changed the audience, budget, tracking configuration or destination URL.

    Ecommerce operations

    An ecommerce agent may be able to edit prices, promotions, stock, orders and product information at machine speed. One broad approval could produce thousands of customer-facing changes.

    A pricing request that looks reasonable could violate a minimum-margin rule. A product-description update could insert an unsupported claim across hundreds of SKUs. A refund agent might perform a legitimate action on the wrong account because its identity boundary is too broad.

    This is why my earlier analysis of what happened when GPT-5.6 ran a real business recommends controlled leverage: let an agent investigate broadly, recommend clearly, draft quickly and execute only inside narrow technical limits.

    Customer service

    Support agents often need access to personal data and account actions. The risks include:

    • exposing one customer’s information to another;
    • issuing excessive refunds or credits;
    • changing account details without strong identity checks;
    • sending invented policy explanations;
    • closing or modifying the wrong case;
    • being manipulated by malicious instructions contained in a customer message or attachment.

    In each case, human approval is helpful only if the reviewer sees the relevant customer, data, policy, proposed change and maximum impact in one place.

    Before giving an AI agent permission, ask these five questions

    The fix is not to remove humans from the process. It is to stop using humans as the only enforcement layer.

    Use these five questions before granting an AI agent any meaningful permission.

    1. What is the smallest permission this agent actually needs?

    Do not grant access based on everything the agent *might* do. Grant only what it needs for the current, defined job.

    For example:

    • Prefer “read performance for these 20 products” over “store administrator.”
    • Prefer “draft an email” over “send to all subscribers.”
    • Prefer “propose a refund” over “issue unlimited refunds.”
    • Prefer “adjust bids within ±5%” over “manage the advertising account.”

    Use a dedicated service identity, separate read and write permissions, restrict accessible records and make elevated access temporary where possible.

    2. What is the maximum damage one action can cause?

    Assume the model is mistaken, manipulated or operating on bad data. Then calculate the blast radius.

    Ask:

    • How much money can it spend or refund?
    • How many customers can it contact?
    • How many products, prices or orders can it change?
    • Which personal or confidential data can it read?
    • Can it delete, overwrite or publish anything?

    Enforce financial, volume, recipient and frequency limits outside the prompt. “Be careful” is an instruction; a €100 transaction ceiling is a control.

    3. Can the action be previewed, reversed and tested safely?

    The safest default is to let the agent prepare the change without executing it.

    Use:

    • previews for emails, product updates and campaigns;
    • staging environments for code and configuration;
    • dry runs for imports, refunds and bulk changes;
    • small canary groups before a full rollout;
    • before-and-after snapshots;
    • idempotency protection to prevent duplicate actions;
    • automatic rollback when thresholds are breached.

    If an action cannot be reversed, its approval threshold should be much higher.

    4. What evidence will the reviewer see?

    Never show only the final button and a friendly action name.

    The approval screen should display:

    • the exact action and target;
    • all data that will leave the organisation;
    • the system, account and permission being used;
    • a summary of relevant changes made earlier in the workflow;
    • expected benefit and maximum downside;
    • policy checks passed or failed;
    • affected customer count, spend and scope;
    • rollback or recovery plan.

    For high-impact actions, require approval from a named role with the expertise to evaluate that risk. A finance owner should review spending; a privacy owner should review sensitive-data transfers; a merchandiser should review pricing and product claims.

    5. What happens if the reviewer misses the threat?

    This is the most important question because eventually someone will click the wrong button.

    Build controls that remain effective after mistaken approval:

    • sandbox untrusted execution;
    • restrict network destinations;
    • block access to secrets by default;
    • enforce policy at the tool or API gateway;
    • monitor unusual sequences and repeated failures;
    • log every agent identity, tool call, input and output;
    • alert on privilege escalation or scope changes;
    • maintain a kill switch independent of the agent;
    • test incident response and credential revocation.

    Research into coding-agent execution security describes isolation, capability controls, network egress restrictions and auditability as distinct layers—not substitutes for one another. The 2026 execution-security review reinforces why businesses need defence in depth rather than a single approval mechanism.

    A better AI approval workflow

    A responsible workflow separates what the model proposes from what the system permits.

    1. Classify the action. Determine whether it is read-only, reversible, customer-facing, financial, privacy-sensitive or destructive.
    2. Enforce machine-readable policy. Block prohibited tools, data, destinations and limits before asking a human.
    3. Generate a constrained preview. Show the exact change, target, scope, evidence and rollback plan.
    4. Route to the right owner. Ask a qualified, accountable person—not whichever employee happens to be watching the agent.
    5. Execute with narrow credentials. Use temporary, task-specific authority rather than a broad administrator token.
    6. Verify the outcome. Compare the result with the approved action and automatically stop on deviation.
    7. Preserve an audit trail. Record who approved what, based on which evidence, and what actually happened.

    HelpingBrains’ AI Governance Platform is being designed around the organisational side of this challenge, including AI inventory, prompt governance, access monitoring, risk management and audit-ready reporting.

    A practical pre-permission checklist

    Copy this into your agent deployment review:

    • ☐ The agent has one defined owner and one defined business purpose.
    • ☐ It uses a dedicated identity rather than an employee’s shared credentials.
    • ☐ Read, write, publish, send, spend and delete permissions are separated.
    • ☐ Access is limited to the records, tools and time window required.
    • ☐ Customer data and credentials are inaccessible unless strictly necessary.
    • ☐ Spend, refund, recipient, volume and frequency limits are technically enforced.
    • ☐ High-impact actions produce a preview with scope, evidence and downside.
    • ☐ The approver has the knowledge and authority to assess the action.
    • ☐ Untrusted code or tools run inside an isolated environment.
    • ☐ External network destinations are restricted and monitored.
    • ☐ Every important action is logged and attributable.
    • ☐ Repeated failures, unexpected scope changes and policy violations trigger a stop.
    • ☐ Rollback, credential revocation and the independent kill switch have been tested.

    If you cannot tick the relevant boxes, the agent is not ready for that permission.

    The problem is the approval workflow

    The 40,000-play experiment does not prove that humans are useless or that AI agents should never act. It shows why “a human clicked approve” is not an adequate security architecture.

    Humans are strongest when they make a small number of well-framed decisions with the right context. They are weakest when software floods them with repetitive prompts and expects them to reconstruct hidden technical history under time pressure.

    The goal should therefore be fewer approvals, better approvals and smaller consequences when an approval is wrong.

    Before connecting your next AI agent, do not ask only:

    “Will a human approve dangerous actions?”

    Ask:

    “What prevents one mistaken approval from becoming a business incident?”

    The problem is not the AI alone. It is the approval workflow—and that is something businesses can redesign now.

    Frequently asked questions

    Did a scientific study prove humans miss one in three AI threats?

    No. The figure comes from more than 40,000 sessions of a timed browser game and 409,000 approval decisions. It is useful evidence of approval fatigue and context problems, but it was not a controlled, peer-reviewed study of enterprise employees. The threat rate and time pressure were intentionally artificial.

    Is human-in-the-loop approval useless for AI agents?

    No. Human review is valuable for judgment, accountability and exceptional cases. It should sit on top of technical controls such as least privilege, spending limits, sandboxing, network restrictions, logging and rollback—not replace them.

    Which AI agent actions should always require approval?

    Require strong review for irreversible or high-impact actions involving customer communications, production changes, personal data, payments, refunds, pricing, account permissions, deletion and external publication. The exact threshold should depend on scope, reversibility and maximum possible harm.

    How can ecommerce companies reduce AI approval fatigue?

    Automatically allow low-risk actions inside strict policies, automatically block prohibited actions, and escalate only meaningful exceptions. Give reviewers a clear preview of affected products, customers, budget, data and rollback options instead of a raw technical command.

    What is the safest way to deploy an AI agent?

    Start with read-only observation, then recommendations, then drafts. Allow narrow execution only after the workflow has been tested in a sandbox and limited rollout. Use task-specific identities, hard limits, monitoring, audit logs and an independent kill switch.

  • A CVE Was Issued for a SQLite Bug That Never Existed. The AI Wasn’t the Only Failure

    A CVE Was Issued for a SQLite Bug That Never Existed. The AI Wasn’t the Only Failure

    A critical vulnerability was reported in SQLite. It received a CVE identifier, appeared in major security feeds and accumulated severity metadata.

    There was one problem: the vulnerable code did not exist in the affected version.

    That sentence sounds like the setup for another funny AI hallucination story. It is not funny if you operate an ecommerce platform, ship SaaS software or manage a security backlog.

    A CVE identifier is machine-readable authority. It can open tickets, block releases, trigger customer questions, produce compliance findings and send engineers into emergency remediation. When the underlying vulnerability is fictional, the hallucination does not remain inside a chatbot window. It enters the software supply chain.

    My opinion is simple: the model may have generated the fiction, but the trust pipeline converted it into operational reality.

    That is the part the industry needs to fix.

    First, what JFrog actually found

    On July 30, 2026, JFrog Security Research published an analysis of six newly published SQLite vulnerability records:

    • CVE-2026-51296
    • CVE-2026-51297
    • CVE-2026-51300
    • CVE-2026-51302
    • CVE-2026-51303
    • CVE-2026-51304

    The records described use-after-free vulnerabilities with severity scores ranging from High to Critical. JFrog checked the claims against the relevant SQLite source versions, compiled clean builds in isolated Docker containers and executed the supplied proof-of-concept inputs with AddressSanitizer enabled.

    None of the six vulnerabilities reproduced.

    The reports failed in ways that should have been immediately disqualifying:

    • One cited exprComputeOperands(), a function that did not exist in the affected SQLite version.
    • One claimed a fix in SQLite 3.51.3 even though the relevant source file had not changed between the supposedly vulnerable and fixed releases.
    • One cited source lines that contained unrelated code.
    • One referred to jsonBlobEdit(), which was introduced after the allegedly affected version.
    • One referenced line numbers beyond the end of the file.
    • One used an incorrect function signature and described a dangling pointer where the implementation explicitly cleared the pointer.

    Some PoCs failed during parsing. Others ran normally without a crash, memory error or leak.

    SQLite creator Richard Hipp described the reports as fictitious CVEs on the SQLite forum. The records were subsequently rejected after further investigation determined that they were not security issues.

    The viral headline needs one correction

    “An AI got a CVE for a bug that does not exist” is a strong headline. It is not the full fact pattern.

    JFrog found that the advisories appeared machine-generated when analysed with GPTZero, shared repeated structural patterns and contained the kinds of confident technical inventions associated with LLM output. JFrog described them as likely “LLM slop.”

    However, an AI detector cannot prove which tool created a document, and the reporter’s exact workflow has not been independently established.

    What is verified is this:

    ClaimFact-check result
    Six reported SQLite vulnerabilities did not reproduceVerified by JFrog’s source inspection and isolated testing
    The records cited nonexistent or unrelated codeVerified in JFrog’s technical analysis
    The advisories were probably AI-generatedStrongly suspected, but not conclusively proven from the public evidence
    The reports entered CVE and downstream vulnerability systemsVerified
    Some records received High or Critical enrichmentVerified
    The SQLite records were later rejectedVerified
    A fully autonomous AI personally submitted and obtained themNot publicly proven

    That distinction matters. I do not need to exaggerate the story to find it alarming.

    The serious, defensible conclusion is that plausible-looking but technically false vulnerability reports passed into authoritative systems without mandatory reproduction.

    This was not one bad CVE

    The six SQLite records were part of a larger collection associated with a newly created GitHub repository. JFrog said it reviewed 55 advisories from the same account and judged 54 to be fabricated; the remaining advisory described a real bug wrapped in unverified CVE metadata.

    The additional reports concerned other open-source projects, including LibRaw and ESP32-audioI2S. JFrog’s detailed published reproduction work focused on the six SQLite cases, so those are the strongest basis for conclusions.

    This scale changes the story.

    One incorrect submission can be a mistake. Dozens of polished, mutually similar advisories demonstrate how inexpensive it has become to generate vulnerability-shaped text—and how expensive it remains to disprove it.

    An LLM can invent 50 credible-looking reports quickly. A maintainer or security researcher must still:

    1. Locate the exact source version.
    2. Check whether the named functions and lines exist.
    3. Understand the memory-management path.
    4. Build the affected release.
    5. Instrument it correctly.
    6. Execute the PoC.
    7. Interpret the result.
    8. Contact databases and downstream vendors.

    The attacker—or careless submitter—pays the cost of generation. The ecosystem pays the cost of verification.

    That asymmetry is the real vulnerability.

    The three checkpoints that failed

    Everyone will dunk on the hallucinating model. I think that lets the rest of the system off too easily.

    Three checkpoints should have stopped these claims before they acquired operational authority.

    Checkpoint 1: Technical evidence was not required before publication

    The first gate should answer a basic question: Does the claimed behaviour exist in the specified version?

    For these SQLite records, simple evidence checks would have exposed serious problems:

    • Does the function exist in that release?
    • Do the referenced line numbers exist?
    • Does the PoC parse?
    • Does it reach the claimed code path?
    • Does a sanitizer detect the alleged memory error?
    • Is there a vendor acknowledgement, issue, commit or patch?

    According to JFrog, today’s public CVE submission path does not universally require a reproducible PoC or independent reproduction before a record is published. CVE Numbering Authorities often depend on submitters acting in good faith, especially when the CNA does not maintain the affected product.

    That honour-based process was built for a world where producing a detailed vulnerability report required meaningful expertise and effort. Generative AI changed the economics without changing the gate.

    Checkpoint 2: Enrichment looked like validation

    A CVE identifier names a reported issue. A CVSS score estimates severity. A CPE mapping describes affected products. None of these automatically proves the vulnerability exists.

    But once a record receives a Critical score, weakness classification and affected-version metadata, it looks increasingly verified to humans and machines.

    That creates an authority cascade:

    CVE ID → severity score → vendor feed → scanner alert → urgent ticket → emergency change

    Every extra field makes the record appear more mature, even if the underlying technical claim remains untested.

    The National Vulnerability Database historically provided more manual analysis and enrichment, but NIST has struggled with a large processing backlog since 2024. CISA and other data providers have helped enrich records, but the pipeline remains fragmented. As The Register’s follow-up reporting noted, no universal checkpoint requires every claimed vulnerability to be independently reproduced.

    Metadata is useful. Metadata is not evidence.

    Checkpoint 3: Downstream automation can act before a human verifies relevance

    The third failure is inside our own companies.

    Many organisations ingest CVE feeds directly into scanners, risk dashboards, Jira queues, compliance reports and dependency bots. A high score can automatically:

    • Create a Priority 1 security ticket
    • Fail a CI/CD security gate
    • Block a production deployment
    • Trigger a dependency upgrade
    • Escalate to management or a customer
    • Mark a control as noncompliant
    • Ask an AI coding agent to generate a patch

    Automation is not inherently wrong. The problem is treating an external identifier as a verified instruction.

    If an AI remediation agent receives one of these fictional records, it may search for a function that does not exist, modify unrelated code or “fix” a safe component. A false vulnerability can therefore cause a real vulnerability through unnecessary changes.

    I would describe that as hallucination laundering: an uncertain machine-generated claim passes through respected databases and emerges looking authoritative enough for another machine to act on it.

    Why this matters to ecommerce and SaaS teams

    SQLite is extremely widely embedded, but the broader lesson applies to every package in a modern stack.

    An ecommerce or SaaS application may contain hundreds or thousands of direct and transitive dependencies across:

    • Storefront frameworks
    • Mobile applications
    • Payment and checkout services
    • Search and recommendation tooling
    • Analytics SDKs
    • Customer-support integrations
    • CI runners and developer tools
    • Containers and operating-system packages
    • Marketplace plugins

    Your team does not need to use SQLite as its primary production database for a scanner to discover it somewhere in the estate.

    A fictional Critical record can still produce real downstream cost.

    False emergency work

    Developers stop roadmap work to investigate a supposed critical flaw. Security, platform and product teams join calls. Leadership asks for an exposure statement. Even if the answer is eventually “not vulnerable,” those hours are gone.

    Risky emergency upgrades

    Under pressure, a team may update a library, base image or framework without its normal regression window. That can break checkout, authentication, inventory sync or order processing—the systems where availability and correctness directly affect revenue.

    Release delays

    A security gate that treats every Critical CVE as automatically exploitable may block a launch even when the record is false, the package is unreachable or the vulnerable function is absent.

    Compliance noise

    PCI DSS, SOC 2, ISO 27001 and customer security reviews all create pressure to demonstrate timely vulnerability management. A false record can still appear in exports, evidence packs and questionnaires until it is rejected or manually suppressed.

    Alert fatigue

    When engineers repeatedly chase irrelevant or false alerts, they become slower to trust the next one. The worst outcome is not one wasted afternoon. It is a security culture trained to dismiss machine-generated warnings.

    A sane AI-in-the-loop security workflow

    AI should absolutely be used in security. It can summarise advisories, map dependencies, inspect code, generate test cases and help analysts prioritise evidence.

    But the system needs three explicit gates.

    Gate 1: Validate the advisory

    Before a record can drive remediation, verify its identity and technical coherence.

    Require:

    • Current CVE status: published, disputed or rejected
    • Vendor or maintainer acknowledgement
    • Correct affected product and version range
    • Real functions, files and line references
    • Linked issue, commit, patch or advisory where available
    • A PoC with enough detail to reproduce
    • Consistent weakness and severity metadata

    If basic evidence is missing, label the record unverified. Do not silently convert missing evidence into confidence.

    Gate 2: Validate your exposure

    A real CVE does not automatically create real risk for your application.

    Check:

    • Is the affected package actually deployed, or only present in development tooling?
    • Is the affected version running?
    • Is the vulnerable function compiled and reachable?
    • Can untrusted input reach it?
    • Do configuration, sandboxing or network controls reduce exposure?
    • Is exploit activity known?
    • What business service would be affected?

    Combine the advisory with SBOM, runtime, reachability and asset-criticality data. CVSS should be one input, not the final decision.

    Gate 3: Approve the action

    AI may recommend a patch, upgrade, mitigation or suppression. A named human should approve actions that can affect production, customer data or availability.

    Every recommendation should include:

    • Evidence that the vulnerability exists
    • Evidence that your environment is exposed
    • Proposed change
    • Regression and compatibility risks
    • Test plan
    • Rollback plan
    • Decision owner
    • Recheck date if the record remains disputed

    This follows the same principle as my SAFE framework for AI agents: scope the authority, require approvals, impose hard limits and retain evidence. I explored that model in what happened when GPT-5.6 was allowed to run a real business.

    A practical checklist for a new Critical CVE

    Before opening an emergency change, ask:

    • ☐ Is the CVE still active and not rejected?
    • ☐ Has the software maintainer acknowledged it?
    • ☐ Is there a real patch, commit or issue?
    • ☐ Do the named code symbols exist in the affected version?
    • ☐ Has the PoC been reproduced by a credible party?
    • ☐ Is our exact package and version deployed?
    • ☐ Is the vulnerable path reachable in our configuration?
    • ☐ Can untrusted input reach it?
    • ☐ Is there evidence of exploitation?
    • ☐ What is the business impact if we patch now?
    • ☐ What is the business impact if we wait for verification?
    • ☐ Is the proposed change tested and reversible?
    • ☐ Who owns the final decision?

    Security automation should collect these answers, not skip them.

    What I would change in an engineering organisation tomorrow

    If I were reviewing an ecommerce or SaaS security workflow after this incident, I would make five immediate changes.

    1. Add an “evidence state” separate from severity

    Use states such as:

    • Unverified
    • Vendor-confirmed
    • Independently reproduced
    • Disputed
    • Rejected

    A record can be Critical and unverified at the same time. Your tooling should be able to express that.

    2. Quarantine new high-severity records briefly

    Do not automatically suppress them, but do not trigger irreversible remediation solely from the first feed event. Run fast coherence, vendor and reachability checks first.

    3. Stop measuring teams only by CVE closure time

    If the KPI rewards closing every ticket quickly, teams will produce unnecessary upgrades and meaningless suppressions. Measure verified risk reduction, not identifier throughput.

    4. Keep AI in recommendation mode

    AI can prepare the investigation, proposed patch and test plan. It should not merge dependency changes or deploy security fixes without defined approval and rollback controls.

    This is where an AI governance platform must connect policy to runtime action: who may ask an agent to change code, which evidence is required and which decisions remain human-owned.

    5. Record why the decision was made

    Store the advisory version, sources, exposure analysis, decision, reviewer and timestamp. If a CVE is later rejected—or a disputed report becomes real—you can reconstruct the reasoning.

    My take: the CVE system did not suddenly become useless

    The wrong lesson is “never trust CVEs.”

    CVE identifiers remain essential for coordinating vulnerability information across vendors, scanners and organisations. The system processes an enormous volume of reports, and early publication can help defenders respond quickly to real threats.

    The better lesson is: a CVE is a reference, not a verdict.

    The industry has gradually treated identifiers, scores and feed enrichment as interchangeable with verified exploitability. Generative AI exposed the weakness because it can produce security language that satisfies the expected shape without satisfying the technical reality.

    The response should not be to ban AI-generated research. AI-assisted fuzzing and code analysis can find genuine vulnerabilities. The response should be evidence-based gates that apply equally to humans and machines.

    If a human submits a technically impossible advisory, reject it.

    If an AI submits a reproducible vulnerability with correct versions, a working PoC and a verified fix, evaluate it on the evidence.

    Trust the artefacts, not the fluency.

    The real security bug was authority without verification

    These fabricated SQLite records were eventually investigated and rejected. That is the system correcting itself.

    But correction happened after the claims had acquired official identifiers, severity data and downstream visibility. In an increasingly agentic development environment, that delay matters.

    The next false advisory may not merely waste an analyst’s time. It may instruct another AI to modify production software automatically.

    So yes, an apparent AI hallucination made it surprisingly far into the vulnerability ecosystem. But I do not think the most useful response is laughing at the model.

    The model produced plausible nonsense. Humans built a pipeline where plausible nonsense could become machine-readable authority.

    That is the vulnerability we need to patch.

    FAQ

    Did an AI definitely create the fake SQLite CVEs?

    JFrog found strong indicators of AI generation, including repeated patterns, implausible technical details and AI-detector results. However, the exact authoring tool and level of human involvement have not been conclusively established publicly.

    Were the SQLite vulnerabilities real?

    JFrog reproduced the relevant SQLite versions in isolated environments and found that none of the six reported vulnerabilities worked. The reports referenced nonexistent functions, impossible line numbers, unrelated code or invalid PoCs. The records were later rejected.

    Does receiving a CVE number prove a vulnerability exists?

    No. A CVE is a standard identifier for coordinating information about a reported vulnerability. Records can later be disputed, updated or rejected. Teams should verify vendor acknowledgement, technical evidence and their own exposure.

    Should companies stop automatically patching Critical CVEs?

    Companies should respond quickly, but severity alone should not determine the action. Confirm that the record is valid, the affected version is deployed, the vulnerable path is reachable and the remediation is safer than the current exposure.

    Can AI be trusted for vulnerability analysis?

    AI can help research, triage and test vulnerabilities, but its output should be treated as a hypothesis until supported by source inspection, reproducible execution and human review. Production changes should remain controlled and reversible.

  • An AI “News” Site Targeted AI Critics. The Money Trail Leads Close to OpenAI

    An AI “News” Site Targeted AI Critics. The Money Trail Leads Close to OpenAI

    An email arrived from a journalist named Michael Chen.

    He wanted written answers about an artificial-intelligence bill in Tennessee. The proposed headline was already unusually loaded. There was no discoverable reporting history, no personal email address and apparently no real journalist behind the name.

    According to an investigation by Model Republic, “Michael Chen” appears to have been an AI agent operating for an anonymous publication called The Wire by Acutus.

    That is already a serious media-literacy story. But the investigation went further. It reported that Acutus published dozens of articles generated wholly or partly by AI, presented itself as independent journalism, solicited real people for comments through apparent bot identities, and published material that frequently aligned with political and commercial interests.

    The investigators also traced connections from the outlet through public-relations and political consulting firms to the orbit of Leading the Future, a pro-AI super PAC substantially funded by OpenAI president and cofounder Greg Brockman and his wife.

    This has produced a viral shorthand: “OpenAI is funding AI bots to attack its critics.”

    The actual evidence is more complicated—and getting it right matters.

    First, the necessary fact-check

    There is credible evidence for several parts of the story:

    • Acutus operated without a conventional masthead, named editors or normal author bylines.
    • Publicly accessible code reportedly exposed an editorial system with fields for AI background context, AI-generated questions and a “Generate Story Draft” function.
    • The system referred to an “AI interviewer” or “reporter agent.”
    • Model Republic reported that 69% of 94 examined articles were classified by the Pangram detector as fully AI-generated and another 28% as partly AI-generated.
    • The site’s own automated reviewer reportedly marked 42 stories as needing revision, yet they were still published.
    • Some stories criticised AI-safety advocates or supported positions associated with a less-regulated AI industry.
    • Investigators identified relationships connecting people who promoted or appeared in Acutus content to firms in the Leading the Future political network.

    However, the investigation did not publish proof of a direct payment from OpenAI—or even from Leading the Future—to Acutus.

    OpenAI says the company has made no donations to super PACs, candidates or campaigns. In its official statement on political advocacy, OpenAI said Greg and Anna Brockman supported Leading the Future in their personal capacity, that the company does not direct the PAC, and that it has no visibility into its operations.

    Federal Election Commission records confirm that Leading the Future has raised tens of millions of dollars, but they do not establish that OpenAI itself funded Acutus.

    The responsible conclusion is therefore:

    An investigation uncovered an AI-driven pseudo-news operation and reported indirect links to a political network financed by prominent AI-industry figures, including OpenAI’s president and cofounder. OpenAI denies corporate involvement, and direct funding of the outlet has not been demonstrated publicly.

    That wording is less explosive than the viral version. It is also more trustworthy—which is precisely what this story is about.

    What Acutus reportedly did

    The alleged operation was not simply a blog using ChatGPT to speed up drafting.

    The public-facing site described its work as independent, expert-sourced journalism. Behind that presentation, investigators said they found an automated content pipeline that could:

    1. Accept political or commercial background instructions.
    2. Generate interview questions.
    3. Contact real experts through an apparent reporter identity.
    4. Extract or insert quotations.
    5. Generate complete articles.
    6. Run automated editorial and fact-checking passes.
    7. Publish the result through a wire-style feed.

    The significant issue is not that AI helped write the articles. Newsrooms and marketing teams already use AI for research, transcription, editing and drafting.

    The issue is deception about authorship, accountability and motive.

    If a source believes they are speaking to a human journalist, but they are actually responding to a political content bot, informed consent has failed. If a publication claims independence while its operators or funders remain hidden, readers cannot evaluate conflicts of interest. If synthetic stories are packaged for syndication and machine ingestion, the content can travel far beyond the original low-traffic website.

    Why a small, obscure site can still matter

    Acutus reportedly had limited organic reach. That may make the operation look insignificant. But in the age of AI search, visibility is not the only measure of influence.

    An article can become part of the information ecosystem through:

    • Search-engine indexing
    • RSS and Creative Commons syndication
    • Social accounts and paid amplification
    • Citation by other publishers
    • Retrieval by AI assistants
    • Inclusion in future training or evaluation datasets
    • Repetition across newsletters, posts and automated summaries

    The danger is cumulative. One anonymous article rarely changes public opinion. A network of apparently independent pages can create the impression that a position is widely supported, repeatedly reported or already settled.

    That is astroturfing with an AI cost structure.

    Automation makes it possible to create many outlets, personas, interviews and stories without maintaining the expensive human organisation that traditional influence campaigns required. The same message can be adapted to different regions, audiences and political identities at machine speed.

    The second-order risk: AI may consume the influence campaign

    This is where the story moves beyond political drama.

    Synthetic content is increasingly written for both humans and machines. Model Republic reported that Acutus exposed a wire feed, welcomed AI crawlers and provided an llms.txt file describing its material as independent journalism.

    If an AI system later retrieves or summarises that content without understanding its provenance, an influence operation can become part of an apparently neutral answer.

    The loop looks like this:

    1. A stakeholder supplies a narrative.
    2. AI generates “reporting” around that narrative.
    3. Search engines and aggregators index the reporting.
    4. Other creators cite or paraphrase it.
    5. AI assistants retrieve the repeated claims as supporting evidence.
    6. The narrative returns to users with its origin obscured.

    The result is not necessarily a single spectacular falsehood. It can be something harder to detect: selective framing, omission, reputational pressure and manufactured consensus.

    What this means for content creators

    Creators can no longer judge a source only by how professional its website looks.

    Before using an unfamiliar publication, check:

    • Does it name its editors, reporters and ownership?
    • Do the authors have verifiable histories outside the site?
    • Does the publication explain its corrections policy?
    • Are quotations linked to original recordings, documents or named sources?
    • Does it disclose sponsors, clients, political relationships and AI use?
    • Can important claims be confirmed through primary sources?
    • Does its reporting repeatedly benefit the same group while presenting itself as neutral?

    AI detectors can be one signal, but they should never be treated as definitive proof. Writing style is not provenance. Ownership records, source documents, disclosures and reproducible evidence are stronger.

    Creators should also preserve their own research trail. Save primary links, screenshots, dates and relevant quotations. If a source changes or disappears, you should still be able to show why you trusted—or challenged—it.

    What this means for people consuming news

    Media literacy in 2026 requires a new question.

    We used to ask: “Is this story true?”

    Now we also need to ask:

    • Who wanted this story to exist?
    • Who gathered the information?
    • Was the interviewer a real, accountable person?
    • Who paid for distribution?
    • Which facts were selected or excluded?
    • Is this original reporting or an AI-generated remix?
    • Can I find the underlying filing, document, recording or dataset?

    This does not mean distrusting everything created with AI. Human journalism can also be biased, sponsored or wrong. The relevant distinction is not human versus machine. It is accountable versus unaccountable.

    What marketers and ecommerce brands should learn

    The immediate temptation is to treat the scandal as someone else’s political problem. That would be a mistake.

    The same trust collapse is coming to commercial content.

    Thousands of brands can now publish polished product guides, comparison articles, customer stories and “independent” reviews at negligible cost. As synthetic content expands, production volume becomes less valuable. Evidence and identity become more valuable.

    1. Generic expertise will lose its differentiation

    An AI model can produce another “10 Best Skincare Ingredients” article in seconds. A brand earns trust by contributing something the model cannot fabricate responsibly:

    • Original product testing
    • Named expert review
    • Transparent methodology
    • Real measurements and datasets
    • Before-and-after evidence with clear conditions
    • Customer research with consent
    • Known authors with relevant experience

    If you use AI to improve product content, it should strengthen clarity and consistency—not manufacture authority. Our practical benchmark of the best AI tools for ecommerce product listings explains where these tools help and where human review remains essential.

    2. Undisclosed synthetic testimonials are a brand risk

    Fake experts, invented customers and AI-generated interview subjects may create a short-term conversion lift. They also create legal, platform and reputational exposure.

    Every testimonial, endorsement and case study should have:

    • A real source
    • Documented consent
    • Accurate context
    • Clear disclosure of incentives
    • A retrievable evidence record

    The more realistic synthetic media becomes, the more valuable verifiable human proof will be.

    3. SEO needs provenance, not just optimisation

    Traditional SEO asks whether a page matches the query, covers the topic and earns links. Trust-focused SEO adds further questions:

    • Who wrote or reviewed the page?
    • What first-party evidence does it contain?
    • When was it checked and updated?
    • Which claims link to primary sources?
    • Is sponsored or AI-generated material disclosed?
    • Does the organisation have a visible identity and contact path?

    Search visibility will remain important. But as low-cost content floods the web, recognizable authorship, primary evidence and transparent editorial standards become differentiators for users—even when ranking systems are imperfect.

    4. Brands need an AI-content policy before a crisis

    Define which uses are acceptable before an employee or agency quietly automates the entire pipeline.

    At minimum, the policy should cover:

    • Approved AI tools and data-handling rules
    • Content types that require named human review
    • Disclosure requirements
    • Citation and fact-checking standards
    • Prohibited synthetic identities and testimonials
    • Approval for political, medical, financial or legal claims
    • Records of prompts, sources, edits and final approvers
    • Correction and takedown procedures

    This is one reason prompt control and auditability belong inside an AI governance programme, not in an informal document that nobody checks.

    5. Do not turn recommendation into autonomous publication

    AI can research, outline, draft and identify inconsistencies. Publishing is a separate authority.

    The lesson matches what we saw when GPT-5.6 was allowed to run a real business: capability is not judgment, and access is not accountability.

    For high-impact content, keep a named human responsible for:

    • Source selection
    • Factual claims
    • Defamation and fairness review
    • Disclosure
    • Brand alignment
    • Final publication

    An automated editorial score is not a substitute for an editor—especially when the system’s own warning can be ignored in seconds.

    A practical trust checklist for brands

    Before publishing AI-assisted content, ask:

    • ☐ Is a real person accountable for this page?
    • ☐ Are all quoted people and organisations real and correctly represented?
    • ☐ Can every material claim be traced to a reliable source?
    • ☐ Have we linked to the primary evidence where possible?
    • ☐ Are sponsorships, affiliations and conflicts disclosed?
    • ☐ Is AI use disclosed where readers would reasonably expect to know?
    • ☐ Have we separated reporting, opinion and promotion?
    • ☐ Does the content include first-party value rather than a generic synthesis?
    • ☐ Is personal or confidential data removed?
    • ☐ Can we reconstruct which model, prompt, sources and reviewer produced the final version?
    • ☐ Is there a correction and takedown owner?

    If the answer to the final question is “the AI,” the workflow is not ready.

    The uncomfortable question for the AI industry

    OpenAI’s public response says political organisations should identify whom they represent and avoid astroturfing. That is the correct standard.

    It is also reasonable to ask whether senior leaders at powerful AI companies should expect greater scrutiny when their personal political spending supports organisations operating close to opaque influence networks—even if the company itself does not direct those organisations.

    Both ideas can be true:

    1. The headline “OpenAI funded attack bots” goes beyond what has been publicly proven.
    2. The reported conduct and financial proximity are serious enough to demand disclosure, investigation and clear answers.

    Fairness does not require passivity. It requires making the strongest claim the evidence supports—and no stronger.

    The real competitive advantage is becoming believable

    AI has made content production abundant. It has not made trust abundant.

    For creators and brands, the winning strategy is not to appear less automated than everyone else. It is to be more verifiable:

    • Show who created the work.
    • Show what evidence supports it.
    • Show what AI did and what a human checked.
    • Show who paid for it.
    • Correct mistakes visibly.
    • Refuse to create fake people, fake consensus or fake independence.

    The Acutus story is not merely about one obscure site or one political network. It is a preview of an internet where polished content can be generated, interviewed, reviewed, distributed and amplified without an accountable human ever stepping forward.

    In that environment, provenance is not administrative metadata.

    It is the product.

    FAQ

    Did OpenAI directly fund the AI-generated news site?

    No direct payment from OpenAI to Acutus has been publicly demonstrated. Investigators reported indirect links between Acutus, political and public-relations firms, and Leading the Future, a super PAC backed personally by OpenAI president Greg Brockman and his wife. OpenAI says it has not donated to the PAC and does not direct its activities.

    Was the Acutus news site entirely AI-generated?

    Model Republic reported that an AI-detection analysis classified 69% of 94 articles as fully AI-generated and 28% as partly AI-generated. More compellingly, public site code reportedly exposed AI drafting, interviewing and review functions. AI-detector results alone should not be treated as conclusive proof.

    Is using AI to write news unethical?

    Not automatically. AI can assist with transcription, research, translation and drafting. The ethical problems arise when publishers conceal authorship, fabricate reporter identities, misrepresent independence, fail to verify claims or hide financial and political conflicts.

    How can brands make AI-assisted content more trustworthy?

    Use named authors and reviewers, link to primary sources, publish testing methods, disclose relevant AI use and sponsorships, verify every quotation, retain an audit trail and provide a visible corrections process.

    Will AI-generated content hurt SEO?

    AI use by itself does not determine whether content is valuable. Generic, unverified and mass-produced pages are unlikely to build durable reader trust. Original evidence, clear authorship, useful analysis and reliable sourcing create stronger differentiation than publishing volume alone.