When AI follows the wrong voice: Prompt injection in small businesses

Imagine asking an AI assistant to compare three supplier quotations. Prices, services, exclusions: exactly the kind of preparation an assistant can help with. One file, however, contains more than the offer. It includes text telling the AI to leave out certain drawbacks and recommend that particular supplier.

For a person, the boundary is obvious. A supplier can explain its proposal, but cannot set our evaluation rules. For a language model, the distinction is harder. It processes language both as instructions and as material to analyse. An attack tries to blur those roles.

The example is fictional; the mechanism is real. It is called prompt injection. It matters wherever AI reads third-party content and turns it into recommendations or actions. A small business does not need its own AI research team to respond. It does need to decide whose instructions count, and what the assistant is allowed to do afterwards.

A prompt is an input to an AI model. Prompt injection happens when an input steers the system in an unintended direction. A direct attempt comes through an input to the assistant. An indirect attempt arrives through content the assistant processes for a legitimate task, such as a webpage, an email or a file.

OWASP describes both types and lists possible consequences including manipulated answers, disclosure of sensitive information and unauthorised function calls. The actual risk depends on the application’s design and capabilities. A malicious sentence does not magically give a model administrator rights.

AI can make mistakes without an attack. With prompt injection, an outside source tries to redirect the task. Both can cause problems, but some of the controls they need are different.

OWASP: prompt injection and potential effects

An email may contain business information. That does not entitle it to tell the assistant to find another file and forward it. A webpage can describe a product without becoming an authorised source of new company policy.

Third-party content is material to work with. It is not approval for new tasks, additional access or external actions.

This still applies when the text sounds polite, resembles technical instructions or claims to come from IT. Its origin and the original task matter more than persuasive wording.

The UK NCSC cautions against treating prompt injection simply like traditional SQL injection. With language models, the same separation between instructions and data cannot be assumed. Our practical conclusion is to design protection for occasions when detection fails too.

NCSC: why prompt injection is not ordinary SQL injection

For an initial review, it helps to classify tasks by their possible impact. This table is our practical guide, not an official product classification or a claim about a particular system.

That is why “Which model do we use?” is not enough. Also ask which sources it reads, what data it can see, which tools it can call and what happens without human review.

An assistant that drafts a reply can already be valuable. Allowing it to send that reply automatically is a separate decision with a different risk profile. That extension should have a demonstrable benefit rather than being enabled merely for convenience.

Microsoft describes Prompt Shields as detection for manipulative prompts and document content. In a custom application using Azure AI Content Safety, the API returns detection results; the application decides what to do with them. Detection does not automatically mean effective blocking.

The documentation checked on 19 October 2026 distinguishes protection types and integration paths. Controls for document attacks, user prompt attacks and supported agent tools are not simply identical or active everywhere. Some intervention points are labelled Preview. This does not establish a blanket protection status for every Copilot or custom application.

Ask your provider specific questions. Are the documents and tool responses actually used in this workflow inspected? Is a detection blocked or only logged? What happens if a component fails? Which languages have been tested? Microsoft notes that detection performance can vary by language.

“We use Azure” does not answer those questions. Nor does an enabled filter prove that the model produces correct facts. Detecting manipulative instructions and checking factual accuracy are different tasks.

Microsoft: Prompt Shields, limitations and integration behaviour

A clear system instruction can help: external content should be treated as data rather than new commands. Separately marking the task and its sources is useful too. OWASP nevertheless recommends multiple protective layers, including checks on outputs and tool calls. One sentence in a prompt does not replace access controls.

Our recommendation is to start with consequences. A research assistant normally does not need to modify user accounts. A quotation comparison does not need access to the entire HR library. A draft customer reply need not be capable of sending itself to arbitrary recipients.

Those boundaries belong in the application and its connected services. If the model proposes an unauthorised action, the action should fail technically or require meaningful approval. Politely asking the model to follow a list is not sufficient.

Microsoft likewise recommends layered defences, minimum privileges and human review of risky actions. For us, the practical core is straightforward: do not assume every answer is safe; limit the possible consequences of an error.

OWASP: protective layers and technical checks · Microsoft: defending against indirect prompt injection

If a confirmation appears for every trivial step, people eventually click through by habit. That is not a useful security control. Approval belongs where a suggestion becomes a meaningful business action.

Before a message is sent, we want to see its recipients, subject, body and attachments. Before a record changes, the affected record and proposed value. Before information is shared, the intended audience. Without those details, the reviewer cannot make an informed decision.

The preview must match the action actually performed. After approval, an application should not substitute different attachments or recipients without asking again. This is an implementation requirement, not something every AI product automatically provides.

For higher-risk workflows, we recommend separate steps: analyse, produce a controlled draft, approve, then execute. In daily operations, traceability often matters more than an impressive demonstration.

Choose one planned application, such as summarising incoming supplier documents. Create synthetic documents and test accounts with no production data and no real sending or editing permissions. Define in advance what the assistant should do and what is outside its task.

A harmless test document could try to make the assistant output an unrelated advertising statement instead of the requested summary. Check whether the system stays on task. Also test the technical boundary: a proposed unauthorised tool call must not execute. Ask the responsible IT or development team to do this rather than experimenting with a production agent.

Record the input, expected behaviour, actual result and protective layer that worked. Repeat relevant tests after changes to the model, sources or tools. One successful test does not prove that every variation will be blocked.

Use the languages and document types people actually work with. A short English example is not sufficient acceptance testing for German PDF quotations and French customer emails.

The team does not need training on every attack pattern. A short rule is more useful: if AI suddenly wants to do something outside the task, requests additional data or proposes an unexpected external action, do not simply approve it. Have someone review the task, source and suggestion together.

Name an accessible contact for these situations. Employees should be able to report a suspicious interaction without first proving an attack. If an unexpected action has already happened, IT should promptly review the integration and what was actually executed. Logs may help, but can themselves contain confidential information.

Ordinary data protection remains important. In its current guidance on AI at work, Switzerland’s NCSC reminds users to avoid thoughtlessly transferring work data into AI tools. Prompt injection is an additional concern; it does not replace deciding which information may be processed in which approved service.

Swiss NCSC: using AI at work and protecting data (German)

An AI assistant should be able to read a quotation without handing the supplier control of our evaluation rules. It should summarise an email without making its sender the authoriser of further actions. That boundary sounds obvious. In an AI application, it must be deliberately designed.

Review one real workflow this week. Limit its data and tools, separate drafting from execution and run a harmless redirection test. This will not reduce risk to zero. It will help reveal where an error produces a correctable piece of text, and where it already changes something in the business.