Regulation & Policy
AI safety
Also known as: model safety
AI safety is the practice of designing, testing and operating AI systems so they cause less harm, behave predictably and resist misuse. It covers alignment with intended behaviour, evaluation and red teaming, content guardrails, and monitoring once a system is live. For marketers it shapes what models will say, how assistants handle brands, and what compliance teams expect before AI tools go into production.
What it is
AI safety is a field spanning technical research and applied governance. Technical work includes alignment methods, evaluation suites, red teaming, interpretability and guardrail systems that filter inputs and outputs. Applied work includes usage policies, human review, access controls, logging, incident handling and documentation of what a system is and is not approved to do.
Why it matters
Safety settings decide how AI assistants treat regulated claims, health and finance advice, competitor comparisons and unverified statements, which affects how your brand is described in generated answers. Internally, safety and governance requirements often determine whether an AI content or agent project reaches launch, since legal and risk teams need evidence of testing and controls. Publicly, visible harms from AI output carry brand and regulatory exposure.
How it works
Vendors publish usage policies and model documentation, run pre release evaluations and red teaming, and apply moderation and refusal behaviour at runtime. Teams adopting AI add their own layer: prompt and system instructions that set boundaries, retrieval limited to approved sources, human review for regulated or high risk output, audit logs, and escalation routes when something goes wrong. Many organisations map these controls to recognised frameworks and to obligations under emerging AI regulation such as the EU AI Act.
When it applies
It applies whenever AI generates or influences customer facing content, handles personal data, makes automated decisions, or acts through tools and agents on your systems. Scrutiny increases in regulated sectors such as health, finance, legal and children's products.
Examples
- A financial services team routes all AI drafted content through compliance review and blocks the model from stating rates or returns without a cited source.
- An ecommerce brand red teams its AI support agent with hostile prompts before launch to check it cannot issue unauthorised refunds or reveal other customers' order details.
- A marketing team documents which models, prompts and data sources are approved, so an internal audit can trace how any published AI assisted page was produced.
How it is measured
- Rate of policy violating or unsafe outputs found in sampled review, before and after guardrail changes
- Red team pass rate against a maintained set of adversarial prompts and jailbreak attempts
- Human review coverage and escalation volume for high risk content types
- Number and severity of AI related incidents, plus time to detect and resolve
Related terms in Regulation & Policy
- AI complianceAI compliance is the work of making sure AI systems meet the laws, regulations, standards and internal policies that apply to them. It spans data protection, transparency, risk classification, documentation, human oversight and record keeping across the life of a system. In practice it combines legal interpretation, engineering controls and ongoing evidence gathering.
- AI governanceAI governance is the set of policies, roles, controls and review processes an organisation uses to manage how AI systems are built, bought and used. It covers risk assessment, documentation, human oversight, data handling and accountability. It applies both to AI a company develops and to third-party AI tools used by staff.
- AI privacyAI privacy is the set of practices, rights and obligations that govern how personal data is collected, used, stored and exposed when building or using AI systems. It covers training data, prompts and outputs, retention by AI vendors, and the transparency and control offered to the people whose data is involved. For marketing teams it shapes what customer data can safely be put into AI tools and what must be disclosed.
- AI transparencyAI transparency is the practice of making clear how an AI system works, what sits behind it, and when content or an interaction involves AI. It covers regulatory disclosure duties as well as voluntary explanations such as model cards, labels on synthetic media and citations in AI generated answers. For publishers and marketers it sets expectations about when AI involvement should be declared and how clearly.
- Competition and Markets AuthorityThe Competition and Markets Authority (CMA) is the United Kingdom's competition and consumer protection regulator. It investigates mergers, anti-competitive conduct and market practices, and it holds specific powers over large digital firms under the Digital Markets, Competition and Consumers Act 2024. Its decisions shape how search engines, app stores and AI assistants operate in the UK market.
- Content licensingContent licensing is the practice of granting an AI company or platform permission to use your published material, usually for model training, retrieval or display inside an assistant, in return for payment or other terms. Deals set out what content is covered, how it can be used, how it is attributed and for how long. It is the commercial alternative to relying only on crawler blocking or copyright enforcement.
Primary research · August 2026
How ChatGPT Shortlists Software Brands
An audit across 10 categories and 60 buying questions. I recorded what ChatGPT reads, throws away and links to when a buyer asks it which software to buy, and what that decides.
- 60
- Questions asked
- 10
- Software markets
- 2,680
- Results read
- 367
- Links shown
Free download
Get the full report
35 pages · PDF · 536 KB. Enter your details and it downloads straight away.