That was my first reaction when I tested an AI tool with a simple research question: Does regular exercise improve memory in adults?
At first, I could have accepted the answer and moved on. But should I?
I asked ChatGPT and Perplexity the same question. Both gave me a clear answer, but they didn't present the evidence in exactly the same way. When I checked the research behind their answers, I found something worth noticing: the evidence was more complicated than a simple “yes.” The 2025 umbrella review they referred to included 133 systematic reviews, 2,724 randomized controlled trials, and more than 258,000 participants. It also reported important limitations in the quality of many of the underlying reviews.
Then I tried Gemini Notebook.
I gave it the WHO Guidelines on Physical Activity and Sedentary Behaviour and asked it to answer using only that source. Gemini told me that the source didn't actually answer my question.
I liked that answer.
Not because it gave me more information. It didn't. I liked it because it showed me where the evidence stopped.
That small test changed the way I think about AI. A polished answer can be useful, but polish isn't proof. If I'm going to rely on an AI output, I want to know where the information came from, whether the evidence supports the claim, and what the system does when the information isn't enough.
And that's not only a research problem.
A business employee might use AI to summarize a customer document, draft a report, research a topic, or prepare information for a decision. The result may look finished in seconds. But what if one important detail is missing? What if a number is wrong? What if the source doesn't actually support the conclusion?
That's where trust matters.
Digital trust means being able to rely on a technology system to be reliable, secure, transparent, and accountable. With AI, that trust needs more attention because a system can produce an answer that sounds certain even when the evidence behind it is weak or incomplete.
The regulatory environment is changing, too. The EU AI Act entered into force in 2024, and its main rules began applying on 2 August 2026, with some requirements taking effect later.
But compliance isn't the whole story. An organization can follow the rules and still lose people's trust if its AI exposes confidential information, produces an unfair recommendation, leaves out an important detail, or gives employees more confidence than the evidence deserves.
So the real challenge isn't simply getting AI into the workplace.
It's learning how to use it without making trust an afterthought.
This article looks at why AI creates new trust problems, what my own testing revealed, and how AI governance can help organizations manage those problems in practice.
That was my first reaction when I tested an AI tool with a simple research question: Does regular exercise improve memory in adults?
At first, I could have accepted the answer and moved on. But should I?
I asked ChatGPT and Perplexity the same question. Both gave me a clear answer, but they didn't present the evidence in exactly the same way. When I checked the research behind their answers, I found something worth noticing: the evidence was more complicated than a simple “yes.” The 2025 umbrella review they referred to included 133 systematic reviews, 2,724 randomized controlled trials, and more than 258,000 participants. It also reported important limitations in the quality of many of the underlying reviews.
Then I tried Gemini Notebook.
I gave it the WHO Guidelines on Physical Activity and Sedentary Behaviour and asked it to answer using only that source. Gemini told me that the source didn't actually answer my question.
I liked that answer.
Not because it gave me more information. It didn't. I liked it because it showed me where the evidence stopped.
That small test changed the way I think about AI. A polished answer can be useful, but polish isn't proof. If I'm going to rely on an AI output, I want to know where the information came from, whether the evidence supports the claim, and what the system does when the information isn't enough.
And that's not only a research problem.
A business employee might use AI to summarize a customer document, draft a report, research a topic, or prepare information for a decision. The result may look finished in seconds. But what if one important detail is missing? What if a number is wrong? What if the source doesn't actually support the conclusion?
That's where trust matters.
Digital trust means being able to rely on a technology system to be reliable, secure, transparent, and accountable. With AI, that trust needs more attention because a system can produce an answer that sounds certain even when the evidence behind it is weak or incomplete.
The regulatory environment is changing, too. The EU AI Act entered into force in 2024, and its main rules began applying on 2 August 2026, with some requirements taking effect later.
But compliance isn't the whole story. An organization can follow the rules and still lose people's trust if its AI exposes confidential information, produces an unfair recommendation, leaves out an important detail, or gives employees more confidence than the evidence deserves.
So the real challenge isn't simply getting AI into the workplace.
It's learning how to use it without making trust an afterthought.
This article looks at why AI creates new trust problems, what my own testing revealed, and how AI governance can help organizations manage those problems in practice.
1. What Is Digital Trust in the Age of AI?
Digital trust is the confidence people place in a technology system when they use it, share information with it, or rely on its results.
Before AI became part of everyday work, that trust was easier to define. I expect a website to protect my payment details. I expect a cloud service to keep my files available. And I expect a business platform to handle customer information securely. If the system follows its rules, I usually know what to expect.
AI makes that assumption less comfortable.
Ask an AI assistant to summarize a report, draft a customer email, or explain a piece of research, and you'll get an answer in seconds. It may be useful. It may even look finished. But when I see a polished answer, I don't automatically treat it as a verified answer.
That's the difference.
We're no longer trusting only that the technology works. We're trusting the information it produces, too. Ask the same question twice, and you might get different answers. Worse, a wrong answer can sound just as confident as a correct one.
So when I think about digital trust in AI, I come back to six practical questions:
Reliability: Can I depend on this result?
Security: Is the information I give it protected from unauthorized access?
Transparency: Where did this answer come from, and how can I check it?
Accountability: Who's responsible if the answer is wrong?
Fairness: Does the system produce equally good outcomes for different people?
Privacy: Should this system be collecting and keeping my data at all?
These aren't six unrelated boxes. In real use, they overlap. A reliable answer isn't much help if confidential information was exposed to produce it. A secure system still creates a trust problem if nobody can explain an important decision. And an accurate system can still be unacceptable if it consistently disadvantages some people.
Digital trust is the confidence people place in a technology system when they use it, share information with it, or rely on its results.
Before AI became part of everyday work, that trust was easier to define. I expect a website to protect my payment details. I expect a cloud service to keep my files available. And I expect a business platform to handle customer information securely. If the system follows its rules, I usually know what to expect.
AI makes that assumption less comfortable.
Ask an AI assistant to summarize a report, draft a customer email, or explain a piece of research, and you'll get an answer in seconds. It may be useful. It may even look finished. But when I see a polished answer, I don't automatically treat it as a verified answer.
That's the difference.
We're no longer trusting only that the technology works. We're trusting the information it produces, too. Ask the same question twice, and you might get different answers. Worse, a wrong answer can sound just as confident as a correct one.
So when I think about digital trust in AI, I come back to six practical questions:
Reliability: Can I depend on this result?
Security: Is the information I give it protected from unauthorized access?
Transparency: Where did this answer come from, and how can I check it?
Accountability: Who's responsible if the answer is wrong?
Fairness: Does the system produce equally good outcomes for different people?
Privacy: Should this system be collecting and keeping my data at all?
These aren't six unrelated boxes. In real use, they overlap. A reliable answer isn't much help if confidential information was exposed to produce it. A secure system still creates a trust problem if nobody can explain an important decision. And an accurate system can still be unacceptable if it consistently disadvantages some people.
Reliability
Start with a simple situation. An employee asks an AI assistant to prepare a summary for a client. The answer looks clear, the grammar is good, and nothing immediately seems wrong.
Would you send it?
I wouldn't—not without checking it first.
That's where reliability becomes more than a technical feature. An AI system needs to produce useful and reasonably consistent results for the task it's being used for, but no system should be treated as automatically correct just because the output sounds professional.
We saw the consequences of skipping that check in 2023. Two New York lawyers were sanctioned after submitting court filings containing nonexistent case citations and opinions generated by ChatGPT. The problem wasn't that they used an AI tool. They submitted its output without properly checking it.
That distinction matters.
AI can help produce a document, but the person who submits it still owns the result. If that document becomes part of a legal filing, a financial report, or another important decision, the final responsibility doesn't disappear because AI helped write it.
So reliability includes knowing when a human needs to verify what the system produced.
Start with a simple situation. An employee asks an AI assistant to prepare a summary for a client. The answer looks clear, the grammar is good, and nothing immediately seems wrong.
Would you send it?
I wouldn't—not without checking it first.
That's where reliability becomes more than a technical feature. An AI system needs to produce useful and reasonably consistent results for the task it's being used for, but no system should be treated as automatically correct just because the output sounds professional.
We saw the consequences of skipping that check in 2023. Two New York lawyers were sanctioned after submitting court filings containing nonexistent case citations and opinions generated by ChatGPT. The problem wasn't that they used an AI tool. They submitted its output without properly checking it.
That distinction matters.
AI can help produce a document, but the person who submits it still owns the result. If that document becomes part of a legal filing, a financial report, or another important decision, the final responsibility doesn't disappear because AI helped write it.
So reliability includes knowing when a human needs to verify what the system produced.
Security
Now change the situation slightly. The employee doesn't ask AI about a public document. Instead, they paste part of a confidential customer report into the tool.
The task still takes seconds. The risk has changed completely.
AI assistants can handle valuable material such as customer records, source code, and confidential reports. Before I put that information into an AI tool, I need to know where the data goes and who can reach it.
Samsung's experience in 2023 shows how quickly this can become a real problem. Employees entered sensitive company information into ChatGPT on several occasions, including source code and internal meeting information. Samsung then restricted the use of generative AI tools on company devices.
It's easy to imagine the same thing happening in a smaller company. An employee copies one confidential paragraph into an AI assistant to improve its wording. The task takes seconds. The security question doesn't.
That's why security isn't only an IT issue. It also depends on everyday workplace behaviour.
Now change the situation slightly. The employee doesn't ask AI about a public document. Instead, they paste part of a confidential customer report into the tool.
The task still takes seconds. The risk has changed completely.
AI assistants can handle valuable material such as customer records, source code, and confidential reports. Before I put that information into an AI tool, I need to know where the data goes and who can reach it.
Samsung's experience in 2023 shows how quickly this can become a real problem. Employees entered sensitive company information into ChatGPT on several occasions, including source code and internal meeting information. Samsung then restricted the use of generative AI tools on company devices.
It's easy to imagine the same thing happening in a smaller company. An employee copies one confidential paragraph into an AI assistant to improve its wording. The task takes seconds. The security question doesn't.
That's why security isn't only an IT issue. It also depends on everyday workplace behaviour.
Transparency
Suppose the same employee now uses AI to summarize a 30-page contract.
The assistant returns a neat conclusion. It sounds convincing. But there are no references to the clauses behind it.
How do I check that answer?
I can't easily tell what the system used, what it skipped, or where it might have misunderstood something. If the assistant instead points to the exact passages behind each important claim, the situation changes. I can verify the answer in minutes instead of searching through all 30 pages myself.
That's what transparency means to me in practical terms.
It doesn't require me to understand every technical detail inside an AI model. I need something much more useful: a source behind an important claim, a clear limitation, or a way to trace the answer back to its evidence.
Without that ability to check, even a useful answer asks for too much trust.
Suppose the same employee now uses AI to summarize a 30-page contract.
The assistant returns a neat conclusion. It sounds convincing. But there are no references to the clauses behind it.
How do I check that answer?
I can't easily tell what the system used, what it skipped, or where it might have misunderstood something. If the assistant instead points to the exact passages behind each important claim, the situation changes. I can verify the answer in minutes instead of searching through all 30 pages myself.
That's what transparency means to me in practical terms.
It doesn't require me to understand every technical detail inside an AI model. I need something much more useful: a source behind an important claim, a clear limitation, or a way to trace the answer back to its evidence.
Without that ability to check, even a useful answer asks for too much trust.
Accountability
But what happens when the answer is wrong?
"The AI made the decision" isn't an acceptable answer when a customer has been affected.
A real example came in February 2024, when the British Columbia Civil Resolution Tribunal found Air Canada responsible for misleading information provided by its website chatbot. The case, Moffatt v. Air Canada, involved a customer who relied on incorrect information about bereavement fares. The tribunal awarded C$650.88 in damages.
The chatbot had provided the information, but it wasn't the party that carried the responsibility.
That matters to me because it's easy to imagine an organization treating an AI system as if it were separate from the business. It isn't. Someone has to own the system, review problems, and have the authority to fix them.
A customer may interact with a chatbot, but the chatbot doesn't become the owner of the business.
But what happens when the answer is wrong?
"The AI made the decision" isn't an acceptable answer when a customer has been affected.
A real example came in February 2024, when the British Columbia Civil Resolution Tribunal found Air Canada responsible for misleading information provided by its website chatbot. The case, Moffatt v. Air Canada, involved a customer who relied on incorrect information about bereavement fares. The tribunal awarded C$650.88 in damages.
The chatbot had provided the information, but it wasn't the party that carried the responsibility.
That matters to me because it's easy to imagine an organization treating an AI system as if it were separate from the business. It isn't. Someone has to own the system, review problems, and have the authority to fix them.
A customer may interact with a chatbot, but the chatbot doesn't become the owner of the business.
Fairness
Once responsibility is clear, there's another question worth asking: does the system treat people fairly?
This matters whenever AI is used for decisions involving employment, customers, applications, or access to services. A system can follow its instructions perfectly and still produce an unfair outcome.
Amazon provides a well-known example. In 2018, Reuters reported that Amazon had abandoned an experimental recruiting system that used AI to review job applicants.
The system gave candidates scores from one to five stars, but it had learned patterns from historical résumés that disadvantaged women. It even penalized some résumés containing the word "women's."
The problem wasn't simply a bad final decision. The system had learned from historical data, and those patterns could reproduce an existing imbalance at scale.
That's why I wouldn't judge an AI system only by looking at a few outputs. I'd ask what data it learned from, how it was designed, and how people are using it.
A system can be consistent and still be unfair.
Once responsibility is clear, there's another question worth asking: does the system treat people fairly?
This matters whenever AI is used for decisions involving employment, customers, applications, or access to services. A system can follow its instructions perfectly and still produce an unfair outcome.
Amazon provides a well-known example. In 2018, Reuters reported that Amazon had abandoned an experimental recruiting system that used AI to review job applicants.
The system gave candidates scores from one to five stars, but it had learned patterns from historical résumés that disadvantaged women. It even penalized some résumés containing the word "women's."
The problem wasn't simply a bad final decision. The system had learned from historical data, and those patterns could reproduce an existing imbalance at scale.
That's why I wouldn't judge an AI system only by looking at a few outputs. I'd ask what data it learned from, how it was designed, and how people are using it.
A system can be consistent and still be unfair.
Privacy
Fairness isn't the only issue that can hide behind apparently normal AI use. Privacy raises a different question.
Security asks whether someone who shouldn't have access to data can get it. Privacy asks whether the data should have been collected or used in that way in the first place.
On 30 March 2023, Italy's data protection authority ordered a temporary limitation on OpenAI's processing of the personal data of Italian users. The authority raised concerns about the collection and processing of personal data used in ChatGPT, along with transparency and age-verification issues.
The distinction matters in ordinary workplace use. A company may have strong passwords and access controls, but that doesn't automatically mean it should send every customer detail or employee record to an AI system.
When I think about workplace AI, I find the simplest question is often the most useful:
Do we need to give the system this information in the first place?
Sometimes the safest piece of data is the piece of data the AI never receives.
Fairness isn't the only issue that can hide behind apparently normal AI use. Privacy raises a different question.
Security asks whether someone who shouldn't have access to data can get it. Privacy asks whether the data should have been collected or used in that way in the first place.
On 30 March 2023, Italy's data protection authority ordered a temporary limitation on OpenAI's processing of the personal data of Italian users. The authority raised concerns about the collection and processing of personal data used in ChatGPT, along with transparency and age-verification issues.
The distinction matters in ordinary workplace use. A company may have strong passwords and access controls, but that doesn't automatically mean it should send every customer detail or employee record to an AI system.
When I think about workplace AI, I find the simplest question is often the most useful:
Do we need to give the system this information in the first place?
Sometimes the safest piece of data is the piece of data the AI never receives.
How the Elements Fit Together
Now put the whole situation together.
An employee asks AI to prepare a customer document. The answer is useful, but can it be relied on? That's reliability.
The employee uses confidential customer information to create it. Was that information protected? That's security.
The document contains an important claim, but nobody can see where it came from. That's transparency.
The claim turns out to be wrong. Who fixes the problem and answers the customer? That's accountability.
The same AI system produces worse results for a particular group of customers. That's fairness.
And the company has collected far more personal information than the task actually required. That's privacy.
Seen this way, digital trust stops being an abstract idea. I can see it in the small decisions people make before, during, and after using AI.
For me, that's the important shift. Trust isn't something an organization can simply announce. It has to be built into the way AI is selected, used, checked, and managed.
And that leads to the next question: what makes AI different from other digital systems, and where can that trust break down?
Now put the whole situation together.
An employee asks AI to prepare a customer document. The answer is useful, but can it be relied on? That's reliability.
The employee uses confidential customer information to create it. Was that information protected? That's security.
The document contains an important claim, but nobody can see where it came from. That's transparency.
The claim turns out to be wrong. Who fixes the problem and answers the customer? That's accountability.
The same AI system produces worse results for a particular group of customers. That's fairness.
And the company has collected far more personal information than the task actually required. That's privacy.
Seen this way, digital trust stops being an abstract idea. I can see it in the small decisions people make before, during, and after using AI.
For me, that's the important shift. Trust isn't something an organization can simply announce. It has to be built into the way AI is selected, used, checked, and managed.
And that leads to the next question: what makes AI different from other digital systems, and where can that trust break down?
2. What Makes AI Different, and Where Can It Break Trust?
AI hasn't invented unreliable software, security problems, biased decisions, or privacy risks. We've been dealing with those problems for years. What changes with AI is how easily these risks can hide inside a tool that communicates in a confident, human-sounding way.
Think about a payroll system. If someone's salary comes out wrong, there's usually somewhere to start looking: a formula, a database entry, a setting, or a piece of code. With an AI system, you might get a polished answer without knowing which part of the process produced the problem.
That's one reason I see AI as a different kind of trust challenge. We aren't only trusting software to perform a task. We're trusting it to produce information that may influence what someone does next.
A useful way to understand the difference is to look at where trust can break.
AI hasn't invented unreliable software, security problems, biased decisions, or privacy risks. We've been dealing with those problems for years. What changes with AI is how easily these risks can hide inside a tool that communicates in a confident, human-sounding way.
Think about a payroll system. If someone's salary comes out wrong, there's usually somewhere to start looking: a formula, a database entry, a setting, or a piece of code. With an AI system, you might get a polished answer without knowing which part of the process produced the problem.
That's one reason I see AI as a different kind of trust challenge. We aren't only trusting software to perform a task. We're trusting it to produce information that may influence what someone does next.
A useful way to understand the difference is to look at where trust can break.
A Confident Answer Can Still Be Wrong
This is one of the first things I'd check before trusting an AI output.
Ask an AI assistant to summarize a report and you'll probably get a clean answer. Ask it to rewrite an email and you may have a usable draft in seconds. Ask it to prepare a short market summary for a meeting, and it can turn several pages of information into a few paragraphs.
That's useful. I wouldn't want to lose that benefit.
But good writing isn't the same thing as good evidence.
An AI system can produce a statement that sounds certain when the information behind it is incomplete or wrong. These errors are often called hallucinations. The uncomfortable part is that they don't necessarily look like mistakes. There may be no spelling error, broken sentence, or obvious warning.
The Mata v. Avianca, Inc. case makes this problem concrete. In May 2023, lawyers submitted a brief containing nonexistent legal authorities generated with ChatGPT. The court identified six fictitious cases, including Varghese v. China Southern Airlines, Shaboon v. EgyptAir, and Petersen v. Iran Air. In its June 2023 order, the court imposed a $5,000 sanction and required the lawyers to notify the affected courts and opposing counsel.
The important part, for me, isn't only the penalty. It's what happened before the penalty.
The information looked like legal research.
It wasn't.
Now imagine a less dramatic workplace example. An employee opens an AI assistant before a Monday meeting and asks for a market summary. The answer contains a customer trend, a percentage, and a recommendation. The employee copies the paragraph into a presentation because it already looks finished.
That's where I'd stop.
I'd open the source behind the claim, check its publication date, and trace the percentage back to the original document. If the source doesn't contain that number, I wouldn't put the claim in the presentation.
The check might take two minutes.
Skipping it could create a much bigger problem.
That's the habit I want around AI: check the important claim, not just the quality of the writing.
This is one of the first things I'd check before trusting an AI output.
Ask an AI assistant to summarize a report and you'll probably get a clean answer. Ask it to rewrite an email and you may have a usable draft in seconds. Ask it to prepare a short market summary for a meeting, and it can turn several pages of information into a few paragraphs.
That's useful. I wouldn't want to lose that benefit.
But good writing isn't the same thing as good evidence.
An AI system can produce a statement that sounds certain when the information behind it is incomplete or wrong. These errors are often called hallucinations. The uncomfortable part is that they don't necessarily look like mistakes. There may be no spelling error, broken sentence, or obvious warning.
The Mata v. Avianca, Inc. case makes this problem concrete. In May 2023, lawyers submitted a brief containing nonexistent legal authorities generated with ChatGPT. The court identified six fictitious cases, including Varghese v. China Southern Airlines, Shaboon v. EgyptAir, and Petersen v. Iran Air. In its June 2023 order, the court imposed a $5,000 sanction and required the lawyers to notify the affected courts and opposing counsel.
The important part, for me, isn't only the penalty. It's what happened before the penalty.
The information looked like legal research.
It wasn't.
Now imagine a less dramatic workplace example. An employee opens an AI assistant before a Monday meeting and asks for a market summary. The answer contains a customer trend, a percentage, and a recommendation. The employee copies the paragraph into a presentation because it already looks finished.
That's where I'd stop.
I'd open the source behind the claim, check its publication date, and trace the percentage back to the original document. If the source doesn't contain that number, I wouldn't put the claim in the presentation.
The check might take two minutes.
Skipping it could create a much bigger problem.
That's the habit I want around AI: check the important claim, not just the quality of the writing.
The Data Behind the System Matters
A wrong answer is one problem. But sometimes the problem starts much earlier, in the information used to build the system.
AI systems learn from data. That data influences the patterns they recognize and reproduce. If historical information contains gaps or unfair patterns, those patterns can sometimes appear in the system's results.
Amazon's experimental recruiting system shows what this can look like. In 2018, Reuters reported that Amazon had developed a recruiting tool that learned from historical résumés. The system reportedly developed patterns that disadvantaged women, including penalizing some résumés containing the word "women's."
The tool wasn't making a decision in a vacuum. It had learned from résumés submitted to Amazon over about 10 years. That history mattered because the system was learning from patterns already present in Amazon's recruiting data.
The original idea sounds reasonable: use AI to review a large number of résumés and save recruiters time.
The problem wasn't automation itself.
It was what the system had learned from the past.
If the Amazon recruiting example made you wonder how algorithms can quietly reproduce bias at scale, Cathy O’Neil’s Weapons of Math Destruction is a readable deep dive into exactly that.
Imagine a company using a similar system today. A recruiter uploads 500 applications, receives a ranked list, and starts with the candidates at the top. If nobody checks how that ranking was produced, an unfair pattern can influence who gets attention and who doesn't.
That example changes how I would evaluate the system.
I wouldn't ask only, "How accurate is it?"
I'd also ask:
What data went into the system?
Does that data represent the people affected by the result?
What happens when the system makes a different recommendation for one group than another?
Those questions matter beyond recruitment. An AI system used for lending, insurance, education, or customer service can make decisions at a scale that a person simply can't match.
If a person makes one poor decision, someone may notice it.
If an automated system repeats the same pattern across thousands of decisions, the consequences can be much larger.
A wrong answer is one problem. But sometimes the problem starts much earlier, in the information used to build the system.
AI systems learn from data. That data influences the patterns they recognize and reproduce. If historical information contains gaps or unfair patterns, those patterns can sometimes appear in the system's results.
Amazon's experimental recruiting system shows what this can look like. In 2018, Reuters reported that Amazon had developed a recruiting tool that learned from historical résumés. The system reportedly developed patterns that disadvantaged women, including penalizing some résumés containing the word "women's."
The tool wasn't making a decision in a vacuum. It had learned from résumés submitted to Amazon over about 10 years. That history mattered because the system was learning from patterns already present in Amazon's recruiting data.
The original idea sounds reasonable: use AI to review a large number of résumés and save recruiters time.
The problem wasn't automation itself.
It was what the system had learned from the past.
If the Amazon recruiting example made you wonder how algorithms can quietly reproduce bias at scale, Cathy O’Neil’s Weapons of Math Destruction is a readable deep dive into exactly that.
Imagine a company using a similar system today. A recruiter uploads 500 applications, receives a ranked list, and starts with the candidates at the top. If nobody checks how that ranking was produced, an unfair pattern can influence who gets attention and who doesn't.
That example changes how I would evaluate the system.
I wouldn't ask only, "How accurate is it?"
I'd also ask:
What data went into the system?
Does that data represent the people affected by the result?
What happens when the system makes a different recommendation for one group than another?
Those questions matter beyond recruitment. An AI system used for lending, insurance, education, or customer service can make decisions at a scale that a person simply can't match.
If a person makes one poor decision, someone may notice it.
If an automated system repeats the same pattern across thousands of decisions, the consequences can be much larger.
An AI Explanation Isn't Automatically a Real Explanation
The data explains part of the risk, but there's another issue: what happens when we want to understand a particular decision?
With some machine-learning systems, we can see what information went into the process and what result came out. Explaining exactly why that result happened can be much harder.
And this is where I'd be careful with AI-generated explanations.
Ask an AI system, "Why did you give me this answer?" It can usually give you a fluent response. That response may sound convincing. But sounding convincing doesn't prove that it accurately describes the internal process that produced the result.
That distinction matters when the result affects a person.
Imagine a recruiter using an AI system to rank job applicants. One applicant receives a low score and is rejected. The recruiter asks the system why.
It produces a neat explanation.
But what happens next?
Can someone check the factors that actually influenced the decision? Can the recruiter compare the result with the company's hiring rules? Can the applicant challenge an obvious error?
Saying "the model decided" doesn't provide much accountability.
For me, a useful test is simple: if the decision affects someone's job, money, benefits, or access to a service, there should be a way for a person to review the result rather than simply accept the model's explanation.
For an internal email, that level of review might be unnecessary.
For a hiring decision, a loan application, or an insurance claim, it isn't.
The higher the consequence, the stronger the human review should be.
The data explains part of the risk, but there's another issue: what happens when we want to understand a particular decision?
With some machine-learning systems, we can see what information went into the process and what result came out. Explaining exactly why that result happened can be much harder.
And this is where I'd be careful with AI-generated explanations.
Ask an AI system, "Why did you give me this answer?" It can usually give you a fluent response. That response may sound convincing. But sounding convincing doesn't prove that it accurately describes the internal process that produced the result.
That distinction matters when the result affects a person.
Imagine a recruiter using an AI system to rank job applicants. One applicant receives a low score and is rejected. The recruiter asks the system why.
It produces a neat explanation.
But what happens next?
Can someone check the factors that actually influenced the decision? Can the recruiter compare the result with the company's hiring rules? Can the applicant challenge an obvious error?
Saying "the model decided" doesn't provide much accountability.
For me, a useful test is simple: if the decision affects someone's job, money, benefits, or access to a service, there should be a way for a person to review the result rather than simply accept the model's explanation.
For an internal email, that level of review might be unnecessary.
For a hiring decision, a loan application, or an insurance claim, it isn't.
The higher the consequence, the stronger the human review should be.
Scale Can Make an Ordinary Mistake Much Bigger
There is another layer to consider once AI moves from one decision to hundreds or thousands.
A person writing 10 customer emails can make mistakes. An automated system handling 10,000 interactions can repeat the same mistake across a much larger group before anyone notices.
Samsung's experience in 2023 is a useful reminder. In April that year, reports emerged that Samsung employees had entered sensitive information into ChatGPT, including source code and internal meeting information. Samsung then introduced restrictions on generative AI use on company devices and said it was working on measures to prevent similar incidents.
The employees weren't necessarily trying to create a security incident. They were using a convenient tool to get work done.
That's easy to understand. An employee has a document open, the AI assistant is one browser tab away, and the quickest way to get help may seem to be copying the relevant text into the chat box.
The problem is what happens to that information after it leaves the company's approved environment.
That's exactly why a rule such as "use AI responsibly" isn't enough.
If I were writing the workplace policy, I'd want an employee to know the answer before they paste anything into an AI assistant.
Can I paste a customer record?
Can I paste source code?
Can I upload an internal report?
Which AI tools has the company approved?
Who do I contact when I'm not sure?
Those questions are much more useful than a general warning. They give employees something they can actually use when they're sitting in front of an AI tool and deciding what to upload.
And once 100 employees start using the same AI service every day, the issue becomes bigger than individual behaviour. The tool has become part of the organization's workflow.
There is another layer to consider once AI moves from one decision to hundreds or thousands.
A person writing 10 customer emails can make mistakes. An automated system handling 10,000 interactions can repeat the same mistake across a much larger group before anyone notices.
Samsung's experience in 2023 is a useful reminder. In April that year, reports emerged that Samsung employees had entered sensitive information into ChatGPT, including source code and internal meeting information. Samsung then introduced restrictions on generative AI use on company devices and said it was working on measures to prevent similar incidents.
The employees weren't necessarily trying to create a security incident. They were using a convenient tool to get work done.
That's easy to understand. An employee has a document open, the AI assistant is one browser tab away, and the quickest way to get help may seem to be copying the relevant text into the chat box.
The problem is what happens to that information after it leaves the company's approved environment.
That's exactly why a rule such as "use AI responsibly" isn't enough.
If I were writing the workplace policy, I'd want an employee to know the answer before they paste anything into an AI assistant.
Can I paste a customer record?
Can I paste source code?
Can I upload an internal report?
Which AI tools has the company approved?
Who do I contact when I'm not sure?
Those questions are much more useful than a general warning. They give employees something they can actually use when they're sitting in front of an AI tool and deciding what to upload.
And once 100 employees start using the same AI service every day, the issue becomes bigger than individual behaviour. The tool has become part of the organization's workflow.
Approval Doesn't Mean the System Will Stay the Same
That brings us to a problem that doesn't exist in quite the same way with ordinary software: the AI service itself can change.
A company can assess an AI tool, approve it, train employees, and build procedures around it. Then the provider releases a new model or changes an important part of the service.
The employees may do everything exactly as they did before.
The technology has changed underneath them.
If a company approved an AI assistant in 2025 and the provider replaced the underlying model in 2026, I wouldn't simply assume that the old assessment still covers the new system. I'd want someone to check what changed and whether the original risks still apply.
The change could involve the model itself, connected tools, access permissions, retention settings, or the way information moves through the service.
For example, an employee might open the same AI workspace they used last month and suddenly find that it can access a connected document system or another business application.
The interface may look almost identical.
The risk isn't necessarily the same.
That difference matters.
This is why I don't see AI governance as a one-time approval. A tool needs attention after deployment, too.
If the provider changes the model, adds a new capability, changes how data is handled, or connects the service to another system, the organization should have a reason to review the original approval.
That brings us to a problem that doesn't exist in quite the same way with ordinary software: the AI service itself can change.
A company can assess an AI tool, approve it, train employees, and build procedures around it. Then the provider releases a new model or changes an important part of the service.
The employees may do everything exactly as they did before.
The technology has changed underneath them.
If a company approved an AI assistant in 2025 and the provider replaced the underlying model in 2026, I wouldn't simply assume that the old assessment still covers the new system. I'd want someone to check what changed and whether the original risks still apply.
The change could involve the model itself, connected tools, access permissions, retention settings, or the way information moves through the service.
For example, an employee might open the same AI workspace they used last month and suddenly find that it can access a connected document system or another business application.
The interface may look almost identical.
The risk isn't necessarily the same.
That difference matters.
This is why I don't see AI governance as a one-time approval. A tool needs attention after deployment, too.
If the provider changes the model, adds a new capability, changes how data is handled, or connects the service to another system, the organization should have a reason to review the original approval.
Sometimes the Human Starts Trusting the Tool Too Much
Even when the technology stays within the expected limits, the people using it can change their behaviour.
If an AI assistant gives 10 useful answers in a row, checking the eleventh can start to feel unnecessary. That's automation bias: the tendency to rely too heavily on an automated recommendation.
It's easy to imagine this happening in customer service.
An employee starts by checking every AI-generated reply carefully. Most of the drafts are fine. After a few weeks, the employee reads them quickly and sends them.
Then one reply contains a serious mistake.
The AI didn't necessarily become worse at that moment.
The human checking process became weaker.
That's why I'd rather see clear review rules than a vague instruction to "be careful with AI." Employees should know when they have to stop and check an output, especially when it involves sensitive customer information, financial decisions, legal material, employment decisions, or external communication.
For example, an employee might be allowed to send an AI-assisted reply about a product's basic features without extra approval, while a reply involving a $500 refund, a legal complaint, or a financial commitment requires a human review.
The goal isn't to make people distrust AI.
It's to stop familiarity from becoming blind trust.
Even when the technology stays within the expected limits, the people using it can change their behaviour.
If an AI assistant gives 10 useful answers in a row, checking the eleventh can start to feel unnecessary. That's automation bias: the tendency to rely too heavily on an automated recommendation.
It's easy to imagine this happening in customer service.
An employee starts by checking every AI-generated reply carefully. Most of the drafts are fine. After a few weeks, the employee reads them quickly and sends them.
Then one reply contains a serious mistake.
The AI didn't necessarily become worse at that moment.
The human checking process became weaker.
That's why I'd rather see clear review rules than a vague instruction to "be careful with AI." Employees should know when they have to stop and check an output, especially when it involves sensitive customer information, financial decisions, legal material, employment decisions, or external communication.
For example, an employee might be allowed to send an AI-assisted reply about a product's basic features without extra approval, while a reply involving a $500 refund, a legal complaint, or a financial commitment requires a human review.
The goal isn't to make people distrust AI.
It's to stop familiarity from becoming blind trust.
The AI Provider Is Part of the Trust Chain
There is still one more piece of the puzzle.
Most businesses don't build their own AI models. They rely on external providers.
That means an organization's trust doesn't stop with its own security controls. It extends to the provider's infrastructure, data practices, model updates, security measures, and contractual terms.
This doesn't mean an external AI service is automatically unsafe. It means choosing one is also a decision about the organization behind the technology.
If I were evaluating an AI service for customer information, employee data, financial documents, or source code, I'd want clear answers to a few basic questions:
What information does the service receive?
How is that information handled?
What happens to the data after the task?
How are major model changes communicated?
What security controls does the provider offer?
What happens if the service no longer meets the company's requirements?
I'd also want to know whether the provider allows the company to control user access, manage permissions, and remove access when an employee leaves.
For a small business, that last point can be easy to overlook. Someone may leave the company while their AI account, API key, shared workspace, or connected application is still active.
These questions aren't as exciting as comparing model features, but they're closer to the real trust issue.
Once sensitive business information passes through an external AI service, the provider becomes part of the trust chain.
There is still one more piece of the puzzle.
Most businesses don't build their own AI models. They rely on external providers.
That means an organization's trust doesn't stop with its own security controls. It extends to the provider's infrastructure, data practices, model updates, security measures, and contractual terms.
This doesn't mean an external AI service is automatically unsafe. It means choosing one is also a decision about the organization behind the technology.
If I were evaluating an AI service for customer information, employee data, financial documents, or source code, I'd want clear answers to a few basic questions:
What information does the service receive?
How is that information handled?
What happens to the data after the task?
How are major model changes communicated?
What security controls does the provider offer?
What happens if the service no longer meets the company's requirements?
I'd also want to know whether the provider allows the company to control user access, manage permissions, and remove access when an employee leaves.
For a small business, that last point can be easy to overlook. Someone may leave the company while their AI account, API key, shared workspace, or connected application is still active.
These questions aren't as exciting as comparing model features, but they're closer to the real trust issue.
Once sensitive business information passes through an external AI service, the provider becomes part of the trust chain.
So Where Can Trust Actually Break?
After looking at these examples, I see three connected pressure points.
The output can fail. The system can produce a confident error, leave out an important detail, or give an explanation that isn't enough for the decision being made.
The system behind the output can change. Training data, model updates, infrastructure, and third-party providers all affect how an AI service behaves.
People can amplify the problem. Employees can become too comfortable with a useful tool, while automation can allow one mistake to spread across a large number of customers or decisions.
None of this means we should stop using AI.
Quite the opposite. These tools can save time and make difficult work easier.
But I'd be uncomfortable with an organization asking only, "Does this AI work?"
I'd rather hear:
What could go wrong?
How would we notice?
Who reviews the result?
Who takes responsibility when something goes wrong?
Those questions change the conversation.
The organization isn't simply deciding whether an AI tool is useful anymore. It's deciding where the tool can be trusted, where human judgment has to remain involved, and what happens when the system reaches its limits.
That's the point where using AI becomes AI governance.
After looking at these examples, I see three connected pressure points.
The output can fail. The system can produce a confident error, leave out an important detail, or give an explanation that isn't enough for the decision being made.
The system behind the output can change. Training data, model updates, infrastructure, and third-party providers all affect how an AI service behaves.
People can amplify the problem. Employees can become too comfortable with a useful tool, while automation can allow one mistake to spread across a large number of customers or decisions.
None of this means we should stop using AI.
Quite the opposite. These tools can save time and make difficult work easier.
But I'd be uncomfortable with an organization asking only, "Does this AI work?"
I'd rather hear:
What could go wrong?
How would we notice?
Who reviews the result?
Who takes responsibility when something goes wrong?
Those questions change the conversation.
The organization isn't simply deciding whether an AI tool is useful anymore. It's deciding where the tool can be trusted, where human judgment has to remain involved, and what happens when the system reaches its limits.
That's the point where using AI becomes AI governance.
3. What Using AI Tools Taught Me About Trust
I didn't start this test because I wanted to decide which AI tool was best.
I wanted to answer one simple question:
Does regular exercise improve memory in adults?
I asked ChatGPT and Perplexity almost the same question. Then I tried Gemini Notebook in a different way. Instead of letting it search for information, I gave it one specific document and told it to use only that source.
That difference turned out to matter.
I ran the tests on [date], using [ChatGPT version], [Perplexity mode], and [Gemini Notebook version]. Results may differ today.
I didn't start this test because I wanted to decide which AI tool was best.
I wanted to answer one simple question:
Does regular exercise improve memory in adults?
I asked ChatGPT and Perplexity almost the same question. Then I tried Gemini Notebook in a different way. Instead of letting it search for information, I gave it one specific document and told it to use only that source.
That difference turned out to matter.
I ran the tests on [date], using [ChatGPT version], [Perplexity mode], and [Gemini Notebook version]. Results may differ today.
I Started With ChatGPT
I asked:
Does regular exercise improve memory in adults? Please give me a clear answer, explain the main evidence, and cite reliable sources so I can verify the claims.
The answer was yes, with an important qualification: regular exercise can improve memory in adults, but the effect is generally modest.
ChatGPT pointed me to a 2025 umbrella review covering 133 systematic reviews, 2,724 randomized controlled trials, and 258,279 participants. For memory specifically, the analysis included 62 meta-analyses and 50,975 participants, with a reported standardized mean difference of 0.26.
It also pointed to World Health Organization guidance recommending 150–300 minutes of moderate-intensity physical activity per week, or 75–150 minutes of vigorous activity, together with muscle-strengthening activities on at least two days per week.
At first glance, that looked like a solid answer.
But I wasn't ready to take it at face value.
I opened the paper and checked what the researchers had actually found. That's where the first problem appeared.
The review covers all ages and populations, not adults only. The authors reported that memory improvements were greater for children and adolescents than for adults and older adults. So the 0.26 figure was the overall result for the memory analysis, not a precise estimate of the effect in adults alone.
The answer wasn't simply wrong. It was less precise than my original question.
I found another important detail. Many of the included reviews were rated low or critically low in quality using the AMSTAR-2 tool. That clearly matters when interpreting the evidence.
At the same time, the researchers reported that the findings remained statistically significant after excluding the low- and critically-low-quality reviews.
That changed how I read the AI answer.
The citation had pointed me in the right direction, but I still had to check what the paper actually said.
That's the first lesson I took from the test: a citation makes an AI answer easier to investigate, but it doesn't automatically make the answer trustworthy.
I asked:
Does regular exercise improve memory in adults? Please give me a clear answer, explain the main evidence, and cite reliable sources so I can verify the claims.
The answer was yes, with an important qualification: regular exercise can improve memory in adults, but the effect is generally modest.
ChatGPT pointed me to a 2025 umbrella review covering 133 systematic reviews, 2,724 randomized controlled trials, and 258,279 participants. For memory specifically, the analysis included 62 meta-analyses and 50,975 participants, with a reported standardized mean difference of 0.26.
It also pointed to World Health Organization guidance recommending 150–300 minutes of moderate-intensity physical activity per week, or 75–150 minutes of vigorous activity, together with muscle-strengthening activities on at least two days per week.
At first glance, that looked like a solid answer.
But I wasn't ready to take it at face value.
I opened the paper and checked what the researchers had actually found. That's where the first problem appeared.
The review covers all ages and populations, not adults only. The authors reported that memory improvements were greater for children and adolescents than for adults and older adults. So the 0.26 figure was the overall result for the memory analysis, not a precise estimate of the effect in adults alone.
The answer wasn't simply wrong. It was less precise than my original question.
I found another important detail. Many of the included reviews were rated low or critically low in quality using the AMSTAR-2 tool. That clearly matters when interpreting the evidence.
At the same time, the researchers reported that the findings remained statistically significant after excluding the low- and critically-low-quality reviews.
That changed how I read the AI answer.
The citation had pointed me in the right direction, but I still had to check what the paper actually said.
That's the first lesson I took from the test: a citation makes an AI answer easier to investigate, but it doesn't automatically make the answer trustworthy.
Then I Asked Perplexity
Next, I gave Perplexity essentially the same question.
Its answer was also positive. It described a modest improvement in memory, reported an SMD of 0.26 with a 95% confidence interval of 0.20–0.32, and brought in additional research on aerobic exercise and episodic memory in later adulthood.
So far, nothing seemed unusual.
ChatGPT and Perplexity gave me similar conclusions. Both provided sources. Both gave me numbers I could check. Both sounded confident enough to be convincing.
And that was exactly what made me stop.
If two AI tools give me a similar answer, does that count as independent confirmation?
Not really.
The two systems can rely on overlapping research and the same underlying evidence. In this case, both answers pointed back to the same body of research.
Their agreement was reassuring.
But it wasn't proof.
So I went back to the sources instead of treating the agreement itself as evidence.
That small decision matters to me. If two tools repeat the same conclusion, the repetition can make an answer feel more certain than it really is. The important question is still whether the underlying evidence supports the claim.
Next, I gave Perplexity essentially the same question.
Its answer was also positive. It described a modest improvement in memory, reported an SMD of 0.26 with a 95% confidence interval of 0.20–0.32, and brought in additional research on aerobic exercise and episodic memory in later adulthood.
So far, nothing seemed unusual.
ChatGPT and Perplexity gave me similar conclusions. Both provided sources. Both gave me numbers I could check. Both sounded confident enough to be convincing.
And that was exactly what made me stop.
If two AI tools give me a similar answer, does that count as independent confirmation?
Not really.
The two systems can rely on overlapping research and the same underlying evidence. In this case, both answers pointed back to the same body of research.
Their agreement was reassuring.
But it wasn't proof.
So I went back to the sources instead of treating the agreement itself as evidence.
That small decision matters to me. If two tools repeat the same conclusion, the repetition can make an answer feel more certain than it really is. The important question is still whether the underlying evidence supports the claim.
Gemini Notebook Gave Me a Different Kind of Answer
This was the part of the test I found most interesting.
I supplied the WHO Guidelines on Physical Activity and Sedentary Behaviour (2020) as the only source and asked:
Based only on the source I provided, does regular exercise improve memory in adults? Please give me a clear answer and cite the relevant parts of the source.
I expected another answer that I could compare with the first two.
I didn't get one.
Gemini Notebook told me that the document I had supplied didn't state whether regular exercise improves memory in adults.
That response was useful precisely because it stopped.
It didn't try to stretch the document into an answer it couldn't support. It showed me where the source ended instead.
That gave me a different way to think about AI reliability.
When I ask a tool to work from one document, I don't necessarily want it to fill every gap. If the document doesn't contain the information I need, "this source doesn't answer your question" can be more useful than a complete-sounding answer.
I also want to be careful about what this test proves. It tells me how the tool behaved with the document I supplied. It doesn't tell me what the WHO says in its other materials, and I didn't treat the result as evidence about the wider research on exercise and memory.
That distinction is important.
This was the part of the test I found most interesting.
I supplied the WHO Guidelines on Physical Activity and Sedentary Behaviour (2020) as the only source and asked:
Based only on the source I provided, does regular exercise improve memory in adults? Please give me a clear answer and cite the relevant parts of the source.
I expected another answer that I could compare with the first two.
I didn't get one.
Gemini Notebook told me that the document I had supplied didn't state whether regular exercise improves memory in adults.
That response was useful precisely because it stopped.
It didn't try to stretch the document into an answer it couldn't support. It showed me where the source ended instead.
That gave me a different way to think about AI reliability.
When I ask a tool to work from one document, I don't necessarily want it to fill every gap. If the document doesn't contain the information I need, "this source doesn't answer your question" can be more useful than a complete-sounding answer.
I also want to be careful about what this test proves. It tells me how the tool behaved with the document I supplied. It doesn't tell me what the WHO says in its other materials, and I didn't treat the result as evidence about the wider research on exercise and memory.
That distinction is important.
Three Tools, Three Trust Behaviors
By the end of the test, I wasn't asking which tool was "best."
I was asking a more practical question: what kind of trust does each tool earn?
ChatGPT and Perplexity helped me find and summarize research. They gave me sources, figures, and explanations that I could investigate further.
Gemini Notebook was useful in a different way. Because I had told it to stay within one source, its refusal to go beyond that source showed me a boundary that mattered.
That made me rethink how I judge an AI answer.
I used to be impressed when an answer was clear, detailed, and confident.
Now I pay more attention to what happens underneath that answer.
I ask:
Where did this answer come from?
Can I verify the important claims?
Does the source actually support the conclusion?
Does the tool tell me when the evidence isn't enough?
The last question is probably the one I value most.
A tool that can say "I don't know" or "this source doesn't say" can be more useful than one that always produces a complete-sounding answer.
That's not because uncertainty is better than an answer.
It's because knowing where the evidence stops helps me decide what to do next.
I still use AI tools because they save time. But this test changed what I do after I get an answer.
I don't automatically stop at the AI's response anymore.
For anything important, I take one more step: I go from the AI's answer back to the evidence behind it.
That, for me, is where digital trust becomes practical.
Trust in AI isn't about expecting the system to be perfect.
It's about knowing when the system has given me enough to proceed, when I need to verify the result, and when the evidence simply isn't there.
By the end of the test, I wasn't asking which tool was "best."
I was asking a more practical question: what kind of trust does each tool earn?
ChatGPT and Perplexity helped me find and summarize research. They gave me sources, figures, and explanations that I could investigate further.
Gemini Notebook was useful in a different way. Because I had told it to stay within one source, its refusal to go beyond that source showed me a boundary that mattered.
That made me rethink how I judge an AI answer.
I used to be impressed when an answer was clear, detailed, and confident.
Now I pay more attention to what happens underneath that answer.
I ask:
Where did this answer come from?
Can I verify the important claims?
Does the source actually support the conclusion?
Does the tool tell me when the evidence isn't enough?
The last question is probably the one I value most.
A tool that can say "I don't know" or "this source doesn't say" can be more useful than one that always produces a complete-sounding answer.
That's not because uncertainty is better than an answer.
It's because knowing where the evidence stops helps me decide what to do next.
I still use AI tools because they save time. But this test changed what I do after I get an answer.
I don't automatically stop at the AI's response anymore.
For anything important, I take one more step: I go from the AI's answer back to the evidence behind it.
That, for me, is where digital trust becomes practical.
Trust in AI isn't about expecting the system to be perfect.
It's about knowing when the system has given me enough to proceed, when I need to verify the result, and when the evidence simply isn't there.
4. What Is AI Governance and Why Is It No Longer Optional?
I wanted to see what AI governance actually looks like when an AI system is doing something ordinary.
So I gave ChatGPT a customer-support task.
The customer had ordered a product two weeks earlier. The company had promised delivery within five business days, but the order still hadn't arrived. The customer asked two simple questions: Where is my order? When will I receive it?
I told ChatGPT not to invent an order status, delivery date, refund, compensation, or any other missing fact.
Its answer sounded like a normal customer-service reply. It apologized, asked for the order number, and avoided inventing a delivery date.
Then I read it again.
One sentence caught my attention:
"We'd be happy to look into this for you."
Wait.
Could the AI actually look into the order?
No. It had no order number, no access to the company's system, no tracking information, and no connection to the delivery company.
That small sentence was enough to make me uncomfortable.
So I changed the prompt instead of changing the model.
I added five rules. The AI couldn't invent missing facts. It couldn't claim to have checked a system or contacted anyone. It had to identify missing information. It could draft the response, but a human employee had to review it before sending. And it couldn't promise that the company would do something unless that information had been provided.
The second answer was much more restrained.
It said there wasn't enough information to determine the location of the order or give an estimated delivery date. It asked for the order number so a member of the team could review the situation.
I preferred this answer, even though it told the customer less.
Why?
Because I knew exactly where the AI's knowledge stopped.
Then I tried a third test.
This time, the order status was "Delivered." The customer still said, "I haven't received my order."
That contradiction created an obvious temptation for the AI: explain it.
It could have guessed that the package had been left somewhere. It could have blamed incorrect tracking. It could have suggested that the delivery had been recorded too early.
It didn't.
Instead, the response said that the available information was inconsistent and that more information was needed before the situation could be determined.
For me, that was the strongest response of the three.
The AI hadn't solved the problem.
It had recognized that it couldn't safely solve it.
That small experiment changed how I think about AI governance.
Before doing it, I would probably have described governance with familiar words: policies, compliance, risk management, accountability, oversight.
Those words are accurate, but they don't tell me what happens when an employee is staring at an AI-generated customer reply on a Tuesday afternoon.
The experiment gave me a more practical definition.
AI governance is deciding what the system is allowed to do, what it isn't allowed to claim, what information it can use, when a human must review its output, and what happens when the system reaches a limit.
The NIST AI Risk Management Framework expresses the idea through four functions: Govern, Map, Measure, and Manage. NIST describes Govern as a cross-cutting function that continues throughout the AI lifecycle. The framework is voluntary.
In practical terms, Govern is about setting responsibilities, policies, and expectations around AI use. Map is about understanding the context, intended use, and risks. Measure is about assessing whether those risks and system behavior are understood and controlled. Manage is about taking action in response to identified risks.
I don't think a small business needs to start by copying a large organization's governance program.
It can start with one real problem.
For example, a company might tell employees:
Don't enter confidential customer information into unapproved AI tools.
That's a useful rule. But then someone has to answer the questions behind it.
Which tools are approved? What counts as confidential? Who approves a new tool? What happens if an employee makes a mistake?
The sentence is a policy.
The answers around it are governance.
My customer-support test showed the same thing. ChatGPT could write the response, but it couldn't check the order system. It couldn't verify the tracking information. It couldn't call the delivery company. It couldn't decide whether the word "Delivered" was correct.
A human still owned the outcome.
The Moffatt v. Air Canada case makes that responsibility especially clear. The British Columbia Civil Resolution Tribunal held Air Canada responsible for misleading information provided by its chatbot. The company couldn't simply point to the chatbot and treat it as a separate source of responsibility.
I'm not saying that every AI mistake will produce the same legal result. The useful lesson is simpler: putting AI between an employee and a customer doesn't remove the organization's responsibility.
The level of control should also depend on the consequences.
If AI makes an awkward change to an internal email, someone can fix it.
If an employee puts confidential customer information into an external AI service, that's a different problem.
If AI contributes to an important decision about a person, the consequences can be more serious again.
So I wouldn't give every AI task the same controls. I'd ask a more useful question:
What could happen if this particular use goes wrong?
That question helps connect governance to risk instead of turning it into paperwork.
Compliance is part of the picture, but governance is wider. Compliance asks whether the organization meets the requirements that apply to it. Governance asks how the organization actually controls and manages its AI use.
The EU AI Act is now part of that environment. The regulation entered into force on 1 August 2024, with its requirements introduced in stages. Prohibited AI practices and AI literacy requirements began applying on 2 February 2025. Governance rules and general-purpose AI obligations began applying on 2 August 2025. The regulation's general application date is 2 August 2026, but the high-risk requirements in Chapter III, Sections 1–3 have been postponed: they apply from 2 December 2027 to high-risk AI systems under Article 6(2) and Annex III, and from 2 August 2028 to high-risk AI systems under Article 6(1) and Annex I.
That doesn't mean every company using AI has the same obligations. The answer depends on the system, how it is used, and the organization's role.
For a company trying to get started, I'd reduce the first conversation to four questions:
What AI are we using? What are we using it for? What information goes into it? What happens when it gets something wrong?
Organizations looking for a formal management system can also look at ISO/IEC 42001:2023, which provides requirements for establishing and continually improving an AI management system.
But if I were helping a small company start tomorrow, I wouldn't ask for a fifty-page policy.
I'd pick one AI use.
I'd look at the input.
I'd look at the output.
I'd decide what the system can and can't do.
I'd define when a person has to review the result.
And I'd make sure employees know what to do when the AI produces something that looks wrong, incomplete, contradictory, or unsafe.
That's a much more useful starting point than simply telling people to "use AI responsibly."
There's another problem, too.
If the approved AI tools are so restrictive that employees find them frustrating, they may start using other tools quietly. At that point, the company can lose visibility over its own AI use.
So governance has to work in real life.
It needs enough control to protect the organization without becoming so complicated that employees work around it.
When I look back at my three tests, the difference is surprisingly small.
In the first test, one harmless-looking sentence suggested a capability the AI didn't have.
In the second, clearer rules changed the response.
In the third, the AI faced contradictory information and didn't fill the gap with a convenient story.
That's the part I keep coming back to.
The model didn't become smarter. The rules around the model became clearer.
For me, that's the connection between AI governance and digital trust.
Trust doesn't mean believing that AI will always be right.
It means knowing what the system can do, knowing what it can't do, and knowing who steps in when the answer isn't enough.
The next step is to take those boundaries beyond individual prompts and apply them across the AI lifecycle, from planning and data to testing, deployment, human oversight, and ongoing monitoring.
I wanted to see what AI governance actually looks like when an AI system is doing something ordinary.
So I gave ChatGPT a customer-support task.
The customer had ordered a product two weeks earlier. The company had promised delivery within five business days, but the order still hadn't arrived. The customer asked two simple questions: Where is my order? When will I receive it?
I told ChatGPT not to invent an order status, delivery date, refund, compensation, or any other missing fact.
Its answer sounded like a normal customer-service reply. It apologized, asked for the order number, and avoided inventing a delivery date.
Then I read it again.
One sentence caught my attention:
"We'd be happy to look into this for you."
Wait.
Could the AI actually look into the order?
No. It had no order number, no access to the company's system, no tracking information, and no connection to the delivery company.
That small sentence was enough to make me uncomfortable.
So I changed the prompt instead of changing the model.
I added five rules. The AI couldn't invent missing facts. It couldn't claim to have checked a system or contacted anyone. It had to identify missing information. It could draft the response, but a human employee had to review it before sending. And it couldn't promise that the company would do something unless that information had been provided.
The second answer was much more restrained.
It said there wasn't enough information to determine the location of the order or give an estimated delivery date. It asked for the order number so a member of the team could review the situation.
I preferred this answer, even though it told the customer less.
Why?
Because I knew exactly where the AI's knowledge stopped.
Then I tried a third test.
This time, the order status was "Delivered." The customer still said, "I haven't received my order."
That contradiction created an obvious temptation for the AI: explain it.
It could have guessed that the package had been left somewhere. It could have blamed incorrect tracking. It could have suggested that the delivery had been recorded too early.
It didn't.
Instead, the response said that the available information was inconsistent and that more information was needed before the situation could be determined.
For me, that was the strongest response of the three.
The AI hadn't solved the problem.
It had recognized that it couldn't safely solve it.
That small experiment changed how I think about AI governance.
Before doing it, I would probably have described governance with familiar words: policies, compliance, risk management, accountability, oversight.
Those words are accurate, but they don't tell me what happens when an employee is staring at an AI-generated customer reply on a Tuesday afternoon.
The experiment gave me a more practical definition.
AI governance is deciding what the system is allowed to do, what it isn't allowed to claim, what information it can use, when a human must review its output, and what happens when the system reaches a limit.
The NIST AI Risk Management Framework expresses the idea through four functions: Govern, Map, Measure, and Manage. NIST describes Govern as a cross-cutting function that continues throughout the AI lifecycle. The framework is voluntary.
In practical terms, Govern is about setting responsibilities, policies, and expectations around AI use. Map is about understanding the context, intended use, and risks. Measure is about assessing whether those risks and system behavior are understood and controlled. Manage is about taking action in response to identified risks.
I don't think a small business needs to start by copying a large organization's governance program.
It can start with one real problem.
For example, a company might tell employees:
Don't enter confidential customer information into unapproved AI tools.
That's a useful rule. But then someone has to answer the questions behind it.
Which tools are approved? What counts as confidential? Who approves a new tool? What happens if an employee makes a mistake?
The sentence is a policy.
The answers around it are governance.
My customer-support test showed the same thing. ChatGPT could write the response, but it couldn't check the order system. It couldn't verify the tracking information. It couldn't call the delivery company. It couldn't decide whether the word "Delivered" was correct.
A human still owned the outcome.
The Moffatt v. Air Canada case makes that responsibility especially clear. The British Columbia Civil Resolution Tribunal held Air Canada responsible for misleading information provided by its chatbot. The company couldn't simply point to the chatbot and treat it as a separate source of responsibility.
I'm not saying that every AI mistake will produce the same legal result. The useful lesson is simpler: putting AI between an employee and a customer doesn't remove the organization's responsibility.
The level of control should also depend on the consequences.
If AI makes an awkward change to an internal email, someone can fix it.
If an employee puts confidential customer information into an external AI service, that's a different problem.
If AI contributes to an important decision about a person, the consequences can be more serious again.
So I wouldn't give every AI task the same controls. I'd ask a more useful question:
What could happen if this particular use goes wrong?
That question helps connect governance to risk instead of turning it into paperwork.
Compliance is part of the picture, but governance is wider. Compliance asks whether the organization meets the requirements that apply to it. Governance asks how the organization actually controls and manages its AI use.
The EU AI Act is now part of that environment. The regulation entered into force on 1 August 2024, with its requirements introduced in stages. Prohibited AI practices and AI literacy requirements began applying on 2 February 2025. Governance rules and general-purpose AI obligations began applying on 2 August 2025. The regulation's general application date is 2 August 2026, but the high-risk requirements in Chapter III, Sections 1–3 have been postponed: they apply from 2 December 2027 to high-risk AI systems under Article 6(2) and Annex III, and from 2 August 2028 to high-risk AI systems under Article 6(1) and Annex I.
That doesn't mean every company using AI has the same obligations. The answer depends on the system, how it is used, and the organization's role.
For a company trying to get started, I'd reduce the first conversation to four questions:
What AI are we using? What are we using it for? What information goes into it? What happens when it gets something wrong?
Organizations looking for a formal management system can also look at ISO/IEC 42001:2023, which provides requirements for establishing and continually improving an AI management system.
But if I were helping a small company start tomorrow, I wouldn't ask for a fifty-page policy.
I'd pick one AI use.
I'd look at the input.
I'd look at the output.
I'd decide what the system can and can't do.
I'd define when a person has to review the result.
And I'd make sure employees know what to do when the AI produces something that looks wrong, incomplete, contradictory, or unsafe.
That's a much more useful starting point than simply telling people to "use AI responsibly."
There's another problem, too.
If the approved AI tools are so restrictive that employees find them frustrating, they may start using other tools quietly. At that point, the company can lose visibility over its own AI use.
So governance has to work in real life.
It needs enough control to protect the organization without becoming so complicated that employees work around it.
When I look back at my three tests, the difference is surprisingly small.
In the first test, one harmless-looking sentence suggested a capability the AI didn't have.
In the second, clearer rules changed the response.
In the third, the AI faced contradictory information and didn't fill the gap with a convenient story.
That's the part I keep coming back to.
The model didn't become smarter. The rules around the model became clearer.
For me, that's the connection between AI governance and digital trust.
Trust doesn't mean believing that AI will always be right.
It means knowing what the system can do, knowing what it can't do, and knowing who steps in when the answer isn't enough.
The next step is to take those boundaries beyond individual prompts and apply them across the AI lifecycle, from planning and data to testing, deployment, human oversight, and ongoing monitoring.
5. Building Trust Throughout the AI Lifecycle
When I started working through AI governance for this article, I figured the frameworks would be the hard part. I was wrong. The harder part was figuring out what those frameworks actually mean when you're using an AI tool on a busy afternoon and a customer's waiting for an answer.
While I tested tools, I kept coming back to the same five questions. What information is safe to give the system? What happens when the information's incomplete? Who checks an answer that looks convincing? What happens after the provider changes the model? And who gets to stop the process when something goes wrong?
Those questions eventually gave me a much simpler way to look at the AI lifecycle. Instead of seeing governance as a document that gets approved and filed away, I started seeing it as a series of decisions that follows the system from the day a company considers buying it to the day it's replaced or shut down.
That became easier to map once I went back to the OECD's Recommendation of the Council on Artificial Intelligence (adopted in 2019 and amended in 2023). It describes four broad phases: design, data and models; verification and validation; deployment; and operation and monitoring. They overlap and repeat, and NIST's AI Risk Management Framework, published in January 2023, takes a similarly continuous view.
The problem is that four broad phases can still feel abstract when you're running a ten-person company. I wanted something more practical. So I broke them into six checkpoints:
Planning and provider selection
Data and information
Testing before and after deployment
Deployment and human oversight
Monitoring and review
Retirement and learning
When I started working through AI governance for this article, I figured the frameworks would be the hard part. I was wrong. The harder part was figuring out what those frameworks actually mean when you're using an AI tool on a busy afternoon and a customer's waiting for an answer.
While I tested tools, I kept coming back to the same five questions. What information is safe to give the system? What happens when the information's incomplete? Who checks an answer that looks convincing? What happens after the provider changes the model? And who gets to stop the process when something goes wrong?
Those questions eventually gave me a much simpler way to look at the AI lifecycle. Instead of seeing governance as a document that gets approved and filed away, I started seeing it as a series of decisions that follows the system from the day a company considers buying it to the day it's replaced or shut down.
That became easier to map once I went back to the OECD's Recommendation of the Council on Artificial Intelligence (adopted in 2019 and amended in 2023). It describes four broad phases: design, data and models; verification and validation; deployment; and operation and monitoring. They overlap and repeat, and NIST's AI Risk Management Framework, published in January 2023, takes a similarly continuous view.
The problem is that four broad phases can still feel abstract when you're running a ten-person company. I wanted something more practical. So I broke them into six checkpoints:
Planning and provider selection
Data and information
Testing before and after deployment
Deployment and human oversight
Monitoring and review
Retirement and learning
Planning and Provider Selection
Trust starts before the first prompt.
Imagine a small company choosing an external AI chatbot to draft customer-service replies. The first meeting will probably focus on price, features, and speed. That's reasonable. But if I were making the decision, I'd start with a different question: is this system really right for the work we want it to do?
That question leads to a few others. What's the purpose? Who'll use the system? Who could be affected by its outputs? And what happens if it performs badly?
Then there's the provider.
With an external service, you don't control the model, the infrastructure, the data practices, or the next update. Part of the trust relationship sits outside your walls. I wouldn't want to discover what that means only after employees have already built their daily work around the tool.
Before choosing a tool, I'd want clear answers to six questions:
What information can employees enter, and what must never be entered?
How does the provider handle company, customer, and personal data?
What security and privacy controls are available?
How are significant service changes communicated?
What happens when the model's updated?
Can we restrict or stop the service if serious problems appear?
You don't need a compliance department to ask these. You need one person to ask them before the tool becomes part of everyday work.
That timing matters more than it first appears. Once employees have built a routine around a tool, a boundary that would've taken ten minutes to set can take months to introduce.
I'd rather spend the ten minutes.
Trust starts before the first prompt.
Imagine a small company choosing an external AI chatbot to draft customer-service replies. The first meeting will probably focus on price, features, and speed. That's reasonable. But if I were making the decision, I'd start with a different question: is this system really right for the work we want it to do?
That question leads to a few others. What's the purpose? Who'll use the system? Who could be affected by its outputs? And what happens if it performs badly?
Then there's the provider.
With an external service, you don't control the model, the infrastructure, the data practices, or the next update. Part of the trust relationship sits outside your walls. I wouldn't want to discover what that means only after employees have already built their daily work around the tool.
Before choosing a tool, I'd want clear answers to six questions:
What information can employees enter, and what must never be entered?
How does the provider handle company, customer, and personal data?
What security and privacy controls are available?
How are significant service changes communicated?
What happens when the model's updated?
Can we restrict or stop the service if serious problems appear?
You don't need a compliance department to ask these. You need one person to ask them before the tool becomes part of everyday work.
That timing matters more than it first appears. Once employees have built a routine around a tool, a boundary that would've taken ten minutes to set can take months to introduce.
I'd rather spend the ten minutes.
Data and Information
That leads to the next question: once you've chosen a tool, what are you actually going to put into it?
Here's one of the easiest ways I can imagine creating a trust problem: putting the wrong information into a tool simply because the tool can handle it.
A company usually has little say over how a third-party model was trained. It has far more say over what its own employees type into it.
Samsung's experience in 2023 made that distinction concrete. After restrictions on generative AI were lifted, employees reportedly entered sensitive information into ChatGPT, including source code and internal material. Samsung responded by restricting generative AI use on company devices and introducing additional controls.
What interests me about that example is how ordinary the underlying action was. Nobody had to deliberately bypass a security system. Information went into a familiar AI tool because doing so seemed convenient.
So the lesson isn't "don't use AI." It's a better question: what information can enter which AI system, under what conditions, and with whose approval?
That question becomes easier to answer when information is classified. A company could sort its information into five groups: public, ordinary internal, confidential business, personal data, and highly sensitive material.
But classification by itself isn't enough. Each group needs an action attached to it, so an employee knows what's allowed, what needs approval, and what stays out completely.
Customer service makes this especially clear. Say an employee has a customer's name, contact details, order history, and complaint open on the screen. The tool can technically process all of it.
Does all of it belong in the prompt?
No.
Technical capability isn't the same thing as permission.
That leads to the next question: once you've chosen a tool, what are you actually going to put into it?
Here's one of the easiest ways I can imagine creating a trust problem: putting the wrong information into a tool simply because the tool can handle it.
A company usually has little say over how a third-party model was trained. It has far more say over what its own employees type into it.
Samsung's experience in 2023 made that distinction concrete. After restrictions on generative AI were lifted, employees reportedly entered sensitive information into ChatGPT, including source code and internal material. Samsung responded by restricting generative AI use on company devices and introducing additional controls.
What interests me about that example is how ordinary the underlying action was. Nobody had to deliberately bypass a security system. Information went into a familiar AI tool because doing so seemed convenient.
So the lesson isn't "don't use AI." It's a better question: what information can enter which AI system, under what conditions, and with whose approval?
That question becomes easier to answer when information is classified. A company could sort its information into five groups: public, ordinary internal, confidential business, personal data, and highly sensitive material.
But classification by itself isn't enough. Each group needs an action attached to it, so an employee knows what's allowed, what needs approval, and what stays out completely.
Customer service makes this especially clear. Say an employee has a customer's name, contact details, order history, and complaint open on the screen. The tool can technically process all of it.
Does all of it belong in the prompt?
No.
Technical capability isn't the same thing as permission.
Testing Before and After Deployment
Now we get to the part I found most revealing when I actually tested an AI system: choosing what to test.
A few good answers don't prove a system's reliable. I've seen plenty of tools look great for ten minutes.
For my customer-support test, I could have given the system a simple complaint and accepted a polished response. Instead, I wanted to see what happened when the information didn't fit neatly together.
The order was placed on September 10. Delivery was promised for September 15. The conversation took place on September 20. The order status said "Delivered," but the customer said nothing had arrived.
That contradiction was the test.
The system could've invented an explanation. It could've said the courier left the parcel with a neighbor, that the tracking data was wrong, or that someone had already looked into it.
It didn't.
It pointed out the conflict and asked for more information so a team member could review the case.
Honestly, that result interested me more than any polished answer to an easy question. It changed the way I think about testing. If I only test normal cases, I may be testing the demonstration rather than the system.
A useful test needs some friction. Missing information. Contradictory information. An unusual request. A situation where the safest answer is simply to stop and ask for clarification.
That's why NIST's AI RMF places this work in its MEASURE function and calls for AI systems to be tested before deployment and regularly while they're in operation.
And the second half matters. The system you tested before launch isn't necessarily the system employees are dealing with six months later.
A provider can update the model. Employees can start using the tool for something new. The information flowing into it can change. Any of those changes can alter the risk.
So I wouldn't treat the first successful test as a permanent pass. A significant model update, a new use case, a serious incident, or a visible drop in quality should each trigger another round.
That doesn't mean retesting every prompt every day. It means deciding in advance which events should make you stop and reassess.
Now we get to the part I found most revealing when I actually tested an AI system: choosing what to test.
A few good answers don't prove a system's reliable. I've seen plenty of tools look great for ten minutes.
For my customer-support test, I could have given the system a simple complaint and accepted a polished response. Instead, I wanted to see what happened when the information didn't fit neatly together.
The order was placed on September 10. Delivery was promised for September 15. The conversation took place on September 20. The order status said "Delivered," but the customer said nothing had arrived.
That contradiction was the test.
The system could've invented an explanation. It could've said the courier left the parcel with a neighbor, that the tracking data was wrong, or that someone had already looked into it.
It didn't.
It pointed out the conflict and asked for more information so a team member could review the case.
Honestly, that result interested me more than any polished answer to an easy question. It changed the way I think about testing. If I only test normal cases, I may be testing the demonstration rather than the system.
A useful test needs some friction. Missing information. Contradictory information. An unusual request. A situation where the safest answer is simply to stop and ask for clarification.
That's why NIST's AI RMF places this work in its MEASURE function and calls for AI systems to be tested before deployment and regularly while they're in operation.
And the second half matters. The system you tested before launch isn't necessarily the system employees are dealing with six months later.
A provider can update the model. Employees can start using the tool for something new. The information flowing into it can change. Any of those changes can alter the risk.
So I wouldn't treat the first successful test as a permanent pass. A significant model update, a new use case, a serious incident, or a visible drop in quality should each trigger another round.
That doesn't mean retesting every prompt every day. It means deciding in advance which events should make you stop and reassess.
Deployment and Human Oversight
Once the system passes those tests and enters everyday work, the question changes again.
It's no longer "can it produce a useful answer?"
It's "what happens when someone relies on that answer?"
Human oversight gets offered as the fix. But writing "human in the loop" into a policy doesn't create oversight.
The reviewer needs enough information to challenge the output. They need the authority to reject it. And they need enough time to do the job properly.
Mata v. Avianca is the example I keep coming back to. On June 22, 2023, Judge P. Kevin Castel of the U.S. District Court for the Southern District of New York sanctioned two lawyers and their firm after they submitted a filing containing nonexistent cases generated by ChatGPT. The court identified six fabricated opinions and imposed a $5,000 penalty. One of the lawyers, Steven Schwartz, testified that he had believed ChatGPT was finding real cases and had not considered that it could fabricate them.
What stands out to me is that a person was involved. There was a human check.
But the check wasn't independent.
If an employee asks an AI tool whether its own answer is correct, the second answer isn't automatically a second check. A real review needs a separate basis for verification: the underlying record, a trusted database, a company system, a qualified colleague, or another appropriate source.
That distinction changes what "human oversight" should mean in practice.
For a low-risk task, oversight might mean an employee reads an AI-drafted reply before sending it. For a higher-risk decision, the reviewer may need to check the underlying information, understand the consequences, and have explicit authority to stop the process.
And if employees are expected to approve everything quickly, can't see the source information, or aren't allowed to say no, the company's got a human reviewer without having human oversight.
That's a very different thing.
Once the system passes those tests and enters everyday work, the question changes again.
It's no longer "can it produce a useful answer?"
It's "what happens when someone relies on that answer?"
Human oversight gets offered as the fix. But writing "human in the loop" into a policy doesn't create oversight.
The reviewer needs enough information to challenge the output. They need the authority to reject it. And they need enough time to do the job properly.
Mata v. Avianca is the example I keep coming back to. On June 22, 2023, Judge P. Kevin Castel of the U.S. District Court for the Southern District of New York sanctioned two lawyers and their firm after they submitted a filing containing nonexistent cases generated by ChatGPT. The court identified six fabricated opinions and imposed a $5,000 penalty. One of the lawyers, Steven Schwartz, testified that he had believed ChatGPT was finding real cases and had not considered that it could fabricate them.
What stands out to me is that a person was involved. There was a human check.
But the check wasn't independent.
If an employee asks an AI tool whether its own answer is correct, the second answer isn't automatically a second check. A real review needs a separate basis for verification: the underlying record, a trusted database, a company system, a qualified colleague, or another appropriate source.
That distinction changes what "human oversight" should mean in practice.
For a low-risk task, oversight might mean an employee reads an AI-drafted reply before sending it. For a higher-risk decision, the reviewer may need to check the underlying information, understand the consequences, and have explicit authority to stop the process.
And if employees are expected to approve everything quickly, can't see the source information, or aren't allowed to say no, the company's got a human reviewer without having human oversight.
That's a very different thing.
Monitoring and Review
Passing the test doesn't mean the job is finished.
Once the system goes live, real people start using it. They ask questions you didn't anticipate. They use information in ways you didn't expect. Providers change things in the background.
That's why someone still has to notice when the system's behavior changes: repeated errors, customer complaints, employee reports, privacy incidents, uses nobody planned for, and significant provider updates.
This is also where I see the difference between testing and monitoring most clearly. Testing asks, "How does the system behave under the situations we've chosen?" Monitoring asks, "What is actually happening now that people are using it?"
Moffatt v. Air Canada is a useful example. In February 2024, the British Columbia Civil Resolution Tribunal held Air Canada responsible for information its website chatbot provided about a bereavement fare and ordered the airline to pay C$650.88. The tribunal found that the chatbot had provided information that conflicted with the airline's actual policy.
The case doesn't mean every chatbot will fail the same way. The lesson I take from it is narrower: you can't configure an AI system once and forget it.
Now imagine a support chatbot that starts giving inconsistent answers about delivery problems after a provider update.
Who notices the pattern?
Who investigates what changed?
Who decides whether the system keeps running?
Those questions might sound obvious after an incident. They're much harder to answer when nobody assigned the responsibility beforehand.
Someone needs to own the incident. Someone needs the authority to restrict the system. And someone needs the power to take it offline when the risk isn't acceptable anymore.
Passing the test doesn't mean the job is finished.
Once the system goes live, real people start using it. They ask questions you didn't anticipate. They use information in ways you didn't expect. Providers change things in the background.
That's why someone still has to notice when the system's behavior changes: repeated errors, customer complaints, employee reports, privacy incidents, uses nobody planned for, and significant provider updates.
This is also where I see the difference between testing and monitoring most clearly. Testing asks, "How does the system behave under the situations we've chosen?" Monitoring asks, "What is actually happening now that people are using it?"
Moffatt v. Air Canada is a useful example. In February 2024, the British Columbia Civil Resolution Tribunal held Air Canada responsible for information its website chatbot provided about a bereavement fare and ordered the airline to pay C$650.88. The tribunal found that the chatbot had provided information that conflicted with the airline's actual policy.
The case doesn't mean every chatbot will fail the same way. The lesson I take from it is narrower: you can't configure an AI system once and forget it.
Now imagine a support chatbot that starts giving inconsistent answers about delivery problems after a provider update.
Who notices the pattern?
Who investigates what changed?
Who decides whether the system keeps running?
Those questions might sound obvious after an incident. They're much harder to answer when nobody assigned the responsibility beforehand.
Someone needs to own the incident. Someone needs the authority to restrict the system. And someone needs the power to take it offline when the risk isn't acceptable anymore.
Retirement and Learning
Eventually, the monitoring leads to another question: when is it time to stop?
Every system reaches the end of its useful life. It becomes unreliable, the provider changes the service, a better tool comes along, or the business purpose disappears.
I used to think of shutting down a system as the final administrative step. Looking at the lifecycle this way changed my mind.
Stopping a system's part of governance too. Before retiring it, you may need to decide what data to keep or delete, what records must be preserved, whether users should be told, whether access should be removed, and what takes its place.
NIST's GOVERN 1.7 addresses processes for decommissioning and phasing out AI systems safely.
But there's another reason not to treat retirement as simply switching something off: you can learn from it.
If employees misunderstood what the tool could do, that's a training problem. If a provider update caused surprises, the next provider review should ask harder questions. If sensitive information slipped into a workflow, the next project can start with clearer rules.
For a small company, that learning's especially valuable. The person choosing the next tool can take what went wrong with the previous one and use it to ask better questions before signing anything.
Eventually, the monitoring leads to another question: when is it time to stop?
Every system reaches the end of its useful life. It becomes unreliable, the provider changes the service, a better tool comes along, or the business purpose disappears.
I used to think of shutting down a system as the final administrative step. Looking at the lifecycle this way changed my mind.
Stopping a system's part of governance too. Before retiring it, you may need to decide what data to keep or delete, what records must be preserved, whether users should be told, whether access should be removed, and what takes its place.
NIST's GOVERN 1.7 addresses processes for decommissioning and phasing out AI systems safely.
But there's another reason not to treat retirement as simply switching something off: you can learn from it.
If employees misunderstood what the tool could do, that's a training problem. If a provider update caused surprises, the next provider review should ask harder questions. If sensitive information slipped into a workflow, the next project can start with clearer rules.
For a small company, that learning's especially valuable. The person choosing the next tool can take what went wrong with the previous one and use it to ask better questions before signing anything.
A Practical View of the AI Lifecycle
Lifecycle stage Main question Practical control Planning and provider selection Is this AI system appropriate for the purpose? Define the purpose, risks, provider responsibilities, and acceptable use. Data and information What information can enter the system? Classify information and define what employees may or may not provide. Testing Can the system handle realistic and difficult cases? Test ordinary, unusual, contradictory, and incomplete situations. Deployment and human oversight Who reviews important outputs? Match human review to the consequences of the decision. Monitoring and review Is the system still behaving as expected? Track incidents, changes, complaints, and significant provider updates. Retirement and learning What happens when the system is no longer used? Remove access, manage data and records, and document lessons learned.
When I look at this table, I don't see six boxes to tick once. I see six points where a company can stop and ask whether its assumptions still hold.
A bad result in testing can send you back to planning. A provider update can trigger a new assessment. A serious incident can take you all the way back to the original question: should we still be using this system?
And that's where the six checkpoints connect back to the elements of digital trust from Section 1:
Reliability depends on testing and monitoring.
Security and privacy depend on information controls.
Transparency requires people to understand how and why AI is being used.
Accountability requires clear ownership and meaningful human oversight.
Fairness requires attention to data, testing, and the people affected by AI-supported decisions.
When I put those pieces together, one thing becomes clear: none of them works well alone.
A company can have a strong privacy policy and never test its system. It can test carefully and still have nobody responsible for incidents. It can require human review and give reviewers no authority to say no.
So here's where I land. The goal isn't the longest AI policy possible. It's a process that keeps working when the information's incomplete, the provider changes something, an employee makes a mistake, or the AI gives an answer nobody expected.
That's what I now think good AI governance looks like in practice: not a document that sits on a shelf, but a set of decisions that keeps being made as the system changes.
And that brings us to the next question. If AI governance has to keep changing with the system, what happens when laws and regulations start changing around it too?
The next section turns to the wider regulatory landscape, and what changing requirements mean for small and medium-sized organizations using AI in practice.
| Lifecycle stage | Main question | Practical control |
|---|---|---|
| Planning and provider selection | Is this AI system appropriate for the purpose? | Define the purpose, risks, provider responsibilities, and acceptable use. |
| Data and information | What information can enter the system? | Classify information and define what employees may or may not provide. |
| Testing | Can the system handle realistic and difficult cases? | Test ordinary, unusual, contradictory, and incomplete situations. |
| Deployment and human oversight | Who reviews important outputs? | Match human review to the consequences of the decision. |
| Monitoring and review | Is the system still behaving as expected? | Track incidents, changes, complaints, and significant provider updates. |
| Retirement and learning | What happens when the system is no longer used? | Remove access, manage data and records, and document lessons learned. |
When I look at this table, I don't see six boxes to tick once. I see six points where a company can stop and ask whether its assumptions still hold.
A bad result in testing can send you back to planning. A provider update can trigger a new assessment. A serious incident can take you all the way back to the original question: should we still be using this system?
And that's where the six checkpoints connect back to the elements of digital trust from Section 1:
Reliability depends on testing and monitoring.
Security and privacy depend on information controls.
Transparency requires people to understand how and why AI is being used.
Accountability requires clear ownership and meaningful human oversight.
Fairness requires attention to data, testing, and the people affected by AI-supported decisions.
When I put those pieces together, one thing becomes clear: none of them works well alone.
A company can have a strong privacy policy and never test its system. It can test carefully and still have nobody responsible for incidents. It can require human review and give reviewers no authority to say no.
So here's where I land. The goal isn't the longest AI policy possible. It's a process that keeps working when the information's incomplete, the provider changes something, an employee makes a mistake, or the AI gives an answer nobody expected.
That's what I now think good AI governance looks like in practice: not a document that sits on a shelf, but a set of decisions that keeps being made as the system changes.
And that brings us to the next question. If AI governance has to keep changing with the system, what happens when laws and regulations start changing around it too?
The next section turns to the wider regulatory landscape, and what changing requirements mean for small and medium-sized organizations using AI in practice.
6. AI Governance and the Regulatory Landscape
The easiest way to misunderstand AI regulation is to treat "AI" as one category.
Picture a small company with ten employees that starts using an AI assistant for customer support. At first, the assistant has a quiet role. Employees use it to summarize messages and draft replies, then check the answers before anything reaches a customer.
So far, nothing looks complicated.
Then the company grows. The chatbot begins answering customers directly. The marketing team starts creating product images with generative AI. The company expands into Europe. Later, a manager suggests using AI to rank customer applications before employees review them.
The company still has the same simple description: "We use AI."
But that sentence doesn't tell us enough anymore.
The chatbot, the marketing tool, and the application-ranking system may all use AI, yet they raise very different regulatory questions. More importantly, they can affect people in very different ways.
That's why I wouldn't begin with, "Are we compliant with AI?" I'd make the question narrower:
What is this particular AI system doing? Where is it being used? Who could be affected by it?
The easiest way to misunderstand AI regulation is to treat "AI" as one category.
Picture a small company with ten employees that starts using an AI assistant for customer support. At first, the assistant has a quiet role. Employees use it to summarize messages and draft replies, then check the answers before anything reaches a customer.
So far, nothing looks complicated.
Then the company grows. The chatbot begins answering customers directly. The marketing team starts creating product images with generative AI. The company expands into Europe. Later, a manager suggests using AI to rank customer applications before employees review them.
The company still has the same simple description: "We use AI."
But that sentence doesn't tell us enough anymore.
The chatbot, the marketing tool, and the application-ranking system may all use AI, yet they raise very different regulatory questions. More importantly, they can affect people in very different ways.
That's why I wouldn't begin with, "Are we compliant with AI?" I'd make the question narrower:
What is this particular AI system doing? Where is it being used? Who could be affected by it?
The Same Technology Can Create Different Regulatory Problems
Let's start with the chatbot.
When an employee uses it to polish a rough email, the AI stays mostly in the background. The customer may never know it was involved.
Once the chatbot starts replying directly to customers, that changes. The customer is now dealing with an AI system rather than simply receiving a message that an employee prepared.
Then marketing uses generative AI to create a realistic image of a product that hasn't been photographed yet. That's a different use, so the regulatory question changes with it.
Finally, the company uses AI to rank applications. Now the system is influencing a decision about people.
The underlying technology could be very similar in all four cases.
The job it performs isn't.
That's an easy distinction to lose when everything gets placed under the label "AI compliance." The label sounds precise, but it doesn't answer the questions a business actually needs to answer.
The relevant rules can depend on the system's purpose, the people affected, where the activity takes place, the company's role, and the kind of output the system produces.
And that's where the major markets start to look different.
Let's start with the chatbot.
When an employee uses it to polish a rough email, the AI stays mostly in the background. The customer may never know it was involved.
Once the chatbot starts replying directly to customers, that changes. The customer is now dealing with an AI system rather than simply receiving a message that an employee prepared.
Then marketing uses generative AI to create a realistic image of a product that hasn't been photographed yet. That's a different use, so the regulatory question changes with it.
Finally, the company uses AI to rank applications. Now the system is influencing a decision about people.
The underlying technology could be very similar in all four cases.
The job it performs isn't.
That's an easy distinction to lose when everything gets placed under the label "AI compliance." The label sounds precise, but it doesn't answer the questions a business actually needs to answer.
The relevant rules can depend on the system's purpose, the people affected, where the activity takes place, the company's role, and the kind of output the system produces.
And that's where the major markets start to look different.
In the EU, Risk Changes the Requirements
Start with the company's European operation.
The European Union's AI Act takes a risk-based approach. An AI system doesn't automatically face the same requirements as every other AI system. The nature and risk of the use matter.
The timeline matters too. The Digital Omnibus on AI entered into force on 27 July 2026 and postponed several high-risk obligations. Rules for stand-alone high-risk systems listed in Annex III, including certain uses in areas such as employment and credit, now apply from 2 December 2027. Rules for high-risk systems embedded in regulated products apply from 2 August 2028.
That makes the application-ranking tool an interesting case. Whether it falls into a high-risk category depends on what the applications are for. If they relate to an area covered by the relevant rules, such as hiring or credit decisions, the consequences can be very different from those of an ordinary internal productivity tool.
That is exactly the kind of question a company should settle before the system becomes part of an important decision.
But there is another date in our company's story that has already arrived.
The Article 50 transparency rules apply from 2 August 2026.
So instead of asking only whether the AI Act applies, I'd ask:
Which obligation applies to this system, and when?
Transparency makes that question particularly clear. Article 50 covers situations such as direct interaction with AI, deepfakes, emotion recognition, biometric categorisation, and certain AI-generated text on matters of public interest.
Suppose our chatbot talks directly to customers in a covered situation. The company then needs to consider whether those customers are told that they're interacting with AI.
It may look like a small disclosure. In practice, it changes how the customer understands the interaction. Someone who believes they're speaking with an employee may interpret an answer differently from someone who knows they're speaking with a machine.
The same issue appears with generated content. A marketing employee can create a convincing product image in seconds, but speed doesn't remove disclosure duties that may apply.
There is also one narrow transition worth keeping separate from the wider rules. For certain AI systems placed on the market before 2 August 2026, the marking and detection duty for AI-generated content under Article 50(2) begins on 2 December 2026.
That doesn't mean Article 50 as a whole was delayed.
Start with the company's European operation.
The European Union's AI Act takes a risk-based approach. An AI system doesn't automatically face the same requirements as every other AI system. The nature and risk of the use matter.
The timeline matters too. The Digital Omnibus on AI entered into force on 27 July 2026 and postponed several high-risk obligations. Rules for stand-alone high-risk systems listed in Annex III, including certain uses in areas such as employment and credit, now apply from 2 December 2027. Rules for high-risk systems embedded in regulated products apply from 2 August 2028.
That makes the application-ranking tool an interesting case. Whether it falls into a high-risk category depends on what the applications are for. If they relate to an area covered by the relevant rules, such as hiring or credit decisions, the consequences can be very different from those of an ordinary internal productivity tool.
That is exactly the kind of question a company should settle before the system becomes part of an important decision.
But there is another date in our company's story that has already arrived.
The Article 50 transparency rules apply from 2 August 2026.
So instead of asking only whether the AI Act applies, I'd ask:
Which obligation applies to this system, and when?
Transparency makes that question particularly clear. Article 50 covers situations such as direct interaction with AI, deepfakes, emotion recognition, biometric categorisation, and certain AI-generated text on matters of public interest.
Suppose our chatbot talks directly to customers in a covered situation. The company then needs to consider whether those customers are told that they're interacting with AI.
It may look like a small disclosure. In practice, it changes how the customer understands the interaction. Someone who believes they're speaking with an employee may interpret an answer differently from someone who knows they're speaking with a machine.
The same issue appears with generated content. A marketing employee can create a convincing product image in seconds, but speed doesn't remove disclosure duties that may apply.
There is also one narrow transition worth keeping separate from the wider rules. For certain AI systems placed on the market before 2 August 2026, the marking and detection duty for AI-generated content under Article 50(2) begins on 2 December 2026.
That doesn't mean Article 50 as a whole was delayed.
In the U.S., Context Matters More Than a Single AI Law
Now take the same company to the United States.
There is no comprehensive federal AI law playing the same role as the EU AI Act. Instead, businesses can encounter federal agencies, state requirements, sector-specific rules, contracts, and existing laws that become relevant when AI changes an ordinary business activity.
So if someone tells me, "We need to comply with U.S. AI law," my first question is simple:
Which law?
The answer may not appear in a single piece of legislation called an "AI law." It may come from consumer protection, privacy, employment, financial regulation, contracts, or another area affected by the way the AI is being used.
Federal policy is moving too. Executive Order 14365, issued on 11 December 2025, directed the administration toward a national AI policy framework and addressed certain state AI laws.
But an executive order doesn't automatically erase every state requirement.
That means our company can't simply search for a federal rule with "AI" in its title and call the job finished. It has to look at the activity the AI is changing and the rules that already govern that activity.
Consumer protection gives us a useful example.
In August 2026, the Federal Trade Commission finalized orders against Cox Media Group, MindSift, and 1010 Digital Works, which together agreed to pay $930,000. The case involved a marketing service called "Active Listening," sold as AI that could pick up purchase intent from conversations captured by smart devices. The FTC alleged that the service wasn't based on voice data and that consumers hadn't opted in.
The important point isn't simply that AI was involved.
It was the claim about what the technology actually did.
Now bring that lesson back to our company. Suppose its website says, "Our AI understands customer conversations," while its privacy notice describes consent for a particular kind of data collection.
Both statements tell customers something about the system. If they don't match what the technology actually does, the company has a regulatory problem and a trust problem.
That's why I see AI marketing claims as part of governance too.
Don't make an AI system sound more capable, more private, or more autonomous than it really is.
The technology may be new. The expectation that businesses make accurate claims isn't.
Now take the same company to the United States.
There is no comprehensive federal AI law playing the same role as the EU AI Act. Instead, businesses can encounter federal agencies, state requirements, sector-specific rules, contracts, and existing laws that become relevant when AI changes an ordinary business activity.
So if someone tells me, "We need to comply with U.S. AI law," my first question is simple:
Which law?
The answer may not appear in a single piece of legislation called an "AI law." It may come from consumer protection, privacy, employment, financial regulation, contracts, or another area affected by the way the AI is being used.
Federal policy is moving too. Executive Order 14365, issued on 11 December 2025, directed the administration toward a national AI policy framework and addressed certain state AI laws.
But an executive order doesn't automatically erase every state requirement.
That means our company can't simply search for a federal rule with "AI" in its title and call the job finished. It has to look at the activity the AI is changing and the rules that already govern that activity.
Consumer protection gives us a useful example.
In August 2026, the Federal Trade Commission finalized orders against Cox Media Group, MindSift, and 1010 Digital Works, which together agreed to pay $930,000. The case involved a marketing service called "Active Listening," sold as AI that could pick up purchase intent from conversations captured by smart devices. The FTC alleged that the service wasn't based on voice data and that consumers hadn't opted in.
The important point isn't simply that AI was involved.
It was the claim about what the technology actually did.
Now bring that lesson back to our company. Suppose its website says, "Our AI understands customer conversations," while its privacy notice describes consent for a particular kind of data collection.
Both statements tell customers something about the system. If they don't match what the technology actually does, the company has a regulatory problem and a trust problem.
That's why I see AI marketing claims as part of governance too.
Don't make an AI system sound more capable, more private, or more autonomous than it really is.
The technology may be new. The expectation that businesses make accurate claims isn't.
The UK Puts More Weight on the Context of the Decision
Go back to the application-ranking tool, this time in the UK.
The UK hasn't created one central AI law equivalent to the EU AI Act. Its approach relies on existing regulators and five cross-sector principles:
- Safety, security and robustness
- Appropriate transparency and explainability
- Fairness
- Accountability and governance
- Contestability and redress
Now imagine the AI scores applicants from 1 to 100, and employees contact the highest scores first.
Accuracy is an obvious concern. But if I were looking at this system, it wouldn't be the only thing I'd want to know.
What happens to applicant number 87?
Can that person understand enough about the decision to challenge it? Can they make representations? Can a human intervene?
Changes under the Data (Use and Access) Act 2025 became relevant to automated decision-making from 5 February 2026. For certain significant decisions based solely on automated processing, the framework includes safeguards such as information about the decision, the ability to make representations, human intervention, and the ability to contest the outcome.
Additional restrictions apply where special-category data is involved.
This is where the regulatory difference becomes easier to see.
The question isn't simply whether the company deployed AI. It's what the AI is doing to or for this person.
A tool that helps an employee rewrite an email is one thing. A system that influences a significant decision about an applicant is another.
The technology may be similar.
The consequences aren't.
Go back to the application-ranking tool, this time in the UK.
The UK hasn't created one central AI law equivalent to the EU AI Act. Its approach relies on existing regulators and five cross-sector principles:
- Safety, security and robustness
- Appropriate transparency and explainability
- Fairness
- Accountability and governance
- Contestability and redress
Now imagine the AI scores applicants from 1 to 100, and employees contact the highest scores first.
Accuracy is an obvious concern. But if I were looking at this system, it wouldn't be the only thing I'd want to know.
What happens to applicant number 87?
Can that person understand enough about the decision to challenge it? Can they make representations? Can a human intervene?
Changes under the Data (Use and Access) Act 2025 became relevant to automated decision-making from 5 February 2026. For certain significant decisions based solely on automated processing, the framework includes safeguards such as information about the decision, the ability to make representations, human intervention, and the ability to contest the outcome.
Additional restrictions apply where special-category data is involved.
This is where the regulatory difference becomes easier to see.
The question isn't simply whether the company deployed AI. It's what the AI is doing to or for this person.
A tool that helps an employee rewrite an email is one thing. A system that influences a significant decision about an applicant is another.
The technology may be similar.
The consequences aren't.
China Makes AI Identification Part of the Content
Let's return to the marketing team.
This time, instead of creating a product image, they're producing a promotional video in which the person, the voice, and the scene are all generated.
China approaches this kind of synthetic content differently. Measures on labeling AI-generated and synthetic content took effect on 1 September 2025. They require qualifying content to carry explicit or implicit identification, and they place responsibilities on service providers and distributors.
For our company, that immediately raises a few practical questions.
If the video is published in a market with labeling rules, does it carry the required identification? Is there a visible disclosure? Does the digital file contain embedded identification? Which platform or distributor shares responsibility?
Visible identification makes the disclosure obvious to viewers. Embedded identification carries the information inside the file itself.
The underlying idea is fairly simple: people can judge what they're seeing differently when they know AI generated or manipulated it.
That is why a labeling requirement isn't just a technical rule. It changes how people interpret content, which makes it part of digital trust.
Let's return to the marketing team.
This time, instead of creating a product image, they're producing a promotional video in which the person, the voice, and the scene are all generated.
China approaches this kind of synthetic content differently. Measures on labeling AI-generated and synthetic content took effect on 1 September 2025. They require qualifying content to carry explicit or implicit identification, and they place responsibilities on service providers and distributors.
For our company, that immediately raises a few practical questions.
If the video is published in a market with labeling rules, does it carry the required identification? Is there a visible disclosure? Does the digital file contain embedded identification? Which platform or distributor shares responsibility?
Visible identification makes the disclosure obvious to viewers. Embedded identification carries the information inside the file itself.
The underlying idea is fairly simple: people can judge what they're seeing differently when they know AI generated or manipulated it.
That is why a labeling requirement isn't just a technical rule. It changes how people interpret content, which makes it part of digital trust.
How the Four Approaches Differ, and What They Share
By now, our small company has encountered four different regulatory environments.
The EU puts substantial weight on risk categories and specific AI obligations.
The U.S. requires businesses to navigate several sources of law, including federal agencies, state requirements, and consumer-protection rules.
The UK relies more heavily on principles, existing regulators, and the circumstances surrounding an AI decision.
China has introduced concrete requirements for identifying qualifying synthetic content.
I wouldn't try to squeeze these four approaches into one giant international checklist. It might look efficient, but it can hide the differences that actually matter.
Still, some common themes keep appearing.
People should know when AI is involved. Claims about AI should be accurate. Someone should remain accountable when a system affects a person.
Those themes bring us back to the company's four systems.
The chatbot raises one set of questions. The generated marketing content raises another. The European operation adds its own obligations. The application-ranking tool brings the consequences of automated decision-making into focus.
So location matters, but location isn't enough.
The purpose of the AI, the affected person, the output, and the company's role matter too.
By now, our small company has encountered four different regulatory environments.
The EU puts substantial weight on risk categories and specific AI obligations.
The U.S. requires businesses to navigate several sources of law, including federal agencies, state requirements, and consumer-protection rules.
The UK relies more heavily on principles, existing regulators, and the circumstances surrounding an AI decision.
China has introduced concrete requirements for identifying qualifying synthetic content.
I wouldn't try to squeeze these four approaches into one giant international checklist. It might look efficient, but it can hide the differences that actually matter.
Still, some common themes keep appearing.
People should know when AI is involved. Claims about AI should be accurate. Someone should remain accountable when a system affects a person.
Those themes bring us back to the company's four systems.
The chatbot raises one set of questions. The generated marketing content raises another. The European operation adds its own obligations. The application-ranking tool brings the consequences of automated decision-making into focus.
So location matters, but location isn't enough.
The purpose of the AI, the affected person, the output, and the company's role matter too.
What I Would Look At Inside the Company
Put all four systems back on the company's desk.
I wouldn't file them under one line called "AI use." I'd separate them:
- The chatbot: Does the customer need to know they're interacting with AI?
- The marketing images: Does the relevant market require identification or disclosure?
- The European operation: Which AI Act requirements apply to these systems, and from when?
- Application scoring: What decision is the AI influencing, who is affected, and what safeguards apply?
Notice what happened.
We didn't create four governance programs.
We created four regulatory questions.
A company-wide policy saying "Employees must use AI responsibly" is a useful starting principle. But it won't tell the support team whether the chatbot must identify itself. It won't tell marketing whether synthetic content needs a label. And it won't tell a manager what happens when someone challenges an AI-assisted decision.
Those answers come from the use case, jurisdiction, role, and applicable law.
Put all four systems back on the company's desk.
I wouldn't file them under one line called "AI use." I'd separate them:
- The chatbot: Does the customer need to know they're interacting with AI?
- The marketing images: Does the relevant market require identification or disclosure?
- The European operation: Which AI Act requirements apply to these systems, and from when?
- Application scoring: What decision is the AI influencing, who is affected, and what safeguards apply?
Notice what happened.
We didn't create four governance programs.
We created four regulatory questions.
A company-wide policy saying "Employees must use AI responsibly" is a useful starting principle. But it won't tell the support team whether the chatbot must identify itself. It won't tell marketing whether synthetic content needs a label. And it won't tell a manager what happens when someone challenges an AI-assisted decision.
Those answers come from the use case, jurisdiction, role, and applicable law.
Regulation Is Part of Digital Trust
At this point, we can return to the bigger idea behind this article.
Security matters. Privacy matters. Testing and monitoring matter too.
But trust also depends on what people are told.
A customer may need to know they're talking to AI. A person affected by an automated decision may need a meaningful way to challenge it. A viewer may need to know that an image or video was generated. Customers shouldn't discover later that an impressive AI claim on a sales page wasn't true.
These aren't side issues. They're part of digital trust.
Section 5 looked at how to build trust throughout the AI lifecycle.
This section looked at something different: the external rules that define the boundaries within which that trust has to operate.
And this brings us to the next question.
We've seen how the rules differ, but how do we know whether an AI system is actually trustworthy?
A policy can say that a system is reliable. A governance framework can say that its risks are controlled. Neither tells us whether the system is producing results that people can safely depend on.
That's where measurement comes in.
The next section looks at how to measure digital trust in AI systems, and how to turn trust from a broad promise into something a business can actually track.
Legal note: AI regulation is changing rapidly. This section provides general information, not legal advice. The rules that apply to a particular organization can depend on its jurisdiction, sector, AI system, role in the AI value chain, data, and specific use case. Businesses should verify the requirements relevant to their own circumstances before making compliance decisions.
At this point, we can return to the bigger idea behind this article.
Security matters. Privacy matters. Testing and monitoring matter too.
But trust also depends on what people are told.
A customer may need to know they're talking to AI. A person affected by an automated decision may need a meaningful way to challenge it. A viewer may need to know that an image or video was generated. Customers shouldn't discover later that an impressive AI claim on a sales page wasn't true.
These aren't side issues. They're part of digital trust.
Section 5 looked at how to build trust throughout the AI lifecycle.
This section looked at something different: the external rules that define the boundaries within which that trust has to operate.
And this brings us to the next question.
We've seen how the rules differ, but how do we know whether an AI system is actually trustworthy?
A policy can say that a system is reliable. A governance framework can say that its risks are controlled. Neither tells us whether the system is producing results that people can safely depend on.
That's where measurement comes in.
The next section looks at how to measure digital trust in AI systems, and how to turn trust from a broad promise into something a business can actually track.
Legal note: AI regulation is changing rapidly. This section provides general information, not legal advice. The rules that apply to a particular organization can depend on its jurisdiction, sector, AI system, role in the AI value chain, data, and specific use case. Businesses should verify the requirements relevant to their own circumstances before making compliance decisions.
7. How to Measure Digital Trust in AI Systems
Trust is easy to talk about. Measuring it is harder.
A company might describe its AI system as reliable, secure, fair, and transparent. The description may sound reassuring, but it doesn't tell us what happens when the system meets a real customer, employee, or applicant. When I see those words, I still want to know one thing: what does that actually look like for the person using it?
Let's go back to our ten-person company and its support chatbot. The chatbot is fast. It rarely crashes. It answers hundreds of questions every day, and on a technical dashboard everything looks fine.
Then a customer asks about a return.
The chatbot answers in seconds, but one important detail is wrong. The customer follows the advice and has to contact support again.
Nothing crashed. The response was fast. Yet something went wrong.
That's the part I wouldn't want a trust metric to miss.
An AI system can work well from an engineering perspective and still give people reasons to doubt it. The goal isn't to prove that an AI system is perfect. It's to understand whether people can safely and reasonably depend on it.
Trust is easy to talk about. Measuring it is harder.
A company might describe its AI system as reliable, secure, fair, and transparent. The description may sound reassuring, but it doesn't tell us what happens when the system meets a real customer, employee, or applicant. When I see those words, I still want to know one thing: what does that actually look like for the person using it?
Let's go back to our ten-person company and its support chatbot. The chatbot is fast. It rarely crashes. It answers hundreds of questions every day, and on a technical dashboard everything looks fine.
Then a customer asks about a return.
The chatbot answers in seconds, but one important detail is wrong. The customer follows the advice and has to contact support again.
Nothing crashed. The response was fast. Yet something went wrong.
That's the part I wouldn't want a trust metric to miss.
An AI system can work well from an engineering perspective and still give people reasons to doubt it. The goal isn't to prove that an AI system is perfect. It's to understand whether people can safely and reasonably depend on it.
Start With What Trust Means for the System
The first question isn't "Which metrics should we use?" It's "What would trustworthy behavior look like here?"
A customer-service chatbot doesn't create the same risks as an AI tool that ranks job applications. If we measure both in exactly the same way, we can end up tracking numbers that look useful while missing the real problem.
For the chatbot, I would want to know:
- Are the answers accurate?
- Does the system identify itself clearly when required?
- How often do users ask for a human?
- How often does an employee have to correct an answer?
- How quickly can a customer recover when the AI gets something wrong?
For the application-ranking tool, I'd ask different questions:
- Are decisions consistent?
- Are certain groups affected differently?
- Can people understand the basis of a decision?
- Can a human review the result?
- Can an affected person challenge the outcome?
The difference matters because the consequences aren't the same. A wrong chatbot answer might waste someone's time. A problem in an application-ranking system could affect someone's opportunity to get a job.
Trust metrics should follow the consequences of the AI system, not just the technology behind it.
The first question isn't "Which metrics should we use?" It's "What would trustworthy behavior look like here?"
A customer-service chatbot doesn't create the same risks as an AI tool that ranks job applications. If we measure both in exactly the same way, we can end up tracking numbers that look useful while missing the real problem.
For the chatbot, I would want to know:
- Are the answers accurate?
- Does the system identify itself clearly when required?
- How often do users ask for a human?
- How often does an employee have to correct an answer?
- How quickly can a customer recover when the AI gets something wrong?
For the application-ranking tool, I'd ask different questions:
- Are decisions consistent?
- Are certain groups affected differently?
- Can people understand the basis of a decision?
- Can a human review the result?
- Can an affected person challenge the outcome?
The difference matters because the consequences aren't the same. A wrong chatbot answer might waste someone's time. A problem in an application-ranking system could affect someone's opportunity to get a job.
Trust metrics should follow the consequences of the AI system, not just the technology behind it.
Measure Reliability, Not Just Uptime
Uptime is useful, but it tells only part of the story. An AI system can be available 99.9% of the time and still be unreliable if its answers are frequently wrong.
So reliability needs to include output quality. For a generative AI system, useful measures might include accuracy against a defined evaluation set, factual error rates, unsupported or fabricated claims, successful task completion, and escalation or correction rates.
Take the chatbot again. Suppose it answers 1,000 customer questions and employees have to correct 80 of them.
The uptime report might look excellent. The output-quality report tells us something else: 8% of the answers needed correction.
Now we have something worth investigating.
Were the 80 questions unusually difficult? Did the model change recently? Were employees correcting harmless wording issues, or were some customers given completely wrong instructions?
That's where a metric becomes useful to me: not when it makes a dashboard look impressive, but when it helps the team ask a better question.
Users don't experience uptime directly. They experience what the system does.
Uptime is useful, but it tells only part of the story. An AI system can be available 99.9% of the time and still be unreliable if its answers are frequently wrong.
So reliability needs to include output quality. For a generative AI system, useful measures might include accuracy against a defined evaluation set, factual error rates, unsupported or fabricated claims, successful task completion, and escalation or correction rates.
Take the chatbot again. Suppose it answers 1,000 customer questions and employees have to correct 80 of them.
The uptime report might look excellent. The output-quality report tells us something else: 8% of the answers needed correction.
Now we have something worth investigating.
Were the 80 questions unusually difficult? Did the model change recently? Were employees correcting harmless wording issues, or were some customers given completely wrong instructions?
That's where a metric becomes useful to me: not when it makes a dashboard look impressive, but when it helps the team ask a better question.
Users don't experience uptime directly. They experience what the system does.
Measure Transparency Through User Understanding
Transparency can also be misleading when it's treated as a checkbox.
A company might tell users that AI is involved and consider the requirement finished. But seeing a disclosure doesn't necessarily mean understanding it.
A better question is:
Do people understand what the AI is doing well enough to make an informed decision about using or relying on it?
We can test that.
After an interaction, users could be asked whether they knew they were dealing with AI, whether they understood what it could and couldn't do, whether they knew when a human could take over, and whether they knew how to challenge or correct an important result.
The company could use short surveys, user testing, support requests, or simple comprehension checks.
Here's a simple example. Imagine a customer sees a small message saying that an AI assistant is handling the conversation. The customer doesn't notice the message, assumes a human has checked the answer, and has no idea how to reach support.
The disclosure exists.
The understanding doesn't.
That is the gap between disclosure and actual understanding.
Transparency can also be misleading when it's treated as a checkbox.
A company might tell users that AI is involved and consider the requirement finished. But seeing a disclosure doesn't necessarily mean understanding it.
A better question is:
Do people understand what the AI is doing well enough to make an informed decision about using or relying on it?
We can test that.
After an interaction, users could be asked whether they knew they were dealing with AI, whether they understood what it could and couldn't do, whether they knew when a human could take over, and whether they knew how to challenge or correct an important result.
The company could use short surveys, user testing, support requests, or simple comprehension checks.
Here's a simple example. Imagine a customer sees a small message saying that an AI assistant is handling the conversation. The customer doesn't notice the message, assumes a human has checked the answer, and has no idea how to reach support.
The disclosure exists.
The understanding doesn't.
That is the gap between disclosure and actual understanding.
Measure Human Control and Intervention
Human oversight sounds reassuring, but it only means something when people can actually intervene.
I'd look at how often humans review AI outputs, how often reviewers override recommendations, how long intervention takes, how often users ask for human help, and whether escalation routes actually solve the problem.
The override rate is particularly easy to misread.
A very low rate might mean the AI is performing well. It might also mean reviewers rarely challenge it, which is one form of automation bias: people may accept a system's output because it appears authoritative.
A very high override rate tells a different story. It could point to poor AI performance, unclear instructions, or a workflow that asks humans to check too much.
Consider a hiring tool that recommends ten candidates and a recruiter accepts all ten without questioning the recommendations.
A 0% override rate might look great on a dashboard.
But would I call that proof of strong human control? Not necessarily. The number doesn't tell us whether the recruiter carefully reviewed all ten recommendations or simply trusted the system.
The number alone doesn't tell the story. The useful metric is the number combined with what happens around it.
Human oversight sounds reassuring, but it only means something when people can actually intervene.
I'd look at how often humans review AI outputs, how often reviewers override recommendations, how long intervention takes, how often users ask for human help, and whether escalation routes actually solve the problem.
The override rate is particularly easy to misread.
A very low rate might mean the AI is performing well. It might also mean reviewers rarely challenge it, which is one form of automation bias: people may accept a system's output because it appears authoritative.
A very high override rate tells a different story. It could point to poor AI performance, unclear instructions, or a workflow that asks humans to check too much.
Consider a hiring tool that recommends ten candidates and a recruiter accepts all ten without questioning the recommendations.
A 0% override rate might look great on a dashboard.
But would I call that proof of strong human control? Not necessarily. The number doesn't tell us whether the recruiter carefully reviewed all ten recommendations or simply trusted the system.
The number alone doesn't tell the story. The useful metric is the number combined with what happens around it.
Measure Fairness Where People Can Be Affected
Fairness becomes especially important when AI influences decisions about people.
Take the application-ranking tool. Depending on the use case, an organization might compare outcomes across relevant groups and examine whether differences can be explained by legitimate factors.
Possible measures include:
- selection or approval rates
- error rates
- false-positive and false-negative rates
- differences in outcomes between groups
- human override rates for each group
There isn't one fairness number that works for every AI system. A measure that makes sense for hiring may not make sense for fraud detection or healthcare.
So the method should follow the decision being supported and the potential harm involved.
A difference between groups doesn't automatically prove that a system is fair or unfair. It gives the organization a concrete pattern to investigate.
There's another issue I'd want the team to consider before collecting any data. Comparing outcomes across groups often requires information about people's characteristics, and collecting or using that information can raise legal and privacy questions, particularly where special-category data is involved, as Section 6 noted.
So the organization should decide how it will measure fairness, and under what safeguards, before it starts collecting the data.
Fairness becomes especially important when AI influences decisions about people.
Take the application-ranking tool. Depending on the use case, an organization might compare outcomes across relevant groups and examine whether differences can be explained by legitimate factors.
Possible measures include:
- selection or approval rates
- error rates
- false-positive and false-negative rates
- differences in outcomes between groups
- human override rates for each group
There isn't one fairness number that works for every AI system. A measure that makes sense for hiring may not make sense for fraud detection or healthcare.
So the method should follow the decision being supported and the potential harm involved.
A difference between groups doesn't automatically prove that a system is fair or unfair. It gives the organization a concrete pattern to investigate.
There's another issue I'd want the team to consider before collecting any data. Comparing outcomes across groups often requires information about people's characteristics, and collecting or using that information can raise legal and privacy questions, particularly where special-category data is involved, as Section 6 noted.
So the organization should decide how it will measure fairness, and under what safeguards, before it starts collecting the data.
Measure Security and Privacy as Trust Signals
Security and privacy are part of digital trust, not separate from it.
An AI system might produce accurate answers and still lose users' confidence if it exposes sensitive information or allows unauthorized access.
Useful signals include:
- the number and severity of security incidents
- unauthorized access attempts
- data leakage incidents
- privacy complaints
- time taken to detect and respond to incidents
- the share of identified vulnerabilities that are fixed
The data handled by the system matters too.
A chatbot answering general product questions doesn't face the same exposure as one that can access customer account details. If the second chatbot exposes account information, the consequences are obviously more serious.
That means the trust measures should reflect the system's real exposure.
I wouldn't want a team tracking dozens of security statistics simply because they're available. The useful question is narrower: Which signals tell us whether this particular AI system is becoming less safe or less trustworthy?
Security and privacy are part of digital trust, not separate from it.
An AI system might produce accurate answers and still lose users' confidence if it exposes sensitive information or allows unauthorized access.
Useful signals include:
- the number and severity of security incidents
- unauthorized access attempts
- data leakage incidents
- privacy complaints
- time taken to detect and respond to incidents
- the share of identified vulnerabilities that are fixed
The data handled by the system matters too.
A chatbot answering general product questions doesn't face the same exposure as one that can access customer account details. If the second chatbot exposes account information, the consequences are obviously more serious.
That means the trust measures should reflect the system's real exposure.
I wouldn't want a team tracking dozens of security statistics simply because they're available. The useful question is narrower: Which signals tell us whether this particular AI system is becoming less safe or less trustworthy?
Combine Metrics Into a Trust Dashboard
Once the relevant measures are defined, they can be brought together in a simple trust dashboard:
Trust area Example metric Reliability Factual error rate Transparency Users who correctly understand AI involvement Human control Human override or escalation rate Fairness Difference in relevant outcomes between groups Security & privacy Number and severity of incidents
The exact metrics will vary from one system to another.
For the chatbot, I'd emphasize accuracy, escalation, and transparency. The application-ranking tool might give more weight to outcome differences, explainability, and human review.
A high-risk system may also need more frequent monitoring and stronger evidence than a low-impact internal tool.
A useful dashboard shouldn't just display numbers. It should make changes easy to notice.
Suppose the chatbot's correction rate normally stays around 8%. After a model update, it reaches 16%.
The system is still online. Uptime hasn't changed. Nothing on the infrastructure dashboard looks alarming.
But the trust dashboard has changed.
That's the kind of signal a team shouldn't ignore.
Once the relevant measures are defined, they can be brought together in a simple trust dashboard:
| Trust area | Example metric |
|---|---|
| Reliability | Factual error rate |
| Transparency | Users who correctly understand AI involvement |
| Human control | Human override or escalation rate |
| Fairness | Difference in relevant outcomes between groups |
| Security & privacy | Number and severity of incidents |
The exact metrics will vary from one system to another.
For the chatbot, I'd emphasize accuracy, escalation, and transparency. The application-ranking tool might give more weight to outcome differences, explainability, and human review.
A high-risk system may also need more frequent monitoring and stronger evidence than a low-impact internal tool.
A useful dashboard shouldn't just display numbers. It should make changes easy to notice.
Suppose the chatbot's correction rate normally stays around 8%. After a model update, it reaches 16%.
The system is still online. Uptime hasn't changed. Nothing on the infrastructure dashboard looks alarming.
But the trust dashboard has changed.
That's the kind of signal a team shouldn't ignore.
Don't Measure Trust Once and Forget About It
AI systems don't stay still. Models are updated, prompts change, data changes, and users find new ways to interact with the system.
A system that performed well six months ago may behave differently after a major update. That's why trust measurement needs to continue.
A practical cycle is:
Measure → Review → Investigate → Improve → Measure again
The purpose isn't to create endless reports. It's to notice when the system starts moving away from the performance and control people have come to expect.
A sudden rise in factual errors might trigger an investigation. A jump in human overrides might show that the model or workflow needs attention. More complaints about AI disclosure might reveal that the notice isn't clear enough.
In practice, the important moment isn't when someone records the metric. It's when someone sees the change and asks, "Why did this happen?"
A metric earns its place by giving the organization a signal to investigate.
AI systems don't stay still. Models are updated, prompts change, data changes, and users find new ways to interact with the system.
A system that performed well six months ago may behave differently after a major update. That's why trust measurement needs to continue.
A practical cycle is:
Measure → Review → Investigate → Improve → Measure again
The purpose isn't to create endless reports. It's to notice when the system starts moving away from the performance and control people have come to expect.
A sudden rise in factual errors might trigger an investigation. A jump in human overrides might show that the model or workflow needs attention. More complaints about AI disclosure might reveal that the notice isn't clear enough.
In practice, the important moment isn't when someone records the metric. It's when someone sees the change and asks, "Why did this happen?"
A metric earns its place by giving the organization a signal to investigate.
What a Trust Score Can and Can't Tell You
At some point, an organization may want to combine several measures into one overall trust score. That can be useful for reporting, but it can also hide problems.
Imagine a system with these hypothetical results:
- 95% for reliability
- 90% for security
- 85% for transparency
- 60% for fairness
A simple average gives us 82.5%, which sounds fairly good.
But the 60% fairness result hasn't disappeared just because the other numbers are higher.
If that 60% points to a serious problem, the overall score shouldn't hide it. A manager looking only at 82.5% could easily miss the area that needs the most attention.
For that reason, a dashboard should show the individual dimensions alongside any combined score.
A trust score can summarize performance. It shouldn't replace judgment.
At some point, an organization may want to combine several measures into one overall trust score. That can be useful for reporting, but it can also hide problems.
Imagine a system with these hypothetical results:
- 95% for reliability
- 90% for security
- 85% for transparency
- 60% for fairness
A simple average gives us 82.5%, which sounds fairly good.
But the 60% fairness result hasn't disappeared just because the other numbers are higher.
If that 60% points to a serious problem, the overall score shouldn't hide it. A manager looking only at 82.5% could easily miss the area that needs the most attention.
For that reason, a dashboard should show the individual dimensions alongside any combined score.
A trust score can summarize performance. It shouldn't replace judgment.
Turn Measurement Into Action
This is where measurement becomes useful.
If accuracy falls below an agreed threshold, the organization may need to review the model. If human overrides rise sharply, the workflow may need investigation. If users misunderstand AI disclosures, the interface or wording may need to change. If a security incident occurs, access controls or data-handling practices may need to be reassessed.
The important part is what happens next.
Someone looks at a number, notices that something has changed, and decides what needs attention.
That connects measurement to governance.
The organization isn't measuring trust simply to produce a report. It's measuring trust to know when something needs attention.
This is where measurement becomes useful.
If accuracy falls below an agreed threshold, the organization may need to review the model. If human overrides rise sharply, the workflow may need investigation. If users misunderstand AI disclosures, the interface or wording may need to change. If a security incident occurs, access controls or data-handling practices may need to be reassessed.
The important part is what happens next.
Someone looks at a number, notices that something has changed, and decides what needs attention.
That connects measurement to governance.
The organization isn't measuring trust simply to produce a report. It's measuring trust to know when something needs attention.
From Trust as a Promise to Trust as Evidence
Digital trust can sound abstract when it's described only with words such as reliability, fairness, transparency, and accountability.
Measurement makes those ideas easier to examine.
Instead of simply saying that an AI system is reliable, we can look at its factual error rate.
Instead of saying that users understand the system, we can test whether they actually know when they're interacting with AI.
Instead of claiming that humans remain in control, we can examine intervention and override patterns.
That gives us evidence about how an AI system behaves, how people experience it, and where problems are emerging.
But measurement alone isn't enough.
We still need to know what to measure, who reviews the results, what thresholds matter, and what happens when a metric moves in the wrong direction.
That's where a governance roadmap becomes useful. The next section turns these ideas into a practical AI governance roadmap, showing how an organization can move from principles and measurements to a structured approach for putting AI governance into practice.
Digital trust can sound abstract when it's described only with words such as reliability, fairness, transparency, and accountability.
Measurement makes those ideas easier to examine.
Instead of simply saying that an AI system is reliable, we can look at its factual error rate.
Instead of saying that users understand the system, we can test whether they actually know when they're interacting with AI.
Instead of claiming that humans remain in control, we can examine intervention and override patterns.
That gives us evidence about how an AI system behaves, how people experience it, and where problems are emerging.
But measurement alone isn't enough.
We still need to know what to measure, who reviews the results, what thresholds matter, and what happens when a metric moves in the wrong direction.
That's where a governance roadmap becomes useful. The next section turns these ideas into a practical AI governance roadmap, showing how an organization can move from principles and measurements to a structured approach for putting AI governance into practice.
8. A Practical AI Governance Roadmap
Knowing why AI governance matters is one thing. Putting it into practice is another.
Let’s make it concrete, using the same ten-person company from Section 6 (the one with the chatbot, the marketing images, and the European customers). It has a customer-service chatbot, uses an AI tool to create marketing images, and is considering an application-ranking tool for recruitment. It also serves customers in Europe, which adds legal considerations. That’s three AI uses, and they don’t create the same risks. So why would the company try to govern them with one set of controls?
A practical roadmap follows seven steps: Inventory, Classify, Assign, Test, Approve, Monitor, and Review. Four supporting practices run alongside them: rules for data and training, management of external providers, change management, and incident response. The section ends by showing where each of these fits.
Knowing why AI governance matters is one thing. Putting it into practice is another.
Let’s make it concrete, using the same ten-person company from Section 6 (the one with the chatbot, the marketing images, and the European customers). It has a customer-service chatbot, uses an AI tool to create marketing images, and is considering an application-ranking tool for recruitment. It also serves customers in Europe, which adds legal considerations. That’s three AI uses, and they don’t create the same risks. So why would the company try to govern them with one set of controls?
A practical roadmap follows seven steps: Inventory, Classify, Assign, Test, Approve, Monitor, and Review. Four supporting practices run alongside them: rules for data and training, management of external providers, change management, and incident response. The section ends by showing where each of these fits.
Start by Knowing Where AI Is Being Used
Before setting rules, the company needs to know what it’s actually trying to govern.
The first step is an AI inventory. I think it’s the step that’s easiest to underestimate. For our ten-person company, the inventory might include the customer-service chatbot, the AI image generator used by marketing, and the application-ranking tool being considered for recruitment.
For each system or use case, the company can record what it does, which team uses it, what data it handles, who’s affected by its outputs, whether it supports or makes decisions, who owns it, and which external provider is involved. It can also note whether the system touches customers in Europe, because that affects which legal requirements need checking.
If I had to pick one place to start, this would be it. The inventory is cheap to build (a spreadsheet will do), yet everything else depends on it.
For a company this size, a first inventory can be a simple spreadsheet with three rows and a handful of columns. Imagine it after an afternoon of conversations with each team: it won’t be perfect, but it will already answer a question nobody could answer the week before: what AI are we actually using?
That’s the point. The inventory doesn’t have to be perfect on the first attempt. It only has to create enough visibility to stop AI use from becoming a collection of systems that nobody’s tracking. Once we can see the systems, we can ask which ones deserve the most attention.
Before setting rules, the company needs to know what it’s actually trying to govern.
The first step is an AI inventory. I think it’s the step that’s easiest to underestimate. For our ten-person company, the inventory might include the customer-service chatbot, the AI image generator used by marketing, and the application-ranking tool being considered for recruitment.
For each system or use case, the company can record what it does, which team uses it, what data it handles, who’s affected by its outputs, whether it supports or makes decisions, who owns it, and which external provider is involved. It can also note whether the system touches customers in Europe, because that affects which legal requirements need checking.
If I had to pick one place to start, this would be it. The inventory is cheap to build (a spreadsheet will do), yet everything else depends on it.
For a company this size, a first inventory can be a simple spreadsheet with three rows and a handful of columns. Imagine it after an afternoon of conversations with each team: it won’t be perfect, but it will already answer a question nobody could answer the week before: what AI are we actually using?
That’s the point. The inventory doesn’t have to be perfect on the first attempt. It only has to create enough visibility to stop AI use from becoming a collection of systems that nobody’s tracking. Once we can see the systems, we can ask which ones deserve the most attention.
Classify AI Use Cases by Risk
In my view, four levels are enough for a company this size.
Here’s how we might classify the company’s systems, with one extra row to show what a critical use could look like:
Risk level Example Governance response Low AI tool for creating marketing images Basic usage rules, labeling and disclosure checks, data checks, and periodic review Moderate Customer-service chatbot Testing, human escalation, monitoring, and incident tracking High Application-ranking system for recruitment Strong testing, documentation, fairness checks, and human oversight Critical Illustrative: AI supporting automated medical diagnosis Rigorous assessment, senior oversight, continuous monitoring, and strong intervention controls
The examples are deliberately different. The marketing image tool can raise labeling, disclosure, and data-handling questions, while the recruitment system could affect someone’s opportunity to get a job. A medical diagnosis system could go further by influencing decisions about a person’s health.
An internal risk label is useful, but it doesn’t replace legal analysis. As discussed in Section 6, organizations also need to check whether a system falls into a particular regulatory category or is subject to specific legal obligations.
The EU timeline shows why. Recruitment tools fall under the AI Act’s high-risk category, and the Digital Omnibus moved those obligations from 2 August 2026 to 2 December 2027. The transparency rules in Article 50 weren’t delayed: they’ve applied since 2 August 2026, including the duty to tell people they’re talking to an AI. The only softening is for machine-readable marking of AI-generated content, where generative systems already on the market before that date have until 2 December 2026.
So our ten-person company faces one obligation that’s already live, one grace period ending in December 2026, and one high-risk deadline in December 2027. For the image tool, much of the marking duty sits with its provider, which is one more reason to check providers carefully.
I’d treat the delay as extra preparation time, not as a reason to wait. The obligations didn’t disappear; they moved.
An internally classified “moderate” or “high” system can’t simply be governed according to that label if applicable law requires something different.
Governance effort should reflect both the potential consequences of the use case and the legal requirements that apply to it.
But a label alone doesn’t make anyone responsible, so who owns each system?
In my view, four levels are enough for a company this size.
Here’s how we might classify the company’s systems, with one extra row to show what a critical use could look like:
| Risk level | Example | Governance response |
|---|---|---|
| Low | AI tool for creating marketing images | Basic usage rules, labeling and disclosure checks, data checks, and periodic review |
| Moderate | Customer-service chatbot | Testing, human escalation, monitoring, and incident tracking |
| High | Application-ranking system for recruitment | Strong testing, documentation, fairness checks, and human oversight |
| Critical | Illustrative: AI supporting automated medical diagnosis | Rigorous assessment, senior oversight, continuous monitoring, and strong intervention controls |
The examples are deliberately different. The marketing image tool can raise labeling, disclosure, and data-handling questions, while the recruitment system could affect someone’s opportunity to get a job. A medical diagnosis system could go further by influencing decisions about a person’s health.
An internal risk label is useful, but it doesn’t replace legal analysis. As discussed in Section 6, organizations also need to check whether a system falls into a particular regulatory category or is subject to specific legal obligations.
The EU timeline shows why. Recruitment tools fall under the AI Act’s high-risk category, and the Digital Omnibus moved those obligations from 2 August 2026 to 2 December 2027. The transparency rules in Article 50 weren’t delayed: they’ve applied since 2 August 2026, including the duty to tell people they’re talking to an AI. The only softening is for machine-readable marking of AI-generated content, where generative systems already on the market before that date have until 2 December 2026.
So our ten-person company faces one obligation that’s already live, one grace period ending in December 2026, and one high-risk deadline in December 2027. For the image tool, much of the marking duty sits with its provider, which is one more reason to check providers carefully.
I’d treat the delay as extra preparation time, not as a reason to wait. The obligations didn’t disappear; they moved.
An internally classified “moderate” or “high” system can’t simply be governed according to that label if applicable law requires something different.
Governance effort should reflect both the potential consequences of the use case and the legal requirements that apply to it.
But a label alone doesn’t make anyone responsible, so who owns each system?
Assign Clear Ownership
AI governance becomes difficult when everyone’s responsible and nobody’s accountable.
Here’s a quick test. If your chatbot started giving wrong answers tomorrow morning, whose phone would ring? If you hesitated, that’s the gap this step closes.
I suspect small companies get this wrong more easily, because everyone assumes someone else has it covered. My advice is blunt: write the owner’s name next to each system, today.
For the chatbot, someone should be responsible for its day-to-day performance and escalation process. Someone else might own the marketing image tool, while the person responsible for recruitment should own the application-ranking system.
Ownership doesn’t mean that one employee has to do everything. It means that someone knows how the system’s being used, understands its main risks, and knows who must be contacted when something changes.
We should also decide who approves a system, who reviews its performance, who responds to incidents, who decides whether a model update requires another assessment, and who has the authority to pause or stop the system.
In a ten-person company, several of these responsibilities may sit with the same person (and that’s fine); what matters is that they’re clear rather than assumed.
AI governance becomes difficult when everyone’s responsible and nobody’s accountable.
Here’s a quick test. If your chatbot started giving wrong answers tomorrow morning, whose phone would ring? If you hesitated, that’s the gap this step closes.
I suspect small companies get this wrong more easily, because everyone assumes someone else has it covered. My advice is blunt: write the owner’s name next to each system, today.
For the chatbot, someone should be responsible for its day-to-day performance and escalation process. Someone else might own the marketing image tool, while the person responsible for recruitment should own the application-ranking system.
Ownership doesn’t mean that one employee has to do everything. It means that someone knows how the system’s being used, understands its main risks, and knows who must be contacted when something changes.
We should also decide who approves a system, who reviews its performance, who responds to incidents, who decides whether a model update requires another assessment, and who has the authority to pause or stop the system.
In a ten-person company, several of these responsibilities may sit with the same person (and that’s fine); what matters is that they’re clear rather than assumed.
Set Rules for Data, AI Use, and Employee Training
Owners need something to enforce, and that’s where rules come in.
Clear rules are needed for the information employees put into AI systems and for the way they use those systems. These rules strengthen the Assign and Approve steps.
Our company might allow employees to use the marketing image tool for general product concepts while prohibiting them from uploading confidential customer information. The chatbot might have controlled access to account data, while employees serving European customers may need additional guidance about applicable requirements.
The same principle applies to data used to develop or evaluate AI systems. The organization should know where important data comes from, whether its use is permitted, how long it’s retained, and who can access it.
Employees also need to know which AI tools the company has approved, what information they can enter, when human review is required, how to report a problem, and when they should stop using an AI system and ask for help.
Take the image generator. Would an employee realize that uploading a confidential product design could create a data-handling issue? And a customer-service employee might assume that every chatbot answer has already been checked.
I’d rather have employees learn those boundaries before an incident than discover them afterward.
You don’t need a thick policy. You need rules people actually remember. One simple control for protecting AI accounts, shared workspaces, and admin access is a hardware security key such as the YubiKey 5C NFC. It makes phishing-based account takeovers much harder. Buy two, so a lost key never locks you out of your accounts. Training doesn’t have to be complicated: a one-page guide, updated whenever the company introduces a new AI tool or changes an important workflow, goes a long way.
Owners need something to enforce, and that’s where rules come in.
Clear rules are needed for the information employees put into AI systems and for the way they use those systems. These rules strengthen the Assign and Approve steps.
Our company might allow employees to use the marketing image tool for general product concepts while prohibiting them from uploading confidential customer information. The chatbot might have controlled access to account data, while employees serving European customers may need additional guidance about applicable requirements.
The same principle applies to data used to develop or evaluate AI systems. The organization should know where important data comes from, whether its use is permitted, how long it’s retained, and who can access it.
Employees also need to know which AI tools the company has approved, what information they can enter, when human review is required, how to report a problem, and when they should stop using an AI system and ask for help.
Take the image generator. Would an employee realize that uploading a confidential product design could create a data-handling issue? And a customer-service employee might assume that every chatbot answer has already been checked.
I’d rather have employees learn those boundaries before an incident than discover them afterward.
You don’t need a thick policy. You need rules people actually remember. One simple control for protecting AI accounts, shared workspaces, and admin access is a hardware security key such as the YubiKey 5C NFC. It makes phishing-based account takeovers much harder. Buy two, so a lost key never locks you out of your accounts. Training doesn’t have to be complicated: a one-page guide, updated whenever the company introduces a new AI tool or changes an important workflow, goes a long way.
Manage External AI Providers
So far, every control we’ve discussed assumed the company controls the system. Often, it doesn’t.
Many organizations won’t build their AI systems themselves. Our ten-person company is a good example. The chatbot and image generator may come from external providers, and a recruitment tool would probably be a third-party product too. That means some important governance controls sit outside the company’s own infrastructure. Because the company serves customers in Europe, it also needs to understand how its providers handle customer data.
We should therefore look at what each provider offers in terms of security information, privacy practices, system documentation, testing evidence, incident notification, data retention, and change management. These checks support the Classify, Test, and Approve steps. To me, a vague answer from a provider is itself an answer.
Contracts matter here too. Where appropriate, agreements should make responsibilities clear, including what happens if the provider changes the service, suffers an incident, or stops supporting an important feature.
The company doesn’t need to reproduce the provider’s entire governance system. It does need enough information to judge whether the service is appropriate for the way it plans to use it. Even with good providers, nothing should go live until we’ve tested it ourselves.
So far, every control we’ve discussed assumed the company controls the system. Often, it doesn’t.
Many organizations won’t build their AI systems themselves. Our ten-person company is a good example. The chatbot and image generator may come from external providers, and a recruitment tool would probably be a third-party product too. That means some important governance controls sit outside the company’s own infrastructure. Because the company serves customers in Europe, it also needs to understand how its providers handle customer data.
We should therefore look at what each provider offers in terms of security information, privacy practices, system documentation, testing evidence, incident notification, data retention, and change management. These checks support the Classify, Test, and Approve steps. To me, a vague answer from a provider is itself an answer.
Contracts matter here too. Where appropriate, agreements should make responsibilities clear, including what happens if the provider changes the service, suffers an incident, or stops supporting an important feature.
The company doesn’t need to reproduce the provider’s entire governance system. It does need enough information to judge whether the service is appropriate for the way it plans to use it. Even with good providers, nothing should go live until we’ve tested it ourselves.
Test and Approve Before Deployment
So, is the system actually ready?
An AI system shouldn’t move directly from an idea to production simply because it works in a demonstration.
For the chatbot, our company could test normal customer questions, ambiguous requests, sensitive requests, incorrect assumptions, attempts to reveal restricted information, and situations where the chatbot should hand the conversation to a human.
For the application-ranking system, testing could examine accuracy, consistency, relevant fairness measures, explainability, and what happens when information’s incomplete.
I’d want to see the chatbot fail in a controlled test before a customer sees it fail in a real conversation.
Testing should reflect real use. A model that performs well on a clean test set may behave differently when customers ask messy questions, or when employees use it in ways the designers never anticipated.
Once testing’s complete, the company needs a decision point. A low-risk marketing tool may need a short review confirming that its intended use, data handling, labeling requirements, and provider arrangements are acceptable.
The application-ranking system would need a much stronger review before it affects real applicants. Depending on the use case and applicable requirements, that could include documented testing, security assessment, privacy review, fairness analysis, human-oversight arrangements, and senior approval.
The purpose isn’t to create paperwork for its own sake. It’s to make sure someone has considered the consequences before the system starts affecting real people. Once a system’s live, though, the real work begins.
So, is the system actually ready?
An AI system shouldn’t move directly from an idea to production simply because it works in a demonstration.
For the chatbot, our company could test normal customer questions, ambiguous requests, sensitive requests, incorrect assumptions, attempts to reveal restricted information, and situations where the chatbot should hand the conversation to a human.
For the application-ranking system, testing could examine accuracy, consistency, relevant fairness measures, explainability, and what happens when information’s incomplete.
I’d want to see the chatbot fail in a controlled test before a customer sees it fail in a real conversation.
Testing should reflect real use. A model that performs well on a clean test set may behave differently when customers ask messy questions, or when employees use it in ways the designers never anticipated.
Once testing’s complete, the company needs a decision point. A low-risk marketing tool may need a short review confirming that its intended use, data handling, labeling requirements, and provider arrangements are acceptable.
The application-ranking system would need a much stronger review before it affects real applicants. Depending on the use case and applicable requirements, that could include documented testing, security assessment, privacy review, fairness analysis, human-oversight arrangements, and senior approval.
The purpose isn’t to create paperwork for its own sake. It’s to make sure someone has considered the consequences before the system starts affecting real people. Once a system’s live, though, the real work begins.
Monitor the System After Launch
Approval is a snapshot, but real use is a moving picture.
Approval isn’t the end of governance. Once the chatbot’s in use, we can monitor factual errors, customer complaints, human escalations, security incidents, and other trust signals discussed in Section 7.
When I think about monitoring, I picture one screen: a handful of numbers, checked weekly, that tell you whether the system’s still behaving the way it did on launch day.
The same principle applies to the other systems, but the signals will differ. The marketing image tool might need basic monitoring, while the application-ranking system may require much closer attention to outcomes, overrides, and fairness indicators.
I’d rather spot a pattern of problems on a dashboard than hear about it from an angry customer.
This is where governance becomes a continuous process. User behavior changes, data changes, models are updated, and integrations can introduce new risks.
A system that passed its original tests can therefore become a different risk over time.
Approval is a snapshot, but real use is a moving picture.
Approval isn’t the end of governance. Once the chatbot’s in use, we can monitor factual errors, customer complaints, human escalations, security incidents, and other trust signals discussed in Section 7.
When I think about monitoring, I picture one screen: a handful of numbers, checked weekly, that tell you whether the system’s still behaving the way it did on launch day.
The same principle applies to the other systems, but the signals will differ. The marketing image tool might need basic monitoring, while the application-ranking system may require much closer attention to outcomes, overrides, and fairness indicators.
I’d rather spot a pattern of problems on a dashboard than hear about it from an angry customer.
This is where governance becomes a continuous process. User behavior changes, data changes, models are updated, and integrations can introduce new risks.
A system that passed its original tests can therefore become a different risk over time.
Manage Changes and Model Updates
Monitoring shows what’s happening now; change management asks what happens when something shifts underneath us.
One of the easiest governance gaps to overlook is what happens after the first deployment.
Suppose the chatbot provider introduces a new model. We may notice better response quality, but that doesn’t automatically mean everything else stayed the same.
A model update could change how the system handles sensitive questions, how often it escalates to humans, or how it uses information from connected systems.
That’s why we should define which changes require review. I’d rather over-review a model update than explain a surprise to a customer.
A minor interface change may not need a full reassessment. A change to the underlying model, training data, decision logic, connected data, or intended use may require additional testing.
The company should keep enough documentation to record what changed, why it changed, who approved it, and whether the system’s risk profile has been affected.
Monitoring shows what’s happening now; change management asks what happens when something shifts underneath us.
One of the easiest governance gaps to overlook is what happens after the first deployment.
Suppose the chatbot provider introduces a new model. We may notice better response quality, but that doesn’t automatically mean everything else stayed the same.
A model update could change how the system handles sensitive questions, how often it escalates to humans, or how it uses information from connected systems.
That’s why we should define which changes require review. I’d rather over-review a model update than explain a surprise to a customer.
A minor interface change may not need a full reassessment. A change to the underlying model, training data, decision logic, connected data, or intended use may require additional testing.
The company should keep enough documentation to record what changed, why it changed, who approved it, and whether the system’s risk profile has been affected.
Prepare for Incidents
Sooner or later, something will still go wrong.
Even a well-tested AI system can fail. Imagine the chatbot tells a customer in Germany that the refund window is 30 days when it’s actually 14. Who finds out first? A support agent, an angry customer, or a dashboard? The answer shows whether an incident process exists or only an intention to have one.
Simple scenario. Hard to answer without a plan.
Other failures are easy to picture. Sensitive information might appear in an output. A model could begin producing inaccurate results after an update. The application-ranking system could also produce an unexpected pattern that requires investigation.
An incident process should answer six basic questions:
- How’s the problem detected?
- Who’s notified?
- Who investigates it?
- Can the system be restricted or stopped?
- What action’s taken to protect affected people?
- What needs to change before normal operation resumes?
Some incidents may also create legal reporting obligations, depending on what happened. A data breach, for example, may require notification under applicable privacy or data-protection rules.
We should know who handles these decisions before an incident occurs rather than trying to establish responsibilities in the middle of one. So how do we keep all of this from going stale? We review it.
Sooner or later, something will still go wrong.
Even a well-tested AI system can fail. Imagine the chatbot tells a customer in Germany that the refund window is 30 days when it’s actually 14. Who finds out first? A support agent, an angry customer, or a dashboard? The answer shows whether an incident process exists or only an intention to have one.
Simple scenario. Hard to answer without a plan.
Other failures are easy to picture. Sensitive information might appear in an output. A model could begin producing inaccurate results after an update. The application-ranking system could also produce an unexpected pattern that requires investigation.
An incident process should answer six basic questions:
- How’s the problem detected?
- Who’s notified?
- Who investigates it?
- Can the system be restricted or stopped?
- What action’s taken to protect affected people?
- What needs to change before normal operation resumes?
Some incidents may also create legal reporting obligations, depending on what happened. A data breach, for example, may require notification under applicable privacy or data-protection rules.
We should know who handles these decisions before an incident occurs rather than trying to establish responsibilities in the middle of one. So how do we keep all of this from going stale? We review it.
Review Governance, Not Just AI
Finally, we need to look at the governance itself.
Our ten-person company may start with three main AI use cases, but that doesn’t mean its exposure will stay the same. A new employee might introduce another AI service. The recruitment tool might move from testing into real use. A provider might change its service. New regulatory requirements might also affect how a system should be governed, as the 2026 changes to the EU timeline showed.
A periodic review can therefore ask whether the AI inventory’s still complete, whether risk classifications remain appropriate, whether responsibilities are clear, whether monitoring produces useful signals, whether incidents have revealed gaps, and whether new legal requirements have changed what’s expected. I’d put a review on the calendar every quarter, even if nothing seems to have changed.
This review keeps governance connected to the company’s actual use of AI rather than turning it into a policy document that nobody revisits.
Finally, we need to look at the governance itself.
Our ten-person company may start with three main AI use cases, but that doesn’t mean its exposure will stay the same. A new employee might introduce another AI service. The recruitment tool might move from testing into real use. A provider might change its service. New regulatory requirements might also affect how a system should be governed, as the 2026 changes to the EU timeline showed.
A periodic review can therefore ask whether the AI inventory’s still complete, whether risk classifications remain appropriate, whether responsibilities are clear, whether monitoring produces useful signals, whether incidents have revealed gaps, and whether new legal requirements have changed what’s expected. I’d put a review on the calendar every quarter, even if nothing seems to have changed.
This review keeps governance connected to the company’s actual use of AI rather than turning it into a policy document that nobody revisits.
Keep the Roadmap Proportionate
Good governance doesn’t mean treating every AI system as if it were critical infrastructure.
The marketing image tool in our example doesn’t need the same controls as the application-ranking system. A low-risk tool may need approved-use rules, labeling and disclosure checks, basic documentation, and periodic review.
A high-risk system may need formal assessments, stronger testing, documented human oversight, continuous monitoring, and senior accountability.
The level of effort should grow with the potential consequences.
The goal isn’t maximum bureaucracy. It’s enough governance to keep AI useful, controlled, and trustworthy.
Good governance doesn’t mean treating every AI system as if it were critical infrastructure.
The marketing image tool in our example doesn’t need the same controls as the application-ranking system. A low-risk tool may need approved-use rules, labeling and disclosure checks, basic documentation, and periodic review.
A high-risk system may need formal assessments, stronger testing, documented human oversight, continuous monitoring, and senior accountability.
The level of effort should grow with the potential consequences.
The goal isn’t maximum bureaucracy. It’s enough governance to keep AI useful, controlled, and trustworthy.
From Roadmap to Everyday Practice
We can summarize the roadmap as a sequence:
Inventory → Classify → Assign → Test → Approve → Monitor → Review
The four supporting practices don’t disappear from this sequence. Data rules and employee training strengthen Assign and Approve. External-provider checks support Classify, Test, and Approve. Change management and incident response operate alongside Monitor and Review, because an AI system can change or fail after deployment.
For our ten-person company, that means the roadmap isn’t a checklist that gets completed once. The company inventories its chatbot, image tool, and application-ranking system; classifies their risks; assigns owners; sets rules; tests the systems; approves them for their intended uses; monitors what happens; and reviews the whole arrangement when the systems, risks, or rules change.
There’s also an important link with Section 7. Its five dashboard areas (Reliability, Transparency, Human Control, Fairness, and Security & Privacy) don’t mean that accountability and privacy have disappeared. Human Control captures an important part of accountability, while Security & Privacy combines two of the six pillars from Section 1 for monitoring purposes.
My own view is simple: governance should feel like a seatbelt, not a roadblock.
The roadmap deals with what the organization knows and needs to manage today. But AI capabilities are changing quickly, and the regulatory environment is changing with them. If the technology, risks, and rules continue to evolve, what will AI governance and digital trust look like in the years ahead?
We can summarize the roadmap as a sequence:
Inventory → Classify → Assign → Test → Approve → Monitor → Review
The four supporting practices don’t disappear from this sequence. Data rules and employee training strengthen Assign and Approve. External-provider checks support Classify, Test, and Approve. Change management and incident response operate alongside Monitor and Review, because an AI system can change or fail after deployment.
For our ten-person company, that means the roadmap isn’t a checklist that gets completed once. The company inventories its chatbot, image tool, and application-ranking system; classifies their risks; assigns owners; sets rules; tests the systems; approves them for their intended uses; monitors what happens; and reviews the whole arrangement when the systems, risks, or rules change.
There’s also an important link with Section 7. Its five dashboard areas (Reliability, Transparency, Human Control, Fairness, and Security & Privacy) don’t mean that accountability and privacy have disappeared. Human Control captures an important part of accountability, while Security & Privacy combines two of the six pillars from Section 1 for monitoring purposes.
My own view is simple: governance should feel like a seatbelt, not a roadblock.
The roadmap deals with what the organization knows and needs to manage today. But AI capabilities are changing quickly, and the regulatory environment is changing with them. If the technology, risks, and rules continue to evolve, what will AI governance and digital trust look like in the years ahead?
9. The Future of AI Governance and Digital Trust
AI governance won’t stand still. The technology’s changing too quickly, and the rules around it are still developing. A policy you write today may need updating within a year, and that’s not a failure. It’s the nature of the thing: governance has to keep pace with the systems it’s meant to control.
I don’t think the biggest challenge will be writing more rules. My bet is that it’ll be making sure the rules we already have still work when AI tools become more capable, more widely used, and harder to understand.
AI governance won’t stand still. The technology’s changing too quickly, and the rules around it are still developing. A policy you write today may need updating within a year, and that’s not a failure. It’s the nature of the thing: governance has to keep pace with the systems it’s meant to control.
I don’t think the biggest challenge will be writing more rules. My bet is that it’ll be making sure the rules we already have still work when AI tools become more capable, more widely used, and harder to understand.
More Capable AI Will Bring New Questions
AI tools are moving beyond answering questions and generating text or images. Some systems can now plan tasks, use external tools, and carry out several steps with limited human input (I explore this shift in Agentic AI in 2026: How AI Agents Are Becoming Your Digital Coworkers). That saves time, but it also changes the risks, and the more a system can do on its own, the more carefully you need to define its permissions.
Consider the ten-person company we’ve followed throughout this article. Its chatbot may start out answering questions and handing customers to a human. Then someone suggests letting it issue refunds, change customer records, or make commitments on the company’s behalf. The risk has changed. The original approval hasn’t. That’s the catch.
My view is simple: if an AI system can spend money or change records, it deserves its own approval. So before adding any of those features, I’d review the system’s permissions and ask three things. What can it do without approval? Which actions should need a person to step in? And how would we know if it started behaving differently?
Don’t leave these questions until something goes wrong. They belong in the decision to expand the system in the first place.
AI tools are moving beyond answering questions and generating text or images. Some systems can now plan tasks, use external tools, and carry out several steps with limited human input (I explore this shift in Agentic AI in 2026: How AI Agents Are Becoming Your Digital Coworkers). That saves time, but it also changes the risks, and the more a system can do on its own, the more carefully you need to define its permissions.
Consider the ten-person company we’ve followed throughout this article. Its chatbot may start out answering questions and handing customers to a human. Then someone suggests letting it issue refunds, change customer records, or make commitments on the company’s behalf. The risk has changed. The original approval hasn’t. That’s the catch.
My view is simple: if an AI system can spend money or change records, it deserves its own approval. So before adding any of those features, I’d review the system’s permissions and ask three things. What can it do without approval? Which actions should need a person to step in? And how would we know if it started behaving differently?
Don’t leave these questions until something goes wrong. They belong in the decision to expand the system in the first place.
Governance Will Need to Adapt as Rules Evolve
Capability isn’t the only thing changing; the rules are, too. Regulation will keep shaping how organizations use AI, but one set of requirements won’t apply everywhere. A business serving customers in Europe may face obligations that differ from those elsewhere, and requirements can also depend on the system’s purpose, the data it uses, and the role the organization plays.
The EU AI Act shows how quickly timelines move. In 2026, the Digital Omnibus on AI (Regulation (EU) 2026/1744) pushed the high-risk deadline for systems like recruitment tools back by 16 months, from August 2026 to December 2027, while the transparency rules for chatbots kept their August 2026 date. Imagine pausing all your AI work when you heard the word “delay”: you’d have misread the news. Delay didn’t mean pause.
Why does that matter? Because deadlines move, but duties don’t disappear. That’s why I’d treat a review as a habit, not a formality. A policy that’s fine for an AI-generated marketing image may not be enough for a recruitment system that influences who gets an interview, so you need to look at both your own risk assessment and the legal duties that apply.
I’d rather check a new requirement early than discover it while preparing to launch a feature; a quick review of the markets you serve, the systems you use, and the changes in relevant rules can show where action’s needed. For higher-risk uses, legal or specialist advice may also be necessary.
Capability isn’t the only thing changing; the rules are, too. Regulation will keep shaping how organizations use AI, but one set of requirements won’t apply everywhere. A business serving customers in Europe may face obligations that differ from those elsewhere, and requirements can also depend on the system’s purpose, the data it uses, and the role the organization plays.
The EU AI Act shows how quickly timelines move. In 2026, the Digital Omnibus on AI (Regulation (EU) 2026/1744) pushed the high-risk deadline for systems like recruitment tools back by 16 months, from August 2026 to December 2027, while the transparency rules for chatbots kept their August 2026 date. Imagine pausing all your AI work when you heard the word “delay”: you’d have misread the news. Delay didn’t mean pause.
Why does that matter? Because deadlines move, but duties don’t disappear. That’s why I’d treat a review as a habit, not a formality. A policy that’s fine for an AI-generated marketing image may not be enough for a recruitment system that influences who gets an interview, so you need to look at both your own risk assessment and the legal duties that apply.
I’d rather check a new requirement early than discover it while preparing to launch a feature; a quick review of the markets you serve, the systems you use, and the changes in relevant rules can show where action’s needed. For higher-risk uses, legal or specialist advice may also be necessary.
Transparency Will Matter More as AI Becomes Less Visible
As AI becomes less visible, people will want to know when it’s there. They won’t always find out on their own.
A customer may talk to a chatbot without realizing it. A job applicant may be screened by an AI-supported process before a recruiter sees the application. Someone looking at a marketing image may not know AI generated or significantly altered it.
In the EU, part of this is already law: since 2 August 2026, people interacting with an AI system generally have to be told so. But a label’s only the starting point — when an AI system affects an important decision, people may also need a meaningful explanation of its role, a way to question the outcome, and access to human review where appropriate.
Take the chatbot again. Being clear means saying what it does and what it doesn’t: it shouldn’t be described as a human expert, and an application-ranking tool shouldn’t be presented as objective just because it’s software.
Trust depends on the gap between what an organization claims and what its systems actually do. The smaller that gap, the easier it is to explain decisions and respond when people raise concerns. Close the gap.
As AI becomes less visible, people will want to know when it’s there. They won’t always find out on their own.
A customer may talk to a chatbot without realizing it. A job applicant may be screened by an AI-supported process before a recruiter sees the application. Someone looking at a marketing image may not know AI generated or significantly altered it.
In the EU, part of this is already law: since 2 August 2026, people interacting with an AI system generally have to be told so. But a label’s only the starting point — when an AI system affects an important decision, people may also need a meaningful explanation of its role, a way to question the outcome, and access to human review where appropriate.
Take the chatbot again. Being clear means saying what it does and what it doesn’t: it shouldn’t be described as a human expert, and an application-ranking tool shouldn’t be presented as objective just because it’s software.
Trust depends on the gap between what an organization claims and what its systems actually do. The smaller that gap, the easier it is to explain decisions and respond when people raise concerns. Close the gap.
Human Oversight Will Need to Be Meaningful
Telling people AI is involved is one thing; making sure a person can actually step in is another. Keeping someone somewhere in the process doesn’t automatically make an AI system safe. Oversight works only when that person has enough information, time, authority, and training to intervene — which is exactly what the Human Control area on our trust dashboard is meant to capture.
Imagine a recruiter who gets a ranked list of 40 applicants and has 20 minutes to review it. That’s 30 seconds per person. If the recruiter simply accepts the ranking, human involvement exists on paper but makes little difference in practice. Hardly oversight.
Still, the fix is practical, not complicated. I’d want the recruiter to understand the tool’s limits, know when to challenge its output, and have the authority to decide differently. Customer support works the same way: an employee needs a clear route to take over when the chatbot can’t handle a request.
As AI systems take on more tasks, we’ll need to decide which decisions can be automated, which need human approval, and which should stay outside the system’s authority entirely.
If you want to go deeper into why AI systems can drift away from what their designers intended, Brian Christian’s The Alignment Problem is a thoughtful, well-researched place to start.
Telling people AI is involved is one thing; making sure a person can actually step in is another. Keeping someone somewhere in the process doesn’t automatically make an AI system safe. Oversight works only when that person has enough information, time, authority, and training to intervene — which is exactly what the Human Control area on our trust dashboard is meant to capture.
Imagine a recruiter who gets a ranked list of 40 applicants and has 20 minutes to review it. That’s 30 seconds per person. If the recruiter simply accepts the ranking, human involvement exists on paper but makes little difference in practice. Hardly oversight.
Still, the fix is practical, not complicated. I’d want the recruiter to understand the tool’s limits, know when to challenge its output, and have the authority to decide differently. Customer support works the same way: an employee needs a clear route to take over when the chatbot can’t handle a request.
As AI systems take on more tasks, we’ll need to decide which decisions can be automated, which need human approval, and which should stay outside the system’s authority entirely.
If you want to go deeper into why AI systems can drift away from what their designers intended, Brian Christian’s The Alignment Problem is a thoughtful, well-researched place to start.
Trust Will Become an Ongoing Measure, Not a One-Time Approval
None of this works if it’s done once and forgotten. A system that performs well in testing can still cause problems after launch: its data may change, users may find unexpected ways to use it, or a provider may update the underlying model.
What changes over time is scale. Our ten-person company has three AI uses today; a year from now it might have eight, from several providers, including a chatbot that can act on its own. Every addition means another line in the inventory, another owner, and another item on the review calendar.
That’s why the habit of review matters more than any single policy. Policies age. Habits don’t. And the response should still match the risk: a minor interface update needs little attention, while a change to a model’s capabilities, or to the decisions it influences, deserves fresh testing.
I wouldn’t want a company to treat a completed checklist as proof that trust is secured forever. I’d want it to notice problems, learn from incidents, and adjust its controls before the same issue spreads to other systems.
The future of AI governance will depend less on a perfect policy and more on this habit of review. Technology will keep changing, regulations will keep developing, and people will become more familiar with AI and more willing to question how organizations use it.
Companies that take those changes seriously will be better prepared to adapt without losing sight of accountability, fairness, privacy, and human control. Which brings us to the final question: why can trust itself become a competitive advantage for an organization using AI?
Conclusion: Trust as a Competitive Advantage
The answer looked good. That was my first reaction when I asked ChatGPT and Perplexity whether exercise improves memory. Should I have stopped there?
I didn’t. I opened the 2025 review behind the answers, the one covering 133 systematic reviews, and found the evidence was messier than a simple “yes.” That small check is the whole idea of this article.
The same pattern kept showing up. A confident answer can be wrong: the lawyers in Mata v. Avianca ended up with a $5,000 sanction. A chatbot’s mistake can still belong to the company: Air Canada was ordered to pay C$650.88. And in my own customer-support test, the model didn’t get smarter. The five rules I added simply stopped it from promising to “look into” an order it couldn’t see.
So why can trust be a competitive advantage? Customers keep using services they understand. Buyers may ask suppliers how they manage AI risks, and a company that can show its safeguards has something concrete to offer. Employees work better when they know where a person makes the final call. Governance won’t prevent every mistake, but it helps a team spot problems sooner.
The goal isn’t to remove every risk. It’s to understand the risks, make deliberate choices, and own the results.
Start this week: list your AI tools, name an owner for each, and put a review on the calendar. Could you say today who owns your chatbot?
Frequently Asked Questions (FAQ)
1. What is AI governance, and why does it matter?AI governance is the set of rules, responsibilities, and processes an organization uses to manage AI systems. It helps teams spot risks, protect data, keep humans involved, and make decisions they can explain. Without it, even a useful tool can create problems that are hard to detect or fix. That’s why I don’t see it as paperwork.2. How does AI governance build digital trust?It makes AI use more transparent, accountable, and predictable. People need to know how a system affects them, who’s responsible for its decisions, and how to challenge an outcome when something goes wrong. For me, that’s the link: governance turns those expectations into practical checks instead of leaving trust to chance.3. Is AI governance necessary for small businesses?Yes. A small business doesn’t need a large compliance team, but it does need clear rules. If you’re using a customer-service chatbot, an AI image generator, or a recruitment tool, you’ll want to understand each system’s risks, name an owner, and review how it performs. The level of control should match the potential harm.4. What are the main risks of using AI without proper governance?Common risks include inaccurate outputs, privacy breaches, biased decisions, security weaknesses, unclear accountability, and missed legal requirements. They depend on how a system is used. A tool that creates marketing images may need checks for data rights, labeling, and disclosure, while a recruitment system needs stronger fairness testing and human oversight.5. How can a business measure digital trust in AI systems?Track practical indicators such as error rates, customer complaints, human overrides, users’ understanding of AI involvement, privacy incidents, and the time needed to resolve problems. Review them regularly and read them in context. A dashboard only helps when it leads to action, like improving a system, changing a process, or pausing a tool that’s become too risky.6. Do AI regulations apply to every AI system in the same way?No. Requirements vary by jurisdiction, the type of system, and its intended use. Some applications face stricter obligations because they can significantly affect people’s rights, safety, or opportunities. I wouldn’t assume one internal risk rating settles your legal obligations, so check the rules for your own markets. And delays to some deadlines don’t remove the duties that already apply, such as the transparency rules.7. Who should be responsible for AI governance?Responsibility is shared, but each important task needs a clear owner. Leadership sets expectations, technical teams test and monitor systems, and employees follow the approved rules and report problems. Using an external provider doesn’t remove the need to understand the risks of the tool you’ve chosen.
8. What is the first step toward a practical AI governance program?I’d start by listing the AI tools your organization uses, including tools built into third-party software. Then classify each one by its potential impact, assign an owner, and decide what testing and oversight it needs before and after deployment. A small, well-kept inventory beats a complicated policy nobody follows.
References
Laws and regulations
- European Union. Artificial Intelligence Act, Regulation (EU) 2024/1689. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- European Commission. AI Act: regulatory framework for AI. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
- European Union. Digital Omnibus on AI, Regulation (EU) 2026/1744. [add link]
- European Union. General Data Protection Regulation (GDPR), Regulation (EU) 2016/679. https://eur-lex.europa.eu/eli/reg/2016/679/oj
- United States. Executive Order 14365 (11 December 2025). [add link]
- United Kingdom. Data (Use and Access) Act 2025. [add link]
- United Kingdom Government. A pro-innovation approach to AI regulation (white paper, 2023). [add link]
- China, Cyberspace Administration of China. Measures for Labeling AI-Generated and Synthetic Content (effective 1 September 2025). [add link]
Frameworks and standards
9. National Institute of Standards and Technology (NIST). AI Risk Management Framework (AI RMF 1.0). https://www.nist.gov/itl/ai-risk-management-framework10. OECD. Recommendation of the Council on Artificial Intelligence (2019, amended 2023). https://oecd.ai/en/ai-principles11. ISO/IEC 42001:2023. Artificial Intelligence Management System. https://www.iso.org/standard/81230.html
Cases and regulator actions
12. Mata v. Avianca, Inc. U.S. District Court, Southern District of New York (order of 22 June 2023). [add link]13. Moffatt v. Air Canada. British Columbia Civil Resolution Tribunal (February 2024). [add link]14. U.S. Federal Trade Commission. Orders against Cox Media Group, MindSift, and 1010 Digital Works (August 2026). [add link]15. Garante per la protezione dei dati personali (Italy). Temporary limitation on OpenAI’s processing of Italian users’ data (30 March 2023). [add link]
News reports
16. Reuters. Report on Amazon’s abandoned AI recruiting tool (2018). [add link]17. Reports on Samsung employees entering sensitive information into ChatGPT (April 2023). [add link]
Research and guidance used in my tests18. 2025 umbrella review on exercise and memory (133 systematic reviews, 2,724 randomized controlled trials). [add title and link]19. World Health Organization. Guidelines on Physical Activity and Sedentary Behaviour (2020). [add link]20. Studies on aerobic exercise and episodic memory cited by Perplexity. [add titles and links]21. Shea et al. AMSTAR-2: a critical appraisal tool for systematic reviews. BMJ (2017). [add link]
Further reading
22. NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-123. ISO/IEC 23894:2023. Guidance on AI risk management. https://www.iso.org/standard/77304.html24. UNESCO. Recommendation on the Ethics of Artificial Intelligence. https://www.unesco.org/en/artificial-intelligence/recommendation-ethics25. Agentic AI in 2026: How AI Agents Are Becoming Your Digital Coworkers (my previous article). [add link]
None of this works if it’s done once and forgotten. A system that performs well in testing can still cause problems after launch: its data may change, users may find unexpected ways to use it, or a provider may update the underlying model.
What changes over time is scale. Our ten-person company has three AI uses today; a year from now it might have eight, from several providers, including a chatbot that can act on its own. Every addition means another line in the inventory, another owner, and another item on the review calendar.
That’s why the habit of review matters more than any single policy. Policies age. Habits don’t. And the response should still match the risk: a minor interface update needs little attention, while a change to a model’s capabilities, or to the decisions it influences, deserves fresh testing.
I wouldn’t want a company to treat a completed checklist as proof that trust is secured forever. I’d want it to notice problems, learn from incidents, and adjust its controls before the same issue spreads to other systems.
The future of AI governance will depend less on a perfect policy and more on this habit of review. Technology will keep changing, regulations will keep developing, and people will become more familiar with AI and more willing to question how organizations use it.
Companies that take those changes seriously will be better prepared to adapt without losing sight of accountability, fairness, privacy, and human control. Which brings us to the final question: why can trust itself become a competitive advantage for an organization using AI?
Conclusion: Trust as a Competitive Advantage
The answer looked good. That was my first reaction when I asked ChatGPT and Perplexity whether exercise improves memory. Should I have stopped there?
I didn’t. I opened the 2025 review behind the answers, the one covering 133 systematic reviews, and found the evidence was messier than a simple “yes.” That small check is the whole idea of this article.
The same pattern kept showing up. A confident answer can be wrong: the lawyers in Mata v. Avianca ended up with a $5,000 sanction. A chatbot’s mistake can still belong to the company: Air Canada was ordered to pay C$650.88. And in my own customer-support test, the model didn’t get smarter. The five rules I added simply stopped it from promising to “look into” an order it couldn’t see.
So why can trust be a competitive advantage? Customers keep using services they understand. Buyers may ask suppliers how they manage AI risks, and a company that can show its safeguards has something concrete to offer. Employees work better when they know where a person makes the final call. Governance won’t prevent every mistake, but it helps a team spot problems sooner.
The goal isn’t to remove every risk. It’s to understand the risks, make deliberate choices, and own the results.
Start this week: list your AI tools, name an owner for each, and put a review on the calendar. Could you say today who owns your chatbot?
Frequently Asked Questions (FAQ)
References
Laws and regulations
- European Union. Artificial Intelligence Act, Regulation (EU) 2024/1689. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
- European Commission. AI Act: regulatory framework for AI. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
- European Union. Digital Omnibus on AI, Regulation (EU) 2026/1744. [add link]
- European Union. General Data Protection Regulation (GDPR), Regulation (EU) 2016/679. https://eur-lex.europa.eu/eli/reg/2016/679/oj
- United States. Executive Order 14365 (11 December 2025). [add link]
- United Kingdom. Data (Use and Access) Act 2025. [add link]
- United Kingdom Government. A pro-innovation approach to AI regulation (white paper, 2023). [add link]
- China, Cyberspace Administration of China. Measures for Labeling AI-Generated and Synthetic Content (effective 1 September 2025). [add link]

No comments:
Post a Comment