More and more organisations arrange something around AI knowledge: a lunch presentation, a workshop, an email with tips. Well intended, and better than nothing. But ask around three months later what actually stuck, and the answer is usually sobering. Knowledge you hear once and then never use or revisit fades away. That is not a weakness of your team; it is how memory works for everyone.
If you want AI knowledge to stick, and you want to be able to show that your team has mastered the basics, assessment has to be part of the picture. But assessment can be done well or badly. This article covers what meaningful testing looks like.
Why awareness alone fades
An awareness session has a real but short-lived effect: people are more alert for a few weeks. Then everyday pressures take over. There are three reasons the effect does not last:
- There is no processing. Listening is passive. Knowledge only sticks when you have to do something with it: apply it, explain it, make a choice.
- There is no measurement. Without a test, nobody knows, including the participants themselves, what landed and what did not. People systematically overestimate what they have remembered.
- There is no repetition. Hearing something once is not learning. Without moments of return, even well-understood knowledge disappears.
Assessment does not solve this by itself, but it forces processing, it makes the gaps visible, and it creates a natural hook for repetition.
Good test questions: scenarios over trivia
The difference between a pointless and a meaningful AI test lies mainly in the type of questions. A trivia question tests whether someone can reproduce a definition: “What does the abbreviation GPAI stand for?” Fun at a pub quiz, but it says nothing about safe behaviour at work. Someone can know every definition and still paste customer data into a chatbot.
A scenario question tests what someone does in a recognisable situation. For example:
Example of a scenario question: “You want a chatbot to rewrite a complaint email. The email contains the customer’s name, the order number and a promise about compensation. What do you do?” The answer options then describe behaviours, ranging from “just paste it, it is only one email” to “first remove or replace the name, order number and promise”.
Questions like this have two big advantages. First, they test understanding rather than memory: you have to recognise the principle (do not enter traceable data) in a concrete situation. Second, they are a learning moment in themselves: someone who gets the question wrong and reads the explanation will remember that situation better than any presentation slide.
So a good AI test consists largely of scenario questions about situations that genuinely occur in your kind of work, topped up with at most a few knowledge questions about things people really must have at their fingertips.
The pass threshold: strict enough to mean something
A test without consequences is a survey. So there has to be a pass threshold, and it needs to balance two things: achievable for people who have worked through the material seriously, and strict enough that a pass actually means something.
A threshold around seventy to eighty percent is common and defensible for this kind of test. More important than the exact percentage is what happens after a fail: not punishment, but a retake. Going through the material again and retaking the test, with different or shuffled questions, is simply part of the process. The goal is for everyone to master the basics eventually, not to weed people out.
Also be honest about what you are testing: the basics. An AI literacy test does not make anyone an expert. It demonstrates that someone recognises the main risks and knows the main rules of thumb. That is exactly what you can ask of an entire organisation; expertise is something you ask only of specialists.
Refreshers: knowledge needs maintenance
Even a good test is a snapshot. Two developments make refreshers necessary. First, forgetting: even people who passed comfortably lose knowledge they do not use regularly. Second, the subject itself changes: AI tools gain new features, new risks appear, and the regulation continues to be fleshed out.
Practical forms of repetition, in increasing weight:
- Short refresher questions in between. A few scenario questions per quarter, for example in a team meeting. Takes five minutes and keeps the subject alive.
- Discuss real situations. A near-miss or a well-handled doubt from your own practice teaches more than any invented example.
- An annual check-in. A shortened test or updated module, certainly when something substantial has changed in tools or rules in the meantime.
Curious what scenario questions feel like? The free AI knowledge quiz is an accessible example, and it gives you an immediate sense of where you stand.
What a certificate does and does not prove
A certificate after a passed test is useful, provided you are honest about it. What it does show: that at a certain moment, someone worked through the material and passed a test with a set threshold. For an employer that is valuable: you can show, internally and externally, that your team has demonstrably worked on the foundations of AI literacy, and you can see who has the basics down.
What a certificate does not prove: that someone will always behave sensibly in practice, that the knowledge will still be current in two years, or that the organisation automatically complies with all regulation because of it. Anyone presenting a certificate as a legal shield is overplaying their hand. Think of it like a driving licence: it proves you had the basics down when you earned it. Driving safely is something you still have to do every day afterwards.
That is exactly why certificates and refreshers belong together: a certificate backed by a refresher rhythm says far more than a loose piece of paper from three years ago.
In summary
Testing AI knowledge meaningfully comes down to four things: scenario questions that test behaviour rather than definitions, a pass threshold that is both achievable and meaningful, a fixed refresher rhythm so knowledge does not fade, and honesty about what the result proves.
The AI literacy course is built on these principles: practical scenarios, a final test and a certificate on passing. Team licences are available via the for employers page, and if you would rather sample it first, the free module is there for you.