Capability Has Boundaries
Learn to spot tasks that look easy to a chatbot but carry hidden accuracy, privacy, or consequence.
In this lesson, you will learn to:
- Classify low- and high-consequence AI uses.
- Identify missing evidence and affected people before relying on output.
Capability Has Boundaries
Distinguishes assistance from authority and shows how scope, reversibility, affected people, and evidence change the safety of a use.
A capability demo is not a guarantee
A model may answer medical, financial, legal, security, or employment questions because those patterns exist in its training. The ability to produce an answer does not show that the answer is appropriate for your situation, current in your jurisdiction, or reviewed by a qualified person. Similarly, an image model can create realistic evidence-shaped media without documenting a real event.
Before using output, ask four questions: What could happen if it is wrong? Can the action be reversed? Who else is affected? What independent evidence or expertise is available? Drafting a private birthday invitation is low consequence. Deciding whether a symptom is an emergency, accusing a person of misconduct, or changing live access permissions is not. Increase verification and human review as consequences rise.
Outputs can carry several kinds of error
A factual claim may be invented, outdated, or oversimplified. A summary may reverse who did what, omit an exception, or flatten uncertainty. Code may compile while mishandling edge cases or creating a vulnerability. A translation may change tone or legal meaning. A classification may reproduce bias from examples or perform poorly for a group that was underrepresented.
Errors also propagate. A generated citation can enter a report, be copied into another model, and appear to gain credibility through repetition. A mistaken spreadsheet formula can influence a chart and decision. Keep the source and output connected, label generated drafts during review, and test important transformations. “The AI said so” describes an origin, not an evidence standard.
Refusal and confidence are not stable safety measures
Different services, models, versions, settings, and prompts can respond differently. A refusal today does not prove the service will always refuse, and a warning banner does not prevent every unsafe output. Conversely, a model may refuse a harmless request because it misreads context. Treat these behaviors as product characteristics to observe, not promises on which to place all safety.
Your controls should survive model variability: minimize sensitive input, verify consequential claims, keep tools narrowly permissioned, require meaningful approval before external action, and retain a way to undo mistakes. Reliable use is created by the complete workflow around the model, not by assuming the model will always recognize the boundary for you.
Resources
- NIST AI 600-1 Generative AI Profile — Use NIST’s cross-sector profile to understand distinctive generative-AI risks and suggested actions.