Can an AI feature leak private data? Sensitive information disclosure explained
Review how AI features can expose private information and where data controls, permissions and testing can reduce risk.
OWASP's Top 10 for LLM Applications (2025) lists Sensitive Information Disclosure as LLM02. We read the project's page in full on 15 November 2026, and not the references it links. This article summarises that one page. It is not legal advice; if personal data law applies to your product, ask counsel.
What counts as sensitive
The page lists personal identifiable information (PII), financial details, health records, confidential business data, security credentials and legal documents. It also says proprietary model details, such as training methods and source code, can be sensitive. The risk covers the model and the application around it. Leaks happen through the model's output: unauthorised data access, privacy violations and intellectual property breaches.
Three kinds of exposure, per the page
- PII leakage during interactions.
- Proprietary algorithm exposure. Poorly configured outputs can reveal algorithms or data. The page discusses training-data exposure, model inversion and extraction as related risks. This overview does not independently establish any incident or vulnerability identifier.
- Business data disclosure. Responses may include confidential information by accident.
Mitigations OWASP lists
- Sanitisation: scrub or mask sensitive content before it is used in training, and validate inputs.
- Access control: least privilege, and limit which external data sources the model can reach.
- Privacy techniques: federated learning and differential privacy, named as options.
- Users and transparency: tell users not to enter sensitive data, publish retention and deletion policies, and let users opt out of training use.
- Configuration: conceal the system preamble and follow guidance on security misconfiguration so errors do not leak details.
- Advanced: homomorphic encryption, tokenisation and redaction by pattern matching.
These techniques address different threat models and are not interchangeable guarantees. Concealing a prompt cannot replace access control, and specialist privacy techniques need separate design review. On system-prompt rules like "never return personal data", the page warns that such restrictions may not always be honoured and can be bypassed through prompt injection. So they are one layer, not the control.
Scenarios on the page
- A user receives another user's personal data because of weak sanitisation.
- An attacker bypasses input filters to extract sensitive data.
- Sensitive data included carelessly in training later shows up in output.
Questions for a team
- What private data can the model see at runtime, and does each user see only their own?
- What data do we send to a third-party model, and what do its terms say about retention and training? (Read the vendor's own terms; this page does not cover any vendor.)
- Do users know what not to type?
- What would a redaction step catch, and what would it miss?
Related: prompt injection explained, AI agent integration risks, secrets management basics.
Limits
One OWASP page read in full; references and MITRE ATLAS entries unread. No vendor claims, no statistics. The page names techniques such as federated learning and homomorphic encryption without detail, and we do not add any.
By Omnistu.