How to Build and Use Trainable Classifiers in Microsoft Purview

Introduction

Sensitive information types in Microsoft Purview are great at catching structured data – credit card numbers follow a predictable pattern, passport numbers have a known format. But what about documents without neat patterns? How do you automatically identify a resume versus a contract versus source code? That is where trainable classifiers come in.

They use machine learning to recognize document types by their content, structure, and language – not by matching specific character patterns. Once trained, a classifier can automatically apply sensitivity labels to every document of that type across your organization, at scale. This guide walks you through building, training, testing, and deploying your first trainable classifier.

Trainable classifiers require Microsoft 365 E5 or the E5 Compliance add-on. You need at least 200 sample documents of the target type, spread across at least 20 different contributors to provide variety. Documents from a single person teach the classifier that person’s writing style rather than the document type itself. If you need help with permissions, refer to the first-day setup guide.

Seeding Your Classifier with the Right Sample Documents

Go to Data classification and select Trainable classifiers. Click Create trainable classifier. Choose a descriptive name – you will reuse this classifier across auto-labeling policies later. The first training phase is seeding. Purview asks you to provide sample documents from SharePoint or OneDrive that are good examples of what you want to find.

Select at least 200 documents and spread them across multiple sites and authors. If you are building a resume classifier, pick real resumes from actual candidates – not templates. The classifier learns from what you feed it. After providing samples, Purview processes them and builds an initial model. This takes 24 to 48 hours.

After the initial model is built, you move to the testing phase. Purview presents you with documents from your environment and asks you to mark each one as a match or not a match. This is the most important part – the quality of your labeling directly determines classifier quality. Review at least 30 documents during this phase. Be consistent. If you label a cover letter as a resume in one instance and not in another, you confuse the model.

Continue providing feedback until the classifier achieves at least 80% precision and recall. Precision measures how many flagged documents were correct. Recall measures how many actual matching documents were found. Both matter equally. If your classifier is stuck on low accuracy, the troubleshooting guide covers the most common training mistakes and how to fix them.

Publishing and Using Your Classifier in Auto-Labeling Policies

When the classifier reaches acceptable accuracy, click Publish. It becomes available in auto-labeling policies alongside sensitive information types. To use it, go to the sensitivity label you want to apply, create a new auto-labeling policy, and select your classifier as the match condition.

You can combine classifiers with sensitive info types using AND logic. For example, apply a Highly Confidential label to any document that matches both the Contract classifier and contains at least one bank account number. This combination is far more precise than either condition alone.

After deployment, monitor the classifier in Content Explorer. Filter by the sensitivity label the auto-labeling policy applies and review the documents being labeled. If the classifier misclassifies certain documents, retrain it. Go back to Trainable classifiers, select the classifier, and provide additional feedback on misclassified items. The model improves over time.

Microsoft also provides several pre-built classifiers that are already trained – these cover resumes, source code, and offensive language. Check the pre-built list before building your own. Trainable classifiers for documents use the same technology as the classifiers used in Communication Compliance, where they detect policy-violating messages rather than document types.