Multiclass vs Multilabel Annotation: Check Label Cardinality Before Acceptance
· Data Annotation
An original buyer worksheet testing whether labels are mutually exclusive, whether empty sets are allowed, and whether exports preserve the intended meaning.
Specify whether each annotated sample must have exactly one class or may carry multiple non-exclusive labels, then test the delivered export against that rule. A correctly shaped label matrix does not prove that its label meaning matches the annotation brief. Keep an unknown or unreviewed sample distinct from a confirmed empty label set.
This guide is for annotation buyers and dataset reviewers. It concerns a delivery acceptance question, not a company ranking, a job opportunity or a claim about hiring, pay or worker eligibility.
What does the primary documentation establish?
The current scikit-learn multiclass and multioutput guide distinguishes multiclass classification from multilabel classification. It describes a multilabel indicator matrix with one column per class and positive entries identifying labels present for a sample. It permits a sample to have no positive labels in that general multilabel formulation. This software representation does not decide whether an empty set is acceptable for your annotation project.
Data Annotation Companies is independent and is not affiliated with scikit-learn. The acceptance worksheet below is our original review aid, not an official scikit-learn form or a guarantee of model performance.
Which cardinality rule should the buyer approve?
| Question | Decision to record | Delivery check |
|---|---|---|
| Task meaning | One mutually exclusive class or non-exclusive labels | Does the export preserve the intended distinction? |
| Allowed labels per sample | Minimum, maximum and any prohibited combinations | Count labels against the actual rule |
| Empty set | Allowed reviewed outcome or not allowed | Do not silently convert missing work to empty |
| Unknown state | Explicit unresolved or unreviewed representation | Keep separate from a negative label decision |
| Column or value map | Approved class identities and fixed order | Decode with the actual documented mapping |
| Exception response | Owner and correction or approved disposition | Recheck affected samples after resolution |
How can a valid-looking matrix violate the brief?
Illustrative example. A fictional buyer defines three mutually exclusive document classes: inspection record, drawing and commercial form. Each reviewed sample must receive exactly one class. A delivery uses three indicator columns in that approved order. The row for Sample A is [1, 1, 0]. It contains two positive labels and violates the stated exactly-one rule, even though it can be represented as a general multilabel row.
A separate fictional task instead labels whether a document mentions inspection, access and delivery. Those topics are expressly non-exclusive. The row [1, 1, 0] can represent inspection and access together under that different brief. The same numbers do not establish the same task meaning.
| Synthetic row | Exactly-one example | Non-exclusive example |
|---|---|---|
| [1, 0, 0] | One positive class | One positive topic |
| [1, 1, 0] | Two positives; query required | Two topics, if permitted by the brief |
| [0, 0, 0] | No class; query required | Empty set only if explicitly reviewed and allowed |
These synthetic rows contain no real data or measured quality result. The buyer still checks the actual labels, prohibited combinations and sample evidence. A cardinality pass is not a semantic accuracy score.
Why must unknown and empty remain different?
If a provider exports an unreviewed sample as all zeros, the receiver may interpret it as a reviewed decision that none of the topics applies. Define how unresolved work is represented before delivery. Retain the review status separately when needed; do not invent a missing-data rule from a convenient file format.
At acceptance, reconcile expected samples, resolved samples and explicitly open exceptions. Decode the export with its actual label map and compare representative rows with the approved brief. Preserve the supplied version and correction history. Do not repair an ambiguous row by choosing its first positive label or defaulting to all zeros without an authorized disposition.
Frequently asked questions
Does more than one positive label always mean an error?
No. It depends on the approved task and allowed label combinations.
Does an all-zero row mean the sample was reviewed?
No. That conclusion needs an explicit reviewed outcome and the project rule.
Does this check predict model performance?
No. It checks delivery meaning and cardinality; semantic accuracy and downstream evaluation are separate.
Use the company directory to research providers and ask about their actual delivery format and acceptance process. Review pilot acceptance sampling separately from this cardinality check, and use the contact page to suggest a correction to this guide.