Conflicting labels

​
Definition

The conflicting labels test checks if there are rows with identical feature values but differing labels. These are rows that are identical to the model (from an input perspective) but are labeled differently.

​
Taxonomy

  • Task types: Tabular classification, text classification.
  • Availability: and .

​
Why it matters

  • Conflicting rows can be a sign of a lack of standardization in the data labeling process.
  • Rows with conflicting labels hinder model learning. These are rows that are identical to the model (from an input perspective) but are labeled differently.

​
Test configuration examples

If you are writing a tests.json, here are a few valid configurations for the character length test: 
[
  {
    "name": "No rows with conflicting labels",
    "description": "Asserts that there are no rows with identical inputs but different labels",
    "type": "integrity",
    "subtype": "conflictingLabelRowCount",
    "thresholds": [
      {
        "insightName": "conflictingLabelRowCount",
        "insightParameters": null,
        "measurement": "conflictingLabelRowCount",
        "operator": "<=",
        "value": 0
      }
    ],
    "subpopulationFilters": null,
    "mode": "development",
    "usesValidationDataset": true, // Apply test to the validation set
    "usesTrainingDataset": false,
    "usesMlModel": false,
    "syncId": "b4dee7dc-4f15-48ca-a282-63e2c04e0689" // Some unique id
  }
]