Definition
Cleaning supplier product data is an intake procedure run at the catalog boundary: profile the incoming file, map its fields, normalize units and value vocabularies, deduplicate against existing SKUs, and quarantine whatever fails validation. The order is not arbitrary — each step assumes the one before it has already run, and quarantine is a real stop rather than a note attached to a row that loads anyway. Cleaning is done once per delivery, before a supplier's rows become catalog records.
Key points
- Cleaning belongs at the boundary. The canonical catalog should never be the place where a supplier's formatting problems are first discovered.
- Every rule you apply is written down and reused, because the same supplier will send a file shaped the same way next month.
- Quarantine is a legitimate outcome, not a failure. Holding a row back with a stated reason is a decision the process is supposed to produce.
- The durable asset is the per-supplier mapping profile, not the one cleaned file it produced.
What is the sequence for cleaning a supplier file?
- Profile the file. Count rows, list every column, sample its distinct values, and flag columns that are empty, mixed-type, or free text where a code belongs.
- Map fields to your schema. Bind each supplier column to one of your attributes, and record what you deliberately drop so the decision is not re-argued next delivery.
- Normalize units and vocabularies. Convert measurements onto one unit system and fold synonym values onto the controlled term your catalog uses.
- Deduplicate against what you carry. Match on identifiers first, then brand plus manufacturer part number, then decide field by field which record wins.
- Validate and quarantine. Run the category's required-attribute rules, hold each failing row with its reason attached, and return them to the supplier.
Common pitfalls
- Cleaning in a one-off spreadsheet, so the identical work is redone from scratch when the next file lands.
- Deduplicating on product title, which merges genuinely distinct variants while missing true duplicates that were simply named differently.
- Letting failed rows through with filler values such as "N/A" so the load count looks complete, which converts a visible intake problem into an invisible catalog one.
FAQ
Should cleaning happen before or after the data enters the catalog?
Before. Once supplier rows merge into the canonical catalog, their errors propagate into every downstream feed and report, and you lose the ability to say which supplier introduced what. A staging area between the file and the catalog is what keeps cleaning reversible.
What do you do with a supplier who will not change their file format?
Absorb the difference in the mapping layer rather than in the catalog. Build a per-supplier mapping and normalization profile, version it, and run it against every delivery. The cost is then paid once per supplier instead of once per file, and the supplier relationship stays intact.
Source
The classification that the mapping step maps supplier categories onto is exactly what the GS1 Global Product Classification provides — a shared category hierarchy other systems can classify against instead of each maintaining their own. The exchange mechanism around it is described in GS1's Global Data Synchronization Network (GDSN).