Job type A
Cleansing and standardising
Most messy data is not wrong, just inconsistent. Dates written three ways, names in capitals in one place and title case in another, postcodes with and without spaces, phone numbers with stray characters. Each inconsistency is small; together they stop files from matching and make totals unreliable.
We agree a standard for each field, write it down as a rule, and apply it to every row. The run note counts how many values each rule touched, so you can see whether the clean-up was minor or substantial.
- Typical inputs
- CSV, tab-delimited or spreadsheet exports from your own systems
- Outputs
- Standardised file, change counts, held-row sheet
- Decisions left with you
- Which format is the house standard; what to do with values that cannot be interpreted
Job type B
De-duplication and matching
Duplicates creep in through imports, re-keying and merged lists. Exact duplicates are easy. The harder cases are near matches: the same organisation with a slightly different name, or the same product under two codes.
We separate certain matches from possible ones. Certain matches are combined by the rule you choose, for example keeping the most recent record. Possible matches go into a review sheet, side by side, so a person who knows the data can decide.
- Typical inputs
- One or more lists with overlapping records
- Outputs
- Merged list, review sheet of possible matches, before and after counts
- Decisions left with you
- Which record survives a merge; whether a possible match is real
Job type C
Format conversion and reshaping
Systems rarely agree on layout. One produces a wide file with a column per month; another expects one row per month. One splits an address over five fields; another wants two. Conversion work is about mapping one shape onto another without losing meaning.
The mapping is written as a table: source column, target column, transformation. That table is versioned, so when either system changes, the update is deliberate and dated.
- Typical inputs
- Exports you are entitled to share, plus the target layout or import template
- Outputs
- Converted file, mapping table, list of fields that had no target
- Decisions left with you
- What to do with source data the target has no place for
Job type D
Reconciliation runs
Two sources that should agree give you a way to catch problems early, but only if someone compares them. A reconciliation run lines up records from both sides by an agreed key and reports three groups: matched, present on one side only, and present on both but with different values.
Totals are shown for each side so the size of any gap is clear before anyone starts searching for individual rows.
- Typical inputs
- Two exports sharing a reference such as an order number
- Outputs
- Matched set, one-sided lists, value-difference list, side totals
- Decisions left with you
- Which source is authoritative; how differences are corrected
Job type E
Scheduled recurring processing
Once a job is defined and trialled, it can run to a rhythm. Each run is compared against the last, so a sudden change in row counts, a new column or a file that arrives empty is noticed and raised rather than processed blindly.
Schedules work within UK working days. If a file arrives outside the agreed window, it is processed in the next one, and the run note says so.
- Typical inputs
- A file arriving weekly, monthly or at period end
- Outputs
- Output plus run note each cycle, with comparison to the previous run
- Decisions left with you
- What change in volume should stop a run and trigger a question