Bad Data

Owned by Jodie Miners
Nov 12, 2015
2 min read

Duplicates

Reasons why you may have duplicates:

  • They changed work and have two emails at two different work addresses
  • The gave you a work email once and a personal email another time. 
  • They have a legal name and a common name eg William Bruce Smith and Bill Smith
  • They have an anglicized name and their formal name
  • They give you a mobile phone one time and a landline the next time. 
  • All of the above - eg you have an anglicized name with a personal email address and mobile phone number with a full legal name work email and work landline - there is no way you are going to know that this is the same person. 

 

Ways to help with duplicates and dirty data:

  • Be clear in your web forms what data you need
    • Eg, if you have signed up with us before, you would have used your full legal name, please enter that here. 
  • Use functionality like the Nonprofit Success Pack - NPSP that allows for personal and work emails and allows you to choose which one is primary. 
  • Always include a DataSource field that you set to a value to show where the data comes from.
  • Always have a reason that data exists in your database
  • Do not enter people in your database that have no way of contacting them, or if you only do email communication, why are people in your database that do not have an email address - and if the answer is "we are going to clean them up later" - No, you are not. You will NEVER get to them. Export these records to excel and delete them. 
  • If they are there because they have related records - mark them as DIRTY in some way. 
  • Do formulas for Data Quality scores - eg no email, no postcode, no suburb ranks them lower. 
  • Use Rollup Helper to roll up the number of related records and add a field on the contact to count how many of these related records they are - eg that is why they are in the database - they have had 6 previous interactions with us. 
  • Ensure reporting for export, or integration with Mailchimp excludes those low value or DIRTY records. 
  • Don't be afraid to bulk delete data. (export to excel first)
  • Use Demand Tools for cleansing existing duplicates. 
  • Be fastidious about cleaning up your data.