Duplicate Management

If you are using Lightning the standard Salesforce Duplicate management is quite good. 

Add the Duplicates Component on the top right hand side of the page, above Chatter. 


Set only to Report if you can't deal with live duplicates happening - eg if you have Web to Lead and a Duplicate is found, the Web to Lead will report an error. So set Create to Report, not to Alert. 

Duplicate rules interfere with Process Builder, you will get a Process Builder error rather than an alert about Duplicates, so set it to Report rather than Alert if you are using Process Builder on that object. This will be better in #Summer18 though. 

Note: There is a new feature called Duplicate Jobs, which will check for duplicates already in your org, but it is only available if you have Performance or Ultimate edition of Salesforce and it doesn't work on Custom Objects.

Notes

Some notes and observations about standard Duplicate Management. The more I use it, the more I like it. 

Cross Object Matching

  • Create a Duplicate Rule to Match on Contacts using the Standard Lead Matching rule on the cross object. That way if you look at the Contact, it will show the Duplicate to the Lead. Nice. 
  • The rule does not seem to pick up a duplicate IF the Lead is Converted - this is nice. Although I can't see any docco about it, and I can't quite verify it on my data set yet, so stay tuned on this one, or mention in the comments if you can confirm or deny. 

Chatter Does Not Merge!

Report vs Alert

  • Apparently the Duplicate matching rule runs when you open the record, so it may show as a duplicate even though there is no record in the Duplicate Record Sets (not sure how this all works though yet). 
  • When you are bulk importing via the API turn on Report and turn off Alert, otherwise records won't be created 

Making Rules Run

  • See the docco as to when duplicate rules do not run
    • Eg this is yet another reason to NOT turn on syncing of Contacts from Outlook or Google to Salesforce - they don't get matched. 
    • And another reason to NEVER use Quick Create. 
  • Check the docco about fields that have been deleted - they invalidate the duplicate rules. 
  • To force the rules to run, you have to Edit one of the fields included in the matching rule. 
    • So I updated Email to blank then updated Email back to the Email and the duplicates were checked. Nice. 

Duplicates

The duplicates to check for are particular to each organisation but here are some basics. 

  • First Name + Last Name + Email
  • Last Name + Email (depending on if you deal with Families that share email addresses - may not be useful). 
  • First Name + Last Name + Phone (or Mobile)
  • First Name + Last Name + Street
  • Account Name
  • Account Name + Phone
  • Account Name + Street + City

Create your own Duplicate Record Sets

You know that great feature in Classic named Merge Accounts where you could just pick any two or three Accounts with similar names and merge them. You know that great feature in Classic on the Contact Related List where you can mere Contacts? Both of these aren't in Lightning, so unless you are fortunate enough to use the Nonprofit Success Pack - NPSP and their excellent Contact Merge function, then you're out of luck, right? Well not quite... but it's not great. 

Announced in #Winter18 is the ability to create your own Duplicate Record Sets and merge them yourself. It's not very intuitive and it's not very easy to do, but at least you can do it. But I think it might be quicker to switch back to Classic to do it anyway. 

Here is the limited docco on the "feature" https://help.salesforce.com/articleView?id=duplicate_management_duplicate_record_sets.htm&type=5 

You can't use this feature on Custom Objects. 

Duplicate Reporting

When you set Report on Standard Salesforce Duplicate Rules, it's not clear at all what Report does. 

Basically it creates a record in an object named Duplicate Record Sets and Duplicate Record Items, and forever they shall remain there. YES. Duplicate Record Sets do not get deleted when you merge duplicates. 

So just understand that but it's a good to turn on Reporting anyway. 

When to Use Report on Duplicates

  • When using Web to Lead - Alert will stop the Duplicate Lead being created and will cause an error. (Similar for Web to Case or Case or Lead Creation via Communities)
  • When you use Process Builder on that Object. Duplicates will cause a Process Builder error rather than reporting duplicates. This is better in #Summer18 though, so I will post more details on that soon.  

Finding Duplicates

Duplicate Component

When you use Report, and not Alert, this is the first time they will know about the Duplicate, so don't hide it away - add it to the top right hand corner of the page, above Chatter. 

Some people say the Duplicate Component is not visible enough, because it's there all the time, so people don't notice the duplicates. 

There is no "Has Duplicates" field to filter on to show the Duplicate Component ONLY if there was duplicates. See below for some tweaks. 

Reporting

You can create a Salesforce Report Type on Leads or Accounts or Contacts and have a child record of Duplicate Record Items

Group by Duplicate Record Set Name. 

Sort by Date Created Desc. 

This report is OK because you can bring in fields from the Lead or Contact or Account or other duplicated record. 

It's a good idea to bring the Duplicate Record Set Rule Name into the report so you can see that it is a Lead to Contact Duplicate Type as they will only show with one Record in the Report. 

Duplicate Record Item View

Navigate to the Duplicate Record Record Sets tab from the Launcher. 

Create a new View for Lead Duplicates, for example. 

This is OK, but The Duplicate Record Item Lightning Page can NOT be modified, so you have to click Related to get to the related records of Duplicate Record Items, then click through to the record. This is not ideal.

BUT, you can just click Compare and Merge from the Action button on each list item. But you can't merge Lead to Contact. 


On Duplicate Record Items, when there is a Lead to Contact Duplicate Rule, the Duplicate Record Items do NOT tell you that one is a Contact. You have to hover over the Record Name to Find it. 

Related List

Although Duplicate Record Item looks up to any type of Object, you can NOT put Duplicate Record Items Related List on any page. So the only way to see it is to click the View Duplicates link on the Duplicates Component. This is not ideal, but from there you can merge them. 

Some Tweaks

Make Duplicates Component More Visibile

  • Create a field on Lead named No. Duplicates (NumDupes__c) - Number, No Decimals. 
  • Create a Declarative Lookup Rollup Summary rolling up a Count of RecordID on DuplicateRecordItem to Lead
  • Deploy the DLRS trigger to DuplicateRecordItem
  • Set DLRS to Active and Realtime / System
  • Calculate the DLRS to set the value for all existing Duplicates. 

All works well except for bulkification were I get a too many SOQL errors on updating bulk records. 

Now, on the Lead

  • Set the Component Visibility on the Duplicates Component to only show if No. Duplicates >=1
    • >=1 is important because if it is a Lead / Contact it will still have one Lead record. 
  • Set the Component to show both Toast and Card


The toast looks like this. It is also not quite visible. 

Using the new No. Duplicates field you could add a fancier Toast (eg using a component like this) or a Conditional Rich Text Component saying HEY, there's a DUPLICATE, do something about it!  

Process Builder on Duplicate Record Sets 

If you want to use Duplicate Record Sets rather than Reports, then try these tweaks. 

Add a Lookup to Duplicate Record Set to Lead named Lead Record, and another to Contact named Contact Record. 

Create a Process Builder on Duplicate Record Item. 

Create a Criteria for Is a Lead

Then Update the Duplicate Record Set Lead Record field with the RecordID from the Duplicate Record Item

Then modify your Duplicate Record Set View to include the Lead Record, so you can naviogate to the Lead right away

You will just have to see how this goes with importing or updating bulk reocrds though. Yeah, nah, it doesn't work well. I got too many SOQL Statements 101, when doing a bulk update. Ah well, back to the drawing board. 

So, overall Custom Report Types would be good if you want to see all the record details before you merge them. Duplicate Record Set Views are good if you just want to get in there and merge them. 

Cleaning Up

Duplicate Record Items and Duplicate Record Sets DO NOT delete on their own. 

They will be reused apparently if there is another duplicate. 

You can't delete them via Mass Delete

You would have to delete them via Dataloader or similar. 

Or you could create a Flow that deletes them wheneever the Record Count = 1.