First things!

Turn off the advertising - User Interface. Uncheck Enable Salesforce Notification Banner.
Turn off the Lightning nag screen - User Interface. Check Do not show Lightning Experience-enabled end users the "Try Lightning Experience Now" prompt
(NOTE that the two turning off advertising checkboxes are opposite you check one and uncheck the other!)

New Features

Even without the new Lightning Experience, all new orgs come without excellent new features enabled, and you will miss out on great functionality if you do not enable them.

History Tracking on Standard Objects
Chatter Feed Tracking
Feed Layouts (not so much needed in The Lightning Experience, but you will still need to customise EVERY page).
Salesforce Topics (maybe don't use these until they are updated for Lightning now).
Actions! Global and Quick Actions
Salesforce Mobile App (needs to be set up properly)
Lightning for Outlook (if you are having to use Outlook) or Lightning for Gmail
State and Country Picklists.
One Contact to Many Accounts.
Turn off News in Accounts settings - this is only applicable for US Orgs.
Files - Feature Settings > Salesforcde Files > General Settings
- Files uploaded to the Attachments related list on records are uploaded as Salesforce Files, not as attachments.
Files Connect
Out of Office on Chatter

Company Information and Admin

Set Fiscal Year
Set Business Hours
Set Password Policies
Set Session Settings
Set Trusted IP Ranges
Set Login Access Policies to be only for Admins to request Salesforce.com help.

Security Settings

Ask client about restricted users and times they should be restricted to.
Ask about Email Footers. (However, see Emails for issues with Email Footers).

Other

Enable State and Country Picklists.
Set up Salesforce Mobile App settings.
Rename Apps and Tabs if needed.
Enable My Domain

Apps

Create a Lightning App for the organisation.

Global Actions

Add Follow Up Action - New Task, default to 2 days from now, assigned to the owner of the record, with the subject of Follow Up.

Global Actions Layouts

Salesforce Mobile and Lightning Experience Layout

Add Log a Call Action
Add Follow Up Action
Add Mobile Smart Actions
Hide Polls Action
Hide Questions Action
Hide Email Actions
Hide Thanks Actions (unless your organisation has a formal methodology around open thanks and praise, otherwise, it's just a bit naff).
Hide all New record actions unless your organisation is always creating Leads and Opportunities from wherever - (I don't think that is wise at all).

User Interface

Enable the following
- Enhanced Inline Editing
- Show Custom Sidebar on all pages (if using Classic)
- Collapsible Sections
- Hover Details
- Related Lists Hover Links
- Separate Loading of Related Lists
- Enhanced Lists
- Spell Checker
- Spell Checker on Tasks and Events
- Enhanced Profile List Views
- Enhanced Profile User Interface
- New User Interface Theme
- Improved Setup User Interface
Do not enable
- Collapsible Sidebar (if using Classic)
Disable the following
- Show Quick Create
- Check Do not show Lightning Experience-enabled end users the "Try Lightning Experience Now" prompt

Company Profile

Set Company Information
Default Timezone
Set a domain under Company Profile -> My Domain
Turn on Multicurrency if needed.

Chatter

Set up Chatter
- Turn on Emoticons if you are that sort of organisation
- Enable Out of Office

All Objects

Set up Feed Tracking
Set up Record Topics
Set up Feed Layouts (if using Classic)
Assign Feed Layouts
Remove all of the standard "custom fields"
Remove all of the standard page layouts except for one
Remove all Data Import buttons
Remove all Print List View or Print Page Layout Buttons
Remove Discover Companies from Accounts

Next Steps

See Objects for what to do with each object.

Also See Required Apps (only for Classic)

For Lightning:

Recycle Bin

Articles

See other articles on this topic.