Source Tracking in Sandboxes

As of Summer '20 you can now do source tracking in Sandboxes. That is push and pull from Sandboxes and SFDX will track the changes made in your sandbox.

Now it’s only Developer and Developer Pro Sandboxes and the Sandbox must be refreshed AFTER Summer '20 is released.

Here’s the documentation. There is a new Object in the org named SourceMember.

However, before you start go to Dev Hub in setup and Enable Source Tracking. You don’t need to enable Dev Hub.

I would run an SFDX Update before doing this. Ensure you are on API Version 49 and I would create a new SFDX Project for the new Sandbox.

Setup

So you create your sandbox. Log into your sandbox. Then create a new Project in VSCode and then run

sfdx force:auth:web:login -r https://test.salesforce.com -a yourAlias

The next thing you expect to do is

sfdx force:source:status -u yourAlias

And that this would show you all of your org’s metadata and then you would pull ALL your Sandbox’s metadata from your Sandbox to your VS Code, right? Right? Nope. Nothing.

Sooo, we are back being left with creating a package.xml and downloading the Sandbox config first. You would think Org Browser would do it but no, Org Browser does not work with Managed Packages installed in your org, and no matter what the Salesforce DX team say, a real org is very likely to have customisations built on top of managed packages - eg NPSP. And you can’t easily bring down ALL your org with Org Browser. Again it’s only suited to devs who use two or three types of metadata.

So I like Salesforce Package.xml Generator Extension. It’s about as easy, and about as slow as the old way of doing things in MavensMate.

So once you have your package.xml file run

sfdx force:source:retrieve -x force-app\manifest\package.xml

And of course it takes forever.

After that immediately do a

To find out all the metadata you added.

Oops. As per usual with working with DX you can get yourself into trouble.

I try to pull changes and of course it says no changes to pull.

I try to push changes (with no changes made) and it has an error.

ERROR running force:source:push: Expected file at path: C:\MyFolder\force-app\manifest\package.xml-meta.xml

Now i have never seen a package.xml-meta.xml file so I don’t know what that means.

What have I done wrong?

OK, so firstly I was on a pre-release version of the Salesforce CLI. Run sfdx plugins --core to check the version, then run these commands to fix it. See the SFDX Release Notes.

sfdx plugins --core sfdx plugins:uninstall salesforcedx sfdx update sfdx plugins --core

Then you can reset your tracking

sfdx force:source:tracking:reset

Then I did a push again and it worked

Thanks to Miles Robson for that trick to reset.

Suggested Process

Update your production org’s package.xml to the latest version and ensure that you can do a retrieve from production under the latest API version. Ensure your Source is up to date.
Commit your production source to your repo, just to back it up.
Save that whole folder as a new folder.
Refresh your sandbox or create a new sandbox (developer or developer pro)
Authorise the sandbox from that new folder.
(optionally) run sfdx force:source:tracking:reset I’m not quite sure yet if it is needed if you do these steps.
Commit your sandbox to your repo.
Start a new branch in your repo.
Of course add your data in whatever way you add data to your non-data filled Sandbox.
Start making some changes in your org or the metadata.
Push and / or pull as required.

Make Some Changes

Make changes by changing things in the org. Then pull the changes into source.

Or make changes to the metadata then push the changes into the org.

See https://tddprojects.atlassian.net/wiki/spaces/SF/pages/1081212929 for some tips to set up Powershell to make regularly used DX commands easier to use.

Query what changes there are

Query the status of your changes using SOQL https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_sandboxes_source_tracking_sourcemember.htm

But use the Tooling API.

If you get stuck

There are two new commands

Use with care. But if you mess things up, you may use reset first, then clear at the last resort, to start again. But have your changes committed to your source tracking first before you do that.

Org Structure

There is a new feature to use different folders in your VS Code project to push and pull into different folders. So maybe if you are working on a new feature you can use that different folder to collect all your changes in there, and then use https://tddprojects.atlassian.net/wiki/spaces/SF/pages/652345352 to deploy that folder to production. I need to check this out.

But in the worst case use the SOQL statement above, using the tooling API and it will tell you which metadata has changed so you could put it in a Change Set. But no, we don’t want to use Change Sets anymore, so you could at least put that metadata into a separate folder and deploy from there?

With the new feature of multiple folders, any changes to where you move files is not tracked in force:source:status, so you have to use your git tracking to work out where files and folders have moved to. I dunno, is this too hard. It is something I need to work on. YOU have to track your own dependencies and work out your own folder structure. It’s a lot of work to set up these different folders so it may not be worth it?

More work to be done on this to decide which is the best and easiest way for Admins to deploy.

Packages

No you do NOT need to use packages to use DX. But there may be some benefit to creating org-dependent unlocked packages (in beta as of Summer '20). But there is a LOT of metadata that can NOT be in unlocked packages. So check the Metadata Coverage API documentation, and for example, if your solution includes simple things like Picklists or Assignment Rules or Auto Response Rules and many other things, then it might not be right to be packaged.

It’s all about knowing your org’s metadata, and that is the first step to becoming successful with DX.