I’ve been wanting for a while now to explore using the Apex JSON serialize and deserialize support with SObject’s. I wanted to see if it was possible to build a generic native import/export solution natively, that didn’t just handle a single record was able to follow relationships to other records. Such as master-detail and lookup relationships. Effectively to give a means to extract and import a set of related records as one ‘bundle’ of record sets. With the flexibility of something like this, one could consider a number of uses…

• Import/Export. Provide Import and Export solutions directly a native application without having to use Data Loader.
• Generic Super Clone. Provide a super Clone button that can be applied to any object (without further coding), that not only clones the master record but any related child detail records.
• Post Install Setup. Populate default data / configuration as part of a Package post installation script.
• Org 2 Org Data Transfer. Pass data between orgs, perhaps via an attachment in an email received via an Apex Email Handler that receives an email from Apex code in anthor org?

For me, some interesting use cases and touch apone a number of topics I see surfacing now and again on Salesforce StackExchange and in my own job. However before having a crack at those I felt I first needed to crack open the JSON support in Apex in a generic way or at least with some minimal config.

Background and Credit

A fellow developer Agustina García had recently done some excellent research into Apex JSON support. She found that while JSON.serialise would in fact work with SObject’s containing related child records (queried using subqueries), by outputting the child records within the JSON (as nested JSON objects). Sadly however, the JSON.deserialize method ignores them. The solution was to utilise some wrapper Apex classes to contain the SObject records and provide the necessary relationship structure. This gave me the basis to develop something more generic that would work with any object.

Introducing SObjectDataLoader

Like the JSON class, the SObjectDataLoader class has two main methods, ‘serialize‘ and ‘deserialize‘. To serialize records all you need give it is a list of ID’s. It will automatically discover the fields to include and query the records itself. Seeking out (based on some default rules) related child and lookup relationship records to ‘follow’ as part of the recursive serialisation process. Outputting a serialised RecordsBundle. Due to how the serialisation process executes the RecordSetBundle’s are output in dependency order.

Consider the following objects and relationships with a mix of master-detail and lookup relationships as shown using the Schema Builder tool.

Here is a basic example (utilising auto configure mode) that will export records in the above structure, when given the Id of the root object, Object A.

String serialisedData =
  SObjectDataLoader.serialize(
      new Set<Id>; { 'a00d0000007kUms' });

The abbreviated resulting JSON looks like this (see full example here)…

{
    "RecordSetBundles": [
        {
            "Records": [
                {
                    "Id": "a00d0000007kUmsAAE",
                    "Name": "Object A Record"
                }
            ],
            "ObjectType": "ObjectA__c"
        },
        {
            "Records": [
                {
                    "Id": "a03d000000EHi6tAAD",
                    "Name": "Object D Record",
                }
            ],
            "ObjectType": "ObjectD__c"
        },
        {
            "Records": [
                {
                    "Id": "a01d0000006JdysAAC",
                    "Name": "Object B Record",
                    "ObjectA__c": "a00d0000007kUmsAAE",
                    "ObjectD__c": "a03d000000EHi6tAAD"
                }
            ],
            "ObjectType": "ObjectB__c"
        },
        {
            "Records": [
                {
                    "Id": "a04d00000035cFAAAY",
                    "Name": "Object E Record"
                }
            ],
            "ObjectType": "ObjectE__c"
        },
        {
            "Records": [
                {
                    "Id": "a02d000000723fvAAA",
                    "Name": "Object C Record",
                    "ObjectB__c": "a01d0000006JdysAAC",
                    "ObjectE__c": "a04d00000035cFAAAY"
                }
            ],
            "ObjectType": "ObjectC__c"
        }
    ]
}

The ‘deserialize‘ method takes a JSON string representing a RecordsBundle (output from the ‘serialize’ method). This will insert the records in the required order and take care of making new relationships (as you can see the original Id’s are retained in the JSON to aid this process) as each record set is processed.

Set<Id> recordAIds =
    SObjectDataLoader.deserialize(serialisedData);

The auto configuration route has its limitations and assumptions, so a manual configuration mode is available. This can be used if you know more about the objects. Though you can merge both configuration modes. The test methods in the class utlise some more advanced usage. For example…

String serializedData =
    SObjectDataLoader.serialize(createOpportunities(),
       new SObjectDataLoader.SerializeConfig().
        // Serialize any related OpportunityLineItem's
        followChild(OpportunityLineItem.OpportunityId).
          // Serialize any related PricebookEntry's
          follow(OpportunityLineItem.PricebookEntryId).
            // Serialize any related Products's
            follow(PricebookEntry.Product2Id).
            // Skip UnitPrice in favour of TotalPrice
            omit(OpportunityLineItem.UnitPrice));

The deserialize method in this case takes an implementation of SObjectDataLoader.IDeserializerCallback. This allows you to intercept the deserialization process and populate any references before the objects are inserted into the database. Useful if the exported data is incomplete.

Set<ID>; resultIds =
   SObjectDataLoader.deserialize(
      serializedData, new ApplyStandardPricebook());

Some interesting aspects of the implementation…

• It discovers the SObjectType via the new method Id.getSObjectType.
• It uses the SObject.clone method to remove the old Id (after making a note of it for later reference) from the deserialized SObject allowing it to be inserted.
• The native JSON.serialize and JSON.deserialize are used only once to serialise and deserialize the internal RecordsBundle object.
• The SerializeConfig object uses the Fluent API model.
• JSONLint rocks!
• Unit tests with over 95% code coverage are included in the same class, so the single class is ready to drop in your projects!

Summary

I hope this will be of use to people and would love to hear about more use cases and/or offers to help extend (see TODO’s below if you fancy helping) or fix any bugs in it to grow it further. Maybe one day Salesforce might even consider implementing into the Apex runtime! In the meantime, enjoy! GitHub repo link.

TODO’s

This is an initial release that I think will suite some use cases within the current tolerances of the implementation. As always, life and work commitments prevent me from spending as much time on this as I would really like. So here are a few TODO’s, until next time…

• Support Recursive Lookup References, e.g. Custom Object A has a lookup to Custom Object A.
• Scalability, Improve scalability via Batch Apex
• More Extensibility and Callbacks, I add a deserialization Apex callback interface to resolve dynamically required references that have not been included in the serialised output. This can be extended further to customise additional fields or change existing ones.
• Optimise queries and ‘fields’ usage, (for repeat references to the same object further down the follow chain).