Andy in the Cloud

From BBC Basic to Force.com and beyond…

Managing your DML and Transactions with a Unit Of Work

24 Comments

plumbing-equipment

A utility class I briefly referenced in this article was SObjectUnitOfWork. I promised in that article to discuss it in more detail, that time has come! Its main goals are.

  • Optimise DML interactions with the database
  • Provide transactional control
  • Simplify complex code that often spends a good portion of its time managing bulkificaiton and  ‘plumbing’ record relationships together.

In this blog I’m going to show you two approaches to creating a reasonably complex set of records on the platform, comparing the pros and cons of the traditional approach vs that using the Unit of Work approach.

Complex Data Creation and Relationships

Lets first look at a sample peace of code to create a bunch Opportunity records and related, Product, PricebookEntry and eventually OpportunityLine records. It designed to have a bit of a variable element to it, as such the number of lines per Opportunity and thus Products varies depending on which of the 10 Opportunties is being processed. The traditional approach is to do this a stage at a time, creating things and inserting things in the correct dependency order and associating child and related records via the Id’s generated by the previous inserts. Lists are our friends here!

			List opps = new List();
			List productsByOpp = new List();
			List pricebookEntriesByOpp = new List();
			List oppLinesByOpp = new List();
			for(Integer o=0; o<10; o++)
			{
				Opportunity opp = new Opportunity();
				opp.Name = 'NoUoW Test Name ' + o;
				opp.StageName = 'Open';
				opp.CloseDate = System.today();
				opps.add(opp);
				List products = new List();
				List pricebookEntries = new List();
				List oppLineItems = new List();
				for(Integer i=0; i<o+1; i++)
				{
					Product2 product = new Product2();
					product.Name = opp.Name + ' : Product : ' + i;
					products.add(product);
					PricebookEntry pbe = new PricebookEntry();
					pbe.UnitPrice = 10;
					pbe.IsActive = true;
					pbe.UseStandardPrice = false;
					pbe.Pricebook2Id = pb.Id;
					pricebookEntries.add(pbe);
					OpportunityLineItem oppLineItem = new OpportunityLineItem();
					oppLineItem.Quantity = 1;
					oppLineItem.TotalPrice = 10;
					oppLineItems.add(oppLineItem);
				}
				productsByOpp.add(products);
				pricebookEntriesByOpp.add(pricebookEntries);
				oppLinesByOpp.add(oppLineItems);
			}
			// Insert Opportunities
			insert opps;
			// Insert Products
			List allProducts = new List();
			for(List products : productsByOpp)
			{
				allProducts.addAll(products);
			}
			insert allProducts;
			// Insert Pricebooks
			Integer oppIdx = 0;
			List allPricebookEntries = new List();
			for(List pricebookEntries : pricebookEntriesByOpp)
			{
				List products = productsByOpp[oppIdx++];
				Integer lineIdx = 0;
				for(PricebookEntry pricebookEntry : pricebookEntries)
				{
					pricebookEntry.Product2Id = products[lineIdx++].Id;
				}
				allPricebookEntries.addAll(pricebookEntries);
			}
			insert allPricebookEntries;
			// Insert Opportunity Lines
			oppIdx = 0;
			List allOppLineItems = new List();
			for(List oppLines : oppLinesByOpp)
			{
				List pricebookEntries = pricebookEntriesByOpp[oppIdx];
				Integer lineIdx = 0;
				for(OpportunityLineItem oppLine : oppLines)
				{
					oppLine.OpportunityId = opps[oppIdx].Id;
					oppLine.PricebookEntryId = pricebookEntries[lineIdx++].Id;
				}
				allOppLineItems.addAll(oppLines);
				oppIdx++;
			}
			insert allOppLineItems;

Lists and Maps (if your linking existing data) are important tools in this process, much like SOQL, its bad news to do DML in loops, as you only get 150 DML operations per request before the governors blow. So we must index and list items within the various loops to ensure we are following best practice for bulkificaiton of DML as well.  If your using ExternalId fields, you can avoid some of this, but to much use of those comes at a cost as well, and your not always able to add these to all objects, so traditionally the above is pretty much the most bulkified way of inserting Opportunities.

Same again, but with a Unit Of Work…

Now thats take a look at the same sample using the Unit Of Work approach to capture the work and commit it all to the database in one operation. In this example notice first of all its a lot smaller and hopefully easier to see what the core purpose of the logic is. Most notable is that there are no maps at all, and also no direct DML operations, such as insert. 

img_strategy_targetInstead the code registers the need for an insert with the unit of work, for it to perform later via the registerNew methods on lines 8,13,19 and 24. The unit of work is keeping track of the lists of objects and is also providing a kind of ‘stitching’ service for the code, see lines 19, 23 and 24. Because it is given a list of object types when its constructed (via MY_SOBJECT) and these are in dependency order, it knows to insert records its given in that order and then follow up populating the indicated relationship fields as it goes. The result I think is both making the code more readable and focused on the task at hand.


			SObjectUnitOfWork uow = new SObjectUnitOfWork(MY_SOBJECTS);
			for(Integer o=0; o<10; o++)
			{
				Opportunity opp = new Opportunity();
				opp.Name = 'UoW Test Name ' + o;
				opp.StageName = 'Open';
				opp.CloseDate = System.today();
				uow.registerNew(opp);
				for(Integer i=0; i<o+1; i++)
				{
					Product2 product = new Product2();
					product.Name = opp.Name + ' : Product : ' + i;
					uow.registerNew(product);
					PricebookEntry pbe = new PricebookEntry();
					pbe.UnitPrice = 10;
					pbe.IsActive = true;
					pbe.UseStandardPrice = false;
					pbe.Pricebook2Id = pb.Id;
					uow.registerNew(pbe, PricebookEntry.Product2Id, product);
					OpportunityLineItem oppLineItem = new OpportunityLineItem();
					oppLineItem.Quantity = 1;
					oppLineItem.TotalPrice = 10;
					uow.registerRelationship(oppLineItem, OpportunityLineItem.PricebookEntryId, pbe);
					uow.registerNew(oppLineItem, OpportunityLineItem.OpportunityId, opp);
				}
			}
			uow.commitWork();

The MY_SOBJECT variable is setup as follows, typically you would probably just have one of these for your whole app.

	// SObjects (in order of dependency)
	private static List MY_SOBJECTS =
		new Schema.SObjectType[] {
			Product2.SObjectType,
			PricebookEntry.SObjectType,
			Opportunity.SObjectType,
			OpportunityLineItem.SObjectType };

Looking into the Future with registerNew and registerRelationship methods

Screen Shot 2013-06-09 at 15.06.11These two methods on the SObjectUnitOfWork class allow you to see into the future. By allowing you to register relationships without knowing the Id’s of records your inserting (also via the unit of work). As you can see in the above example, its a matter of providing the relationship field and the related record. Even if the related record does not yet have an Id, by the time the unit of work has completed inserting dependent records for you, it will. At this point, it will set the Id on the indicated field for you, before inserting the record.

Delegating this type of logic to the unit of work, avoids you having to manage lists and maps to associate related records together and thus keeps the focus on the core goal of the logic.

Note: If you have some cyclic dependencies in your schema, you will have to either use two separate unit of work instances or simply handle this directly using DML.

Deleting and Updating Records with a Unit Of Work…

This next example shows how the unit of work can be used in a editing scenario, suppose that some logic has taken a bunch of OpportunityLineItem’s and grouped them. You need to delete the line items no longer required, insert the new grouped line and also update the Opportunity to indicate the process has taken place.

			// Consolidate Products on the Opportunities
			SObjectUnitOfWork uow = new SObjectUnitOfWork(MY_SOBJECTS);
			for(Opportunity opportunity : opportunities)
			{
				// Group the lines
				Map<Id, List> linesByGroup = new Map<Id, List>();
				// Grouping logic
				// ...
				// For groups with more than one 1 line, delete those lines and create a new consolidated one
				for(List linesForGroup : linesByGroup.values() )
				{
					// More than one line with this product?
					if(linesForGroup.size()>1)
					{
						// Delete the duplicate product lines and caculate new quantity total
						Decimal consolidatedQuantity = 0;
						for(OpportunityLineItem lineForProduct : linesForGroup)
						{
							consolidatedQuantity += lineForProduct.Quantity;
							uow.registerDeleted(lineForProduct);
						}
						// Create new consolidated line
						OpportunityLineItem consolidatedLine = new OpportunityLineItem();
						consolidatedLine.Quantity = consolidatedQuantity;
						consolidatedLine.UnitPrice = linesForGroup[0].UnitPrice;
						consolidatedLine.PricebookEntryId = linesForGroup[0].PricebookEntry.Id;
						uow.registerNew(consolidatedLine, OpportunityLineItem.OpportunityId, opportunity);
						// Note the last consolidation date
						opportunity.Description = 'Consolidated on ' + System.today();
						uow.registerDirty(opportunity);
					}
				}
			}
			uow.commitWork();

Transaction management and the commitWork method

Database transactions is something you rarely have to concern yourself within Apex…. or do you? Consider the sample code below, in it there is a deliberate bug (line 22). When the user presses the button associated with this controller method, the error occurs, is caught and is displayed to the user via the apex:pagemessages component. If the developer did not do this, the error would be unhandled and the standard Salesforce white page with the error text displayed would be shown to the user, hardly a great user experience.

	public PageReference doSomeWork()
	{
		try
		{
			Opportunity opp = new Opportunity();
			opp.Name = 'My New Opportunity';
			opp.StageName = 'Open';
			opp.CloseDate = System.today();
			insert opp;
			Product2 product = new Product2();
			product.Name = 'My New Product';
			insert product;
			// Insert pricebook
			PricebookEntry pbe = new PricebookEntry();
			pbe.UnitPrice = 10;
			pbe.IsActive = true;
			pbe.UseStandardPrice = false;
			pbe.Pricebook2Id = [select Id from Pricebook2 where IsStandard = true].Id;
			pbe.Product2Id = product.Id;
			insert pbe;
			// Fake an error
			Integer x = 42 / 0;
			// Insert opportunity lines...
			OpportunityLineItem oppLineItem = new OpportunityLineItem();
			oppLineItem.Quantity = 1;
			oppLineItem.TotalPrice = 10;
			oppLineItem.PricebookEntryId = pbe.Id;
			insert oppLineItem;
		}
		catch (Exception e)
		{
			ApexPages.addMessages(e);
		}
		return null;
	}

However using try/catch circumvents the standard Apex transaction rollback during error conditions. “Only when all the Apex code has finished running and the Visualforce page has finished running, are the changes committed to the database. If the request does not complete successfully, all database changes are rolled back.”. Therefore catching the exception results in the request to complete successfully, thus the Apex runtime commits records that lead up to the error occurring. This results in the above code leaving an Opportunity with no lines on the database.

The solution to this problem, is to utilise a Savepoint, as described in the standard Salesforce documentation. To avoid the developer having to remember this, the SObjectUnitOfWork commitWork method creates a Savepoint and manages the rollback to it, should any errors occur. After doing so, it throws again the error so that the caller can perform its own error handling and reporting. This gives a consistant behaviour to database updates regardless of how errors are handled by the controlling logic.

Note: Regardless of using the commitWork method or manually coding your Savepoint logic, review the statement from the Salesforce documentation regarding Id’s.

Summary

As you can see between the two samples in this blog, there is significant reduction of over half the source lines when using the unit of work. Of course the SObjectUnitOfWork class does have its own processing to perform. Because it is a generic library, its never going to be as optimum as if you would write this code by hand specifically for the use case needed as per the first example.

perfect-balance

When writing Apex code, optimisation of statements is a consideration (consider Batch Apex in large volumes). However so are other things such as queries, database manipulation and balancing overall code complexity, since smaller and simpler code bases generally contains less bugs. Ultimately, using any utility class, needs to be considered on balance with a number of things and not just in isolation of one concern. Hopefully you now have enough information to decide for yourself if the benefit is worth it in respect to the complexity of the code your writing.

Unit Of Work is a Enterprise Application Architecture pattern by Martin Fowler.

24 thoughts on “Managing your DML and Transactions with a Unit Of Work

  1. UOW is a great pattern and Andy has done us a huge service in creating this for Apex. It takes something that is complex and makes it very accessible. I’ve been fortunate to be using UOW since I began developing enterprise Apex last year and it is a huge time saver.

  2. Your unit-of-work description reminds me of the “Collecting Parameter” pattern discussed in Kent Beck’s “Smalltalk Best Practice Patterns.”

    In a base class we frequently extend we’ve implemented two virtual methods, save() and saveTo(list). Each subclass typically overrides saveTo() to add its own sobject to the list, then asks each of its contained objects to do the same. The end-result is the save() is implemented as:

    public virtual void save()
    {
    List aList = new List();
    List updateList = new List();
    List insertList = new List();

    saveTo(aList);

    for (sobject each : aList) {
    if (each.id == null)
    insertList.add(each);
    else
    updateList.add(each);
    }

    try {
    update updateList;
    insert insertList;
    }
    catch (DMLException dmlex)
    {
    XedeException.Raise(‘Error adding or updating object : {0}’, dmlex.getMessage());
    }
    }

    A similar approach could be used for custom-generating XML. The top method might be called getXmlString(), and its collector called getXmlStringTo(string aString). The similar pattern to implementation would follow. Each subclass would override getXmlStringTo() to write its own XML, and would ask each of its children to do the same. The same pattern could be used for customized DOM or JSON results.

    • Interesting stuff indeed, good to see others thinking about making this easier. Do you help with stitching references together during commit phase?

      • Now that I think about it, a better mechanism might be to call saveTo() repeatedly until its list is empty, and let the objects solicited decide for themselves if their dependents need to be collected.

        I can imagine a purchase order with lines answering different results for the 2nd saveTo() from the first. The first time it would save its own sobject and on the second call its dependents, the lines. It could use its own sobject’s id as a flag to indicate whether or not it needs to call its chidren’s saveTo()s.

        A more sophisticated dependency mechanism could be contrived, but may be unnecessarily complex.

      • Tom, some great thoughts, loving the dialogue here! One thing to keep in mind is bulkification, normally attributed to SOQL, but also applies to DML as well. For single request, if you for example insert multiple parent and child relationships, to truly bulkify all parents must be inserted then all children. So having each parent delegate to its child as its inserted will break this. Thus the UOW pattern helps aggregate this information until the request is ready to commit all its work and then as you say processes them in dependency order, but as an entire set of related records. In general though I very keen on keeping as much OOP / domain style programming alive on the platform as possible, while working within its governors.

      • Andrew, your reply this morning to my post last night appears, but not my post from last night.

        It also occurs to me that the code may not be obvious that the children’s inserts are deferred until after their parents (all their parents) are inserted. After the parent’s inserts, the dependent list is processed, and ALL the children are inserted at once.

        Best of all, if any of the children have dependents they can be added to the next dependent list and processed–all at once. For new records, each level of dependency is handled with a single INSERT regardless how many sobjects there are.

        The biggest danger is inserting too many at once. But that can now easily be remedied inside save(), for which I need only a single implementation.

  3. To date, we haven’t had the necessity for inserting parents coincident with their children. It may be either we haven’t had the need for it, or the way we’ve built systems we’ve inadvertently avoided the problem.

    Regardless, as I read your looking into the future section, and if I understand the intent, we would likely try to exploit our dependency mechanism, again inspired by Smalltalk’s, so that the save event of one object could notify other objects, or at least activate a second save, on the just-inserted objects, and allow the dependency mechanism to cascade. It would then be the designer’s responsibility to make sure there weren’t more than 100 levels of dependencies to avoid tripping governor limits.

    Of course, Apex is statically typed so we’d have to write more code than in Smalltalk, but the end-result would be similar. Objects could indicate they depend on another object, and when that object’s state changed (its sobject is saved), it could notify its dependents, or issue a subsequent saveTo() across them.

  4. Andrew, I’m running into an interesting circular reference problem. I’m trying to create a contact and it’s parent account in one transaction. We have a field on our Account schema called Default Contact that we want to be set at the same time as creating the records. I’ve tried something like this:

    uow.registerNew(a);
    uow.registerNew(c);
    uow.registerRelationship(c, Contact.AccountId, a);
    uow.registerRelationship(a, Account.Default_Contact__c, c);
    uow.commitWork();

    This works except for the last step, the Default Contact on the account does not appear to get set (even after requerying). I was able to get it work using multiple uow’s and setting the default contact after the first one committed.

    So a couple of questions:

    Am I missing something with UOW? Is there a way to handle a circular reference cleanly?

    If not, should I be managing my own savepoints and just skip UOW? (I was a bit concerned about putting DML into a service class, but perhaps that’s okay?)

    As always I appreciate your insights. Good luck at Dreamforce – wish I could have made it this year!

    • Yeah such a shame we could not share a coffee and/or beer, the event was amazing, the good news is, it just keeps getting better and better! Anyway back to your question, yes, you’ve hit apone a currently limitation, the workaround is as you suggest to break it out to another UOW. Would you mind raising this on the new GitHub repo for the library, this ought to be something the library can do. https://github.com/financialforcedev/fflib-apex-common. Btw, feel free if you want to have a go at a fix yourself, and make a pull request. Hope to catch you next year and thanks for the continued support! 🙂

    • sstone, the collecting parameter pattern handles your scenario quite easily.

      First, you might consider remodeling your data. Uncomfortable DMLs sometime hint that there may be simpler designs. For instance, “isDefault” might be a better attribute on the contact record than a defaultContract attribute on the account. This approach, at least, could be accomplished with two DMLs rather than three.

      I think three DMLs for your model are inescapable because there are two dependencies rather than one. But rather than go-on on Andrew’s real estate I’ll resolve to describe it elsewhere.

  5. Pingback: Doing more work with the Unit Of Work | Andy in the Cloud

  6. Hi Andy,

    I have a question about managing duplicates within a uow.

    Lets say I have an account domain class:

    public class Accounts extends fflib_SObjectDomain
    {
    private static Schema.SObjectType[] MY_SOBJECTTYPES = new Schema.SObjectType[]{ Account.SObjectType, Entitlement.SObjectType };

    public override void onAfterUpdate(Map oldRecords)
    {
    fflib_SObjectUnitOfWork uow = new fflib_SObjectUnitOfWork(MY_SOBJECTTYPES);
    methodA(uow);
    methodB(uow);
    uow.commitWork();
    }

    public void methodA(fflib_SObjectUnitOfWork uow)
    {
    for(Account acc : (List)Records)
    acc.FieldA = ‘Field Update’;
    uow.RegisterDirty(acc);
    }

    public void methodB(fflib_SObjectUnitOfWork uow)
    {
    for(Account acc : (List)Records)
    acc.FieldB = ‘Field Update’;
    uow.RegisterDirty(acc);
    }

    }

    In these instances, I’m finding that I get duplicate ids. Is there a best practice on how to handle this type of scenario so that I don’t get duplicate ids when the work is committed?

    Thanks!

  7. Re: registerDirty(someRec); Worth noting when using a common UoW and multiple dirtying methods for the same domain. For example, imagine an onAfterUpdate in Opportunity that creates a uow to push some changes down into OpportunityProducts (OLI).

    Pseudo code is something like:

    Opportunity[] opposWithChildren = new OpportunitySelector().selectByIdWithChildren(someSetOfOppoIds);
    for (Opportunity o : opposWithChildren)
    olisToTouchA.addAll(o.isSomethingA__c ? o.OpportunityLineItems : new List());

    // Start a UoW
    fflib_ISobjectOfWork uow = Application.UnitOfWork.getInstance();

    // Now ask OLI Domain class to change the OLI – Method A
    OpportunityLineItems oliAs = new OpportunityLineItems(olisToTouchA);
    oliAs.methodA(uow); // MethodA uses uow.registerDirty() on each OLI altered

    // …. other stuff

    // But wait, we have some other encapuslated domain change we need OpportunityLineItems to do (MethodB)
    for (Opportunity o : opposWithChildren)
    olisToTouchB.addAll(o.isSomethingB__c ? o.OpportunityLineItems : new List());

    // Now ask OLI Domain class to change the OLI – Method B
    OpportunityLineItems oliBs = new OpportunityLineItems(olisToTouchB); // instantiate anew
    oliBs.methodB(uow); // MethodB also uses uow.registerDirty() on each OLI altered


    // commit the changes
    uow.commitWork();

    Now, if sharing a uow between MethodA and MethodB (same uow passed to each from Opportunities domain method):

    1. Because MethodA and MethodB both touch OpportunityLineItems and ..
    2. Both MethodA and MethodB registerDirty on potentially the same OLI instance …
    3. When commitWork() comes along, only the last (Method B’s) changes will be committed …
    4. UNLESS the UoW creator takes care to pass to Method B the OLI Sobjects sent to Method A (and presumably changed in situ by Method A) , then the Method A changes are lost.

    Now, this isn’t a flaw in the pattern but more of a consideration to remember as records to change are being fetched in class Opportunities but are updated in some distant class OpportunityLineItems and the committed in Opportunities. All mediated by the UoW class.

    Something perhaps worth noting in the 2nd edition on page 140 of the Force.com Enterprise Architecture Book

    Meta-comment: Most of the above comes about from rewiring my brain to use the fflib’s patterns after years of ‘classic Apex’. At first, the brilliance of the pattern lulls one into complacency but then as you code/re-engineer code, and you try and super-encapsulate behavior into where it belongs, you start asking yourself questions – does my code make it obvious to the next person that the changes made by MethodA are being passed into MethodB for subsequent changes?

    I could obviously have used two UoWs but that goes against the grain of minimizing database operations so as to minimize SOQL.

    • Thanks for this great observation, such feedback from real world usage is vital to the community. Your correct i don’t think its something that can be really delt with technically, its more of an awareness thing. Though i do wonder if there was a facility to merge a result set from a query with the current contents of the UOW that might help. e.g. List results = mySelector.selectById(myIds, myUow); However i think this is a step to far as i can already see complexities emerging from this approach. In the end the sometimes the developer just has to keep things in mind, so long as on balance a framework is always adding value thats the main thing.

    • And yes this is really good feedback for a second edition of the book…. 😉

  8. Hi Andy,

    I am trying to get the product names of an opportunity through dynamic apex and SOQL.
    below is the piece of code
    for(sObject eachchildRec : data.getsObjects(eachchild)){
    if(childFieldName.countMatches(‘.’)==2){
    value += eachchildRec.getsObject(‘PricebookEntry’).getsObjects(‘Product2’)[0].get(‘Name’) + ‘,’ ;
    }
    else
    value += eachchildRec.get(childFieldName) + ‘,’ ;
    }

    where data is opportunity record which is queried through dynamic SOQL, childFieldName can be ‘pricebookEntry.product2.Name’

    When I try to execute, I am getting an error like “System.SObjectException: Invalid aggregate relationship Product2 for PricebookEntry”

    Can you help in getting rid of this error?

    • What does the soql query look like in the debug log? It’s hard to tell what’s wrong from your code as I don’t know it

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s