Andy in the Cloud

From BBC Basic to Force.com and beyond…

Salesforce Winter’13 Chaining Batch Jobs… With great power…

13 Comments

… comes great responsibility!

Ok so I am a Spiderman fan, you rumbled me!

But seriously thats the first phrase that popped into my head when I read the Winter’13 release notes regarding this ability. Previously you could not start another Batch Apex job from another Batch Apex job. So now you can! So the next question is why would you want to? Closely followed by should you? Before answering those, lets take a step back and consider what it takes to design and implement a single Batch Apex job…

As you probably already know the transaction semantics of Batch Apex means that you have to give careful consideration to what happens if one or more of the scopes or ‘chunks’ of the job fails and others succeed. Since only the changes to the database made by the failing chunk of the job are rolled back.

A Batch Apex Design and Implementation Checklist

So here is my Batch Apex design check list for developers and business analysis to discuss before a single line of code is written…

  1. How do users start the job? VF Page, Custom Buttons, Scheduler?
  2. How do you deliver feedback about the job? When is it queued, in progress and completed. Under high service load the fact that a job is queued can be quite important information from a usability and end user satisfaction perspective. Also an interesting System Admin vs End User consideration is permitting the default error handling by the platform to occur vs catching your exceptions and feeding them back some other way?
  3. Can multiple jobs of the same type be run at the same time? If yes, how do you manage the dataset each job consumes to avoid overlap with other jobs in a concurrent user environment? If no, how do you plan to stop users running multiple jobs?
  4. How do users get informed about failures? Chatter posts? Email? Custom Objects? Status fields? Does the information have enough context about the job, the user and the record that failed?
  5. How do end users recover and retry from failures? Can they rerun selective parts of the job and/or cancel others? Consider combining point 4’s implementation, so that they can review, address and restart all from one place.
  6. Can or should they just restart the job again? If yes, how does the job sense parts of the dataset a previous run has failed to process? If no, how does the system clean up likely volumes of data to allow the whole job to restart again? Has another job processed those records in the meantime?

These are all great design questions developers and your business analysis need to work through when planning even a single Batch Apex Job implementation. And each in their own right worthy of more technical discussion than I have time to share on this blog entry right now. The main thing is to ensure these questions are asked and your test engineers assert the answers match the software!

To Chain or Not to Chain?

So where does this leave effectively expanding the already considerable scope of the above design, by adding more complexity by linking multiple jobs together? Well first of all why would you do this?

  • Maybe a second jobs dataset does not exist or is incomplete? And as such needs to wait for the first one to create or update data it needs.
  • Maybe you want to have one job spawn sub-jobs to process information in parallel into different buckets or outputs.
  • Maybe you want to kick off a clean up job, following partial failure of a job that cannot support incremental running.

So if you do plan to do utilise this, keep in mind the above questions get quite a lot trickier to answer, design and implement if do start chaining jobs.

One final thought, is if your planning on utilising daisy chaining because you want to process a dataset of mixed record types / custom objects. Thats certainly a use case, though I would personally first consider using a custom Iterator for the job. This would allow you to construct a logic data set that drives a single job which can multiple physical datasets. Thats after all why Salesforce added this often overlooked feature of Batch Apex.

Thanks for reading and please leave comments regarding other implementation considerations you have come across to add to the checklist above!

13 thoughts on “Salesforce Winter’13 Chaining Batch Jobs… With great power…

  1. Many thanks for the checklist. I will use it from today forward.
    We can start the job via SF e-mail service. User attaches 2 documents and each one will populate a different staging table with the file information. It would be nicer for user to have everything ready in one go instead of having 2 steps, as long as these tables are not related, and there is a plan B like re-run it manually in case there is an issue. This would be a good example to use new Winter’13 functionality.

    • Thanks Agustina, great use case and good to see your considering re-run in this case, as it would be poor to ask your users to resend the emails with the attachments. Nice!

  2. Many thanks for the reblog!

  3. Really good points to consider!
    In addition to that, I was not sure if the finish method was included on the processing status. So I was wondering if calling a chasing Batch Job could achieve the “5 queued or active batch jobs governor limit” on an ORG with already 4 jobs queued.

    I did a little testing. The second batch is not in the queue until the first one has completely finished. I added the debug call to the start method on the second batch

    system.debug(‘Number of active Batch Jobs on START METHOD: ‘ + getNumberOfActiveIOrQueuedBatches());

    and I got:
    Number of active Batch Jobs on START METHOD: 1

    Thanks a lot!

  4. Great post Andy.

  5. Hello Andy,

    Thanks for sharing the knowledge. Really great info.

    Do you have any insight on Batch jobs in Group and Professional Edition?? Are we allowed to run Batch ( also chaining ) jobs in GE/PE?.

    Here is a link to a detailed description of what I wanna ask : http://boards.developerforce.com/t5/Apex-Code-Development/Batch-Jobs-on-Group-and-Prof-Edition/m-p/625457

    Any help would be highly appreciated:

    • Thanks your very welcome! As regarding your issue posted on the forum, I’m not 100% sure it is directly related to Batch jobs in PE/GE, perhaps more to do with your code using a feature not supoprted in those environments? Can you produce another button or maybe temporarly make some code ‘global’ (to call via Developer Console) to test it outside of Batch Apex? Also have you checked the Apex Jobs page for errors and/or enabled Debug Logs? Hope some of these suggestions help, please do let me know what it was if you’ve found it already! 🙂

  6. Pingback: Testing a Database.Batchable implementation | Force 201

  7. Pingback: Avoiding “too many concurrent batch jobs” in an InstallHandler | Force 201

  8. Pingback: Automating Org Setup via Process Builder and Metadata API | Andy in the Cloud

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s