… comes great responsibility!
Ok so I am a Spiderman fan, you rumbled me!
But seriously thats the first phrase that popped into my head when I read the Winter’13 release notes regarding this ability. Previously you could not start another Batch Apex job from another Batch Apex job. So now you can! So the next question is why would you want to? Closely followed by should you? Before answering those, lets take a step back and consider what it takes to design and implement a single Batch Apex job…
As you probably already know the transaction semantics of Batch Apex means that you have to give careful consideration to what happens if one or more of the scopes or ‘chunks’ of the job fails and others succeed. Since only the changes to the database made by the failing chunk of the job are rolled back.
A Batch Apex Design and Implementation Checklist
So here is my Batch Apex design check list for developers and business analysis to discuss before a single line of code is written…
- How do users start the job? VF Page, Custom Buttons, Scheduler?
- How do you deliver feedback about the job? When is it queued, in progress and completed. Under high service load the fact that a job is queued can be quite important information from a usability and end user satisfaction perspective. Also an interesting System Admin vs End User consideration is permitting the default error handling by the platform to occur vs catching your exceptions and feeding them back some other way?
- Can multiple jobs of the same type be run at the same time? If yes, how do you manage the dataset each job consumes to avoid overlap with other jobs in a concurrent user environment? If no, how do you plan to stop users running multiple jobs?
- How do users get informed about failures? Chatter posts? Email? Custom Objects? Status fields? Does the information have enough context about the job, the user and the record that failed?
- How do end users recover and retry from failures? Can they rerun selective parts of the job and/or cancel others? Consider combining point 4’s implementation, so that they can review, address and restart all from one place.
- Can or should they just restart the job again? If yes, how does the job sense parts of the dataset a previous run has failed to process? If no, how does the system clean up likely volumes of data to allow the whole job to restart again? Has another job processed those records in the meantime?
These are all great design questions developers and your business analysis need to work through when planning even a single Batch Apex Job implementation. And each in their own right worthy of more technical discussion than I have time to share on this blog entry right now. The main thing is to ensure these questions are asked and your test engineers assert the answers match the software!
To Chain or Not to Chain?
So where does this leave effectively expanding the already considerable scope of the above design, by adding more complexity by linking multiple jobs together? Well first of all why would you do this?
- Maybe a second jobs dataset does not exist or is incomplete? And as such needs to wait for the first one to create or update data it needs.
- Maybe you want to have one job spawn sub-jobs to process information in parallel into different buckets or outputs.
- Maybe you want to kick off a clean up job, following partial failure of a job that cannot support incremental running.
So if you do plan to do utilise this, keep in mind the above questions get quite a lot trickier to answer, design and implement if do start chaining jobs.
One final thought, is if your planning on utilising daisy chaining because you want to process a dataset of mixed record types / custom objects. Thats certainly a use case, though I would personally first consider using a custom Iterator for the job. This would allow you to construct a logic data set that drives a single job which can multiple physical datasets. Thats after all why Salesforce added this often overlooked feature of Batch Apex.
Thanks for reading and please leave comments regarding other implementation considerations you have come across to add to the checklist above!
October 15, 2012 at 11:02 am
Many thanks for the checklist. I will use it from today forward.
We can start the job via SF e-mail service. User attaches 2 documents and each one will populate a different staging table with the file information. It would be nicer for user to have everything ready in one go instead of having 2 steps, as long as these tables are not related, and there is a plan B like re-run it manually in case there is an issue. This would be a good example to use new Winter’13 functionality.
October 28, 2012 at 10:33 am
Reblogged this on Sutoprise Avenue, A SutoCom Source.
October 28, 2012 at 4:04 pm
Thanks Agustina, great use case and good to see your considering re-run in this case, as it would be poor to ask your users to resend the emails with the attachments. Nice!
October 28, 2012 at 4:03 pm
Many thanks for the reblog!
November 2, 2012 at 1:23 pm
Really good points to consider!
In addition to that, I was not sure if the finish method was included on the processing status. So I was wondering if calling a chasing Batch Job could achieve the “5 queued or active batch jobs governor limit” on an ORG with already 4 jobs queued.
I did a little testing. The second batch is not in the queue until the first one has completely finished. I added the debug call to the start method on the second batch
system.debug(‘Number of active Batch Jobs on START METHOD: ‘ + getNumberOfActiveIOrQueuedBatches());
and I got:
Number of active Batch Jobs on START METHOD: 1
Thanks a lot!
November 2, 2012 at 6:34 pm
Nice investigation Jesus, useful info, thanks!
January 10, 2013 at 10:59 am
Great post Andy.
May 29, 2013 at 12:58 pm
Thanks for sharing the knowledge. Really great info.
Do you have any insight on Batch jobs in Group and Professional Edition?? Are we allowed to run Batch ( also chaining ) jobs in GE/PE?.
Here is a link to a detailed description of what I wanna ask : http://boards.developerforce.com/t5/Apex-Code-Development/Batch-Jobs-on-Group-and-Prof-Edition/m-p/625457
Any help would be highly appreciated:
June 1, 2013 at 5:42 pm
Thanks your very welcome! As regarding your issue posted on the forum, I’m not 100% sure it is directly related to Batch jobs in PE/GE, perhaps more to do with your code using a feature not supoprted in those environments? Can you produce another button or maybe temporarly make some code ‘global’ (to call via Developer Console) to test it outside of Batch Apex? Also have you checked the Apex Jobs page for errors and/or enabled Debug Logs? Hope some of these suggestions help, please do let me know what it was if you’ve found it already! 🙂
Pingback: Testing a Database.Batchable implementation | Force 201
Pingback: Avoiding “too many concurrent batch jobs” in an InstallHandler | Force 201
Pingback: Automating Org Setup via Process Builder and Metadata API | Andy in the Cloud
Pingback: Getting your users attention with Custom Notifications | Andy in the Cloud