Andy in the Cloud

From BBC Basic to Force.com and beyond…

Tag Archives: apex design


by 7 Comments

Apex Enterprise Patterns – Separation of Concerns

Software is often referred to as living thing that changes and evolves over time. As with complex organisms that are expected to endure and evolve over time, it is important to understand what role each part of the organism plays. To keep the organism growing while still remaining strong and support its evolution or even rapid growth through in take of more resources. And so the same is true for complex applications, often categorised as Enterprise level, reflecting the complexity and scale of importance to the ‘eco-system’ the application lives within. So what can we learn from this when engineering such applications?

Separation of Concerns

Complex code gets out of hand when you don’t partition it properly and it becomes heavily intermixed. Making it hard to work from, error prone and worst still hard to learn, only serving to worsen the problem as you bring new developers into the party! Avoiding this to some degree at the basic level is often referred to as ‘code re-use’. Which is of course a very good thing as well! The act of creating modules or libraries to share common calculations or processes amongst different parts of your application.

Is SOC just a posh word for ‘code re-use’ then?

If your considering SOC properly, your doing some upfront thinking about the internal plumbing of your application (down to class naming conventions and coding guidelines) that you feel will endure and hopefully be somewhat self describing to others going forward. Rather than the usual adhoc approach to code re-use which sees fragments of code get moved around once two or more areas start to need it. Often just placed in MyUtil classes or some other generic dumping area. Which is fine, and certainly recommended vs copy and paste!

So what are the benefits of SOC?

At a high level applications have storage, logic and one or more means to interact with them, be that by humans and/or other applications. Once you outline these, you can start to define layers within your application each with its own set of concerns and responsibilities to the application and other layers. Careful consideration and management of such layers is important to adopting SOC.

  • Evolution. Over time, as technology, understandings and requirements (both functional and technical) evolve, any one of these layers may need to be significantly extended, reworked or even dropped. Just look at UI technology over the last 10 years as a prime example of this.
  • Impact Management. Performing work or even dropping one or more of these layers  should not unduly impact (if at all) the others, unless the requirements of course lead to that.
  • Roles and Responsibility. Each layer has its own responsibility and should not drop below or over extend that responsibility. For example dropping one client technology / library in favour of another, should not mean loosing business logic, as this is the responsibility of another layer. If the lines of responsibility get blurred it erodes the purpose and value of SOC.

The typical SOC layers are shown below. On the Force.com platform, there are two distinct approaches to development, both can be used standalone or to complement each together. The Declarative and traditional Coding style approaches, broadly speaking, fit into the standard SOC layers like so…

why CONSIDER SOC on Force.com?

One of the key benefits of Force.com, is its declarative development model, the ability to create objects, fields, layouts, validation rules, workflows, formula fields etc without a single line of code. So these for sure should be your first port of call, typically if your app is heavily data centric in its use cases, you’ll get by with delivering a large portion of your application this way! So while not code, what you can achieve with declarative development is still very much an architecture layer in your application, one that I will talk about more later.

So if your app is process centric and/or getting pushed to implement more complex calculations, validations or richer UI experiences. You’ll be dipping into the land of Apex and thus code. Out of all the logic you invest developer and testing hours in, it is the business logic that you should be most concerned about protecting. Force.com provides many places to place Apex code, Triggers, VF Controllers, Web Services, REST Services, Batch Apex, Email Handlers, the list goes on. We will start to explore some rules for defining what goes into these in the further parts of this series. For now consider the following…

  1. If you had to replace or add another UI to your app (e.g. Mobile) how much of that code would you be rewriting / porting that is effectively nothing to do with the UI’s responsibility? But more to do with inserting, updating, validating and calculating in your app.
  2. If you wanted to provide a public facing API (such as REST) to your logic how would you do this? What parts of your existing code base would you call to implement the API? Action methods in your VF controllers, passing in bits of view state?!?!
  3. If you got asked to scale your application logic via Batch Apex as well continue to provide an interactive experience (for smaller volumes) via your existing UI (VF controllers). How would you share logic between the two to ensure the user gets consistant results regardless.
  4. Visualforce provides a means for you to partition your code via MVC (a form of SOC for client development), but simply using Visualforce and Controllers does not guarantee you have implemented it.  For example how complex are your action methods in your controllers? Do you have any code does more than just deal with handling content to and from the user?
  5. How easily can new developers find their way around your code base? How much time would they need to learn where to put new code and where to find existing behaviour?

Depending on how you have or plan to partition / placed your code you may already be in good shape to tackle some of the above. If not or just curious, hopefully the upcoming articles will help shed a bit of light for further thought.

Summary

This blog entry is the first describing Enterprise Application Architecture patterns, particularly focused on applying them on the Force.com platform. If you attended Dreamforce 2012 this year, you may have caught a presentation on this topic. In the meantime, if you missed the session and fancy a preview of the patterns that help support SOC. You can find the Github repo here along with slides and a recent re-run recording I did. For now, please feel free to comment, make suggestions and requests for topics / use cases you would like to be covered and I’ll do my best to include and answer them!

LINKS

Here are a few links to other resources, the DF12 session and discussions of late that relate to this post. These will give you some foresight into upcoming topics I’ll be discussing here. Enjoy!


by 25 Comments

Handling Office Files and Zip Files in Apex – Part 1

I recently found a number of developers asking questions about Zip file handling in Apex, which as most find out pretty soon, does not exist. Statement governor concerns aside, nor is there any binary data types to make implementing support for it possible. Most understandably most want to stay on platform and not call out to some external service for this. And rightly so!

Another developer and I recently had the same need before the API provider decided to handle compression over HTTP. However while subsequently doing some work to get the Metadata API working from Apex, which also requires zip support in places. I came across a partial native solution and in addition to that I’ve also had in mind to try out some day the JSZip Javascript library.

And so I decided to write some blog entries covering these two approaches and hopefully help a few people out…

Approach A: Using Static Resources and Metadata API

A static resource is as the name suggests some that changes rarely in your code, typically used for .js, .css and images. Salesforce allows you to upload these type of files individually or within a zip file. Using its development UI and tools.

Allowing you then reference in your Visualforce pages like this

$URLFOR($Resource.myzip, '/folder/file.js').

Further more you can also do this in Apex.

PageReference somefileRef = 
   new PageReference('/resource/myzip/folder/file.js');
Blob contentAsBlob = somefileRef.getContent();
String contentAsText = contextAsBlob.toString();

So in effect Salesforce does have a built in Zip handler, at least for unzipping files anyway. The snag is, uploading a zip file dynamically once your user has provided it to you. If you review the example code that calls the Metadata API from Apex you might have spotted an example of doing just this. To create a Static Resource you can do the following.

MetadataService.MetadataPort service = createService(); 
MetadataService.StaticResource staticResource = 
    new MetadataService.StaticResource();
staticResource.fullName = 'test';
staticResource.contentType = 'text';
staticResource.cacheControl = 'public';
staticResource.content = 
   EncodingUtil.base64Encode(Blob.valueOf('Static stuff'));
MetadataService.AsyncResult[] results = 
    service.create(
      new List; { staticResource });

They key parts are the assignment of the ‘contentType’ and ‘content’ members. The ‘content’ member is a Base64 encoded string. In the example above its a static peace of text, however this could easily be the zip file the user has just uploaded for you and then Base64 encoded using the Apex EncodingUtil. You also need to set the ‘contentType’ to ‘application/zip’.

This example presents a user with the usual file upload, you will also need to ask or know what the path of the file is you want to extract. Of course if you know this or can infer it e.g. via file extension such as .docx or .xslx, which uses the Office Open XML format, then your good to go.

I’ve shared the code for this here.

Known Issues: If your zip file contains files with names that contain spaces or special characters, I have found issues with this approach. Since this also applies to URLFOR and generating links in VF pages, I plan to raise a support case to see if this is a bug or limitation. Also keep in mind the user will need to have the Author Apex permission enabled on their profile to call the Metadata API.

Summary and Part 2

This blog entry covered only unzip for known file types, but what if you wanted to have the code inspect the zip contents? Part 2 will extend some of the components I’ve been using in the apex-mdapi to use the JSZip library, examples of which are here and here. In extending those components, I’ll be looking at making them use JavaScript Remoting and local HTML5 file handling to unzip the file locally in the page and transmit the files to the server via an Apex Interface the controller implements. Likewise I want to use the same approach to prepare a zipped file by requesting zip content from the server.


by 14 Comments

Salesforce Winter’13 Chaining Batch Jobs… With great power…

… comes great responsibility!

Ok so I am a Spiderman fan, you rumbled me!

But seriously thats the first phrase that popped into my head when I read the Winter’13 release notes regarding this ability. Previously you could not start another Batch Apex job from another Batch Apex job. So now you can! So the next question is why would you want to? Closely followed by should you? Before answering those, lets take a step back and consider what it takes to design and implement a single Batch Apex job…

As you probably already know the transaction semantics of Batch Apex means that you have to give careful consideration to what happens if one or more of the scopes or ‘chunks’ of the job fails and others succeed. Since only the changes to the database made by the failing chunk of the job are rolled back.

A Batch Apex Design and Implementation Checklist

So here is my Batch Apex design check list for developers and business analysis to discuss before a single line of code is written…

  1. How do users start the job? VF Page, Custom Buttons, Scheduler?
  2. How do you deliver feedback about the job? When is it queued, in progress and completed. Under high service load the fact that a job is queued can be quite important information from a usability and end user satisfaction perspective. Also an interesting System Admin vs End User consideration is permitting the default error handling by the platform to occur vs catching your exceptions and feeding them back some other way?
  3. Can multiple jobs of the same type be run at the same time? If yes, how do you manage the dataset each job consumes to avoid overlap with other jobs in a concurrent user environment? If no, how do you plan to stop users running multiple jobs?
  4. How do users get informed about failures? Chatter posts? Email? Custom Objects? Status fields? Does the information have enough context about the job, the user and the record that failed?
  5. How do end users recover and retry from failures? Can they rerun selective parts of the job and/or cancel others? Consider combining point 4’s implementation, so that they can review, address and restart all from one place.
  6. Can or should they just restart the job again? If yes, how does the job sense parts of the dataset a previous run has failed to process? If no, how does the system clean up likely volumes of data to allow the whole job to restart again? Has another job processed those records in the meantime?

These are all great design questions developers and your business analysis need to work through when planning even a single Batch Apex Job implementation. And each in their own right worthy of more technical discussion than I have time to share on this blog entry right now. The main thing is to ensure these questions are asked and your test engineers assert the answers match the software!

To Chain or Not to Chain?

So where does this leave effectively expanding the already considerable scope of the above design, by adding more complexity by linking multiple jobs together? Well first of all why would you do this?

  • Maybe a second jobs dataset does not exist or is incomplete? And as such needs to wait for the first one to create or update data it needs.
  • Maybe you want to have one job spawn sub-jobs to process information in parallel into different buckets or outputs.
  • Maybe you want to kick off a clean up job, following partial failure of a job that cannot support incremental running.

So if you do plan to do utilise this, keep in mind the above questions get quite a lot trickier to answer, design and implement if do start chaining jobs.

One final thought, is if your planning on utilising daisy chaining because you want to process a dataset of mixed record types / custom objects. Thats certainly a use case, though I would personally first consider using a custom Iterator for the job. This would allow you to construct a logic data set that drives a single job which can multiple physical datasets. Thats after all why Salesforce added this often overlooked feature of Batch Apex.

Thanks for reading and please leave comments regarding other implementation considerations you have come across to add to the checklist above!