Andy in the Cloud

From BBC Basic to Force.com and beyond…


46 Comments

Going Native with the Apex UML Tool and Tooling API!

Screen Shot 2013-10-14 at 22.16.58 Over the last month or so, John M DanielsJames Loghry (a fellow MVP) and myself have been collaborating together in order to accomplish two things; firstly remove the need to utilise a Heroku backend for the Apex UML tool (i first launched for Dreamforce 2013). Secondly and importantly for the first objective, create an Apex wrapper around the Tooling API.

John and I first chatted about collaborating at Dreamforce 2013 to work on the Apex UML tool and move it forward. He later put forward the suggestion that we should consider utilising the Tooling API directly from Apex and do away with the Heroku code. We quickly spun up a branch to try this out, and quite quickly re-architected it to use Visualforce Remoting and started on the plumbing!

Meanwhile James and I had also started thinking about a Apex wrapper around the Tooling API, initially independently, but ultimately decided to join forces! The combination between these two initiatives and the three of us has thus far born excellent fruit, with an initial release of the Apex UML tool running totally native, powered by an early version of the Apex Tooling API…

Apex UML Tool

As before there is a managed package version of the tool for you to install, you can actually upgrade from v1.2 (the Heroku Canvas app based version) directly. Unfortunately if you have v1.2 installed, you will have to upgrade at this stage, as i inadvertently deleted the Connected App from my development org.

  1. Install the package from the install links section here.
  2. Ensure your Apex classes are compiled
  3. Assign the Apex UML permission set
  4. Go to Apex Classes page and click the Compile all classes link
  5. Navigate to the Apex UML page
  6. You will see a Remote Site setting message, follow it and then reload the page.
  7. From this point on the functionality and usage is currently, as per the introduction given here.

UPDATE: Step 6 is not required if you have My Domain enabled (since v1.7).

NativeApexUML

If your interested in ideas we have for future versions of the tool take a look here, ideas always welcome!

Powered by the Apex Tooling API

Initially both James and I had the same thought on building out an Apex wrapper around the Tooling API’s SOAP variant, using the WSDL2Apex tool. However while this approach works quite well for the Metadata API, the Tooling API’s makes extensive use of xsd:extension aka polymorphic XML, for example in the querying of its SObject’s and Symbol Table.

Unfortunately the fact is that the XML deserialiser does not know how to deserialise into different types or how to access base class members. So we reached for the more flexible JSON deserializer. James discovered a cunning combination of manual JSON parsing and typed de-serialisation to get things moving! I’ll leave him to go into more detail on this. You can also read a little more about the strategy decision on the GitHub repository here and current discussions here.

Please note the API is still work in progress,  while the Apex UML tool has given it quite a good shake down so far, keep in mind the API design is still forming. Here are a few examples, starting with code to create an Apex Class

// Create an Apex class in 5 lines!
ToolingAPI toolingAPI = new ToolingAPI();
ToolingAPI.ApexCLass newClass = new ToolingAPI.ApexClass();
newClass.Name = 'HelloWorld';
newClass.Body = 'public class HelloWorld { }';
toolingAPI.createSObject(newClass);

This code queries the Symbol Table for a class and dumps the methods to the debug log…

		ToolingApi toolingAPI = new ToolingApi();
		List apexClasses = (List)
			toolingAPI.query(
				'Select Name, SymbolTable ' +
				'From ApexClass ' +
				'Where Name = \'UmlService\'').records;
		ToolingApi.SymbolTable symbolTable = apexClasses[0].symbolTable;
        for(ToolingApi.Method method : symbolTable.methods)
            System.debug(method.name);

The following is an updated version of a very early version of the API, retrieving a list of Custom Objects and Fields.

		// Constructs the Tooling API wrapper (default constructor uses user session Id)
		ToolingAPI toolingAPI = new ToolingAPI();

		// Query CustomObject object by DeveloperName (note no __c suffix required)
		List customObjects = (List)
			toolingAPI.query('Select Id, DeveloperName, NamespacePrefix From CustomObject Where DeveloperName = \'Test\'').records;

		// Query CustomField object by TableEnumOrId (use CustomObject Id not name for Custom Objects)
		ToolingAPI.CustomObject customObject = customObjects[0];
		Id customObjectId = customObject.Id;
		List customFields = (List)
			toolingAPI.query('Select Id, DeveloperName, NamespacePrefix, TableEnumOrId From CustomField Where TableEnumOrId = \'' + customObjectId + '\'').records;

		// Dump field names (reapply the __c suffix) and their Id's
		System.debug(customObject.DeveloperName + '__c : ' + customObject.Id);
		for(ToolingAPI.CustomField customField : customFields)
			System.debug(
				customObject.DeveloperName + '__c.' +
				customField.DeveloperName + '__c : ' +
				customField.Id);

This code from the Apex UML tool starts a compilation of a given Apex class to later access external references from the ApexClassMember objects SymbolTable.

		// Delete any existing MetadataContainer?
		ToolingApi tooling = new ToolingApi();
		List containers = (List)
			tooling.query(
				'SELECT Id, Name FROM MetadataContainer WHERE Name = \'ApexNavigator\'').records;
		if(containers!=null &&  containers.size()>0)
			tooling.deleteSObject(ToolingAPI.SObjectType.MetadataContainer, containers[0].Id);

		// Create MetadataContainer
		ToolingAPI.MetadataContainer container = new ToolingAPI.MetadataContainer();
		container.name = 'ApexNavigator';
		ToolingAPI.SaveResult containerSaveResult = tooling.createSObject(container);
		if(!containerSaveResult.success)
			throw makeException(containerSaveResult);
		Id containerId = containerSaveResult.id;

		// Create ApexClassMember and associate them with the MetadataContainer
		ToolingAPI.ApexClassMember apexClassMember = new ToolingAPI.ApexClassMember();
		apexClassMember.Body = classes.get(className).Body;
		apexClassMember.ContentEntityId = classes.get(className).id;
		apexClassMember.MetadataContainerId = containerId;
		ToolingAPI.SaveResult apexClassMemberSaveResult = tooling.createSObject(apexClassMember);
		if(!apexClassMemberSaveResult.success)
			throw makeException(apexClassMemberSaveResult);

		// Create ContainerAysncRequest to deploy (check only) the Apex Classes and thus obtain the SymbolTable's
		ToolingAPI.ContainerAsyncRequest asyncRequest = new ToolingAPI.ContainerAsyncRequest();
		asyncRequest.metadataContainerId = containerId;
		asyncRequest.IsCheckOnly = true;
		ToolingAPI.SaveResult asyncRequestSaveResult = tooling.createSObject(asyncRequest);
		if(!asyncRequestSaveResult.success)
			throw makeException(asyncRequestSaveResult);
		asyncRequest = ((List)
			tooling.query(
				'SELECT Id, State, MetadataContainerId, CompilerErrors ' +
				'FROM ContainerAsyncRequest ' +
				'WHERE Id = \'' + asyncRequestSaveResult.Id + '\'').records)[0];

In later blogs between us, we will be extending the demo code and more cool stuff!

Next Steps

I’m looking forward to what happens next! Both initiatives have got off to a great start and we will keep incrementing. Certainly, its good to have the Apex UML tool to help drive the development and testing of the Apex Tooling API. Follow my fellow collaborators here @JohnDTheMaven and @dancinllama.


31 Comments

Spring’14 Visualforce Remote Objects Introduction

Salesforce have provided further support for JavaScript in the upcoming Spring’14 release. With a new flavour of the popular Visualforce Remoting facility. Visualforce Remote Objects is a pilot feature I have been trying out in a pre-release org. It’s aim is effectively to make performing database operations, like create, read, update and delete in JavaScript as easy as possible without the need for Apex, and without consuming the orgs daily API limits. This blog introduces the feature and contrasts it with its very much still relevant Visualforce Remoting brother.

Consider a master detail relationship between WorkOrder__c and WorkOrderLineItem__c. The first thing you need to do is declare your intent to access these objects via JavaScript on your Visualforce page with some new tags.

	<apex:remoteObjects >
	    <apex:remoteObjectModel name="WorkOrder__c" fields="Id,Name,AccountName__c,Cost__c"/>
	    <apex:remoteObjectModel name="WorkOrderLineItem__c" fields="Id,Name,Description__c,Hours__c,WorkOrder__c"/>
	</apex:remoteObjects>

The following JavaScript can now be used to access the JavaScript objects the above automatically injects into the page (note that while not shown there is further control over object and field naming, i used the defaults here).

It is an async API, so you provide a function call back to handle the result of the operations (create, update, delete and select are supported), it does not throw exceptions. In the example below if the insert of the Work Order master record is successful the child is then inserted. Note the event parameter actually contains the Id of the inserted record.

		function doSomethingJS(answer)
		{
			// Create work order
			var workOrder = new SObjectModel.WorkOrder__c();
			workOrder.set('AccountName__c','Hitchhikers.com');
			workOrder.set('Cost__c', answer * 100);
			workOrder.create(function(error, result, event)
				{
					// Success?
					if(error == null)
					{
						// Create work order line item
						var workOrderLineItem = new SObjectModel.WorkOrderLineItem__c();
						workOrderLineItem.set('Description__c', 'Answering the question');
						workOrderLineItem.set('Hours__c', answer);
						workOrderLineItem.set('WorkOrder__c', result[0]);
						workOrderLineItem.create(function(error, result, event)
							{
								// Errors?
								if(error!=null)
									alert(error);
								else
									alert('Success');
							} );
						return;
					}
					// Display error
					alert(error);
				} );
		}

As per the documentation, this means you no longer need an Apex controller to do this. You can also query using this object model as well, as per this the pre-release documentation querying a Wharehouse object. However before we cast aside our Apex thoughts, lets look at what the above would like implemented via a Remote Action.

		function doSomethingApex(answer)
		{
			// Create work order and line item via Apex
			Visualforce.remoting.Manager.invokeAction(
				'{!$RemoteAction.RemoteObjectDemoController.doSomething}',
				answer,
				function(result, event){
					alert(event.status ? 'Success' : event.message);
				});
		}

The following Apex code implements the remote action.

	@RemoteAction
	public static void doSomething(Integer answer)
	{
		WorkOrder__c workOrder = new WorkOrder__c();
		workOrder.AccountName__c = 'Hitchhikers.com';
		workOrder.Cost__c = answer * 100;
		insert workOrder;
		WorkOrderLineItem__c workOrderLineItem = new WorkOrderLineItem__c();
		workOrderLineItem.Description__c = 'Answering the question';
		workOrderLineItem.Hours__c = answer;
		workOrderLineItem.WorkOrder__c = workOrder.Id;
		insert workOrderLineItem;
	}

On the face of it, it may appear both accomplish the same thing, but there is a very important and critical architecture difference between them. To help illustrate this i created a page to invoke both of these options. Screen Shot 2014-01-22 at 08.48.56

Screen Shot 2014-01-22 at 09.03.07While also creating a strategically placed  Validation Rule on the Work Order Line item, one which would fail the insert if anything other than 42 was entered in the UI.

So given the validation rule in place, lets perform a test.

  1. Ensure there are no Work Order records present
  2. Enter an invalid value, say 32, click the buttons, observe the expect error
  3. Correct the value to 42, click the button again and observe the outcome on the database.
  4. The expected result is one Work Order record with a single Work Order Line Item record.

Testing the ‘Do Something (Apex)’ Button

After going through the test above, the button initially gives the error as expected…

Screen Shot 2014-01-22 at 09.00.05

When the value is corrected and button pressed again, the result is this…

Screen Shot 2014-01-22 at 08.54.07

The test passed.

Testing the ‘Do Something (JavaScript)’ Button

Going through the tests again, this button initially gives the error as expected…

Screen Shot 2014-01-22 at 09.00.25

When the value is corrected and button pressed again, the result is this…

Screen Shot 2014-01-22 at 08.56.54
While Visualforce Remote Objects technically performed as I believe Salesforce intended, this functional tests expectation failed. Since we have two Work Order records, one with no Work Order Lines and one that is what we expected. So why did this additional rogue Work Order record get created, what did we do wrong?

The answer lies in the scope of the database transaction created…

  • In the Visualforce Remote @RemoteAction use case the platform automatically wraps a transaction around the whole of the Apex code and rolls back everything if an error occurs, you can read more about this here.
  • In the Visualforce Remote Objects use case the database transaction is only around the individual database operations not around the whole JavaScript function. Hence by the time the Work Order Line Item fails to insert the Work Order has already been committed to the database. The user then corrects their mistake and tries again, hence we end up with two Work Orders and not one as the user expected.

Since Apex transaction management is so transparent most of the time (by design), its likely that the same assumption might be made of Remote Objects, however as you can see its not a valid one. Those of you that know core thoughts on patterns will also be thinking something else at this point, that presents a potentially even more compelling reason to be watchful over what logic you implement this way…

Separation of Concerns

As you may have gathered by now if you’ve been reading my blog for the last year, my other passion is Apex Enterprise Patterns. A key foundation of this is Separation of Concerns or SOC for short. SOC sees us layer our code so that aspects such as business logic are located in the same place, easily accessible and reusable (not mention easily testable) from other existing and future parts of our applications, such as API’s, Batch Apex etc.

While the above example is not that complex it illustrates when code in your JavaScript layer might start to become business logic and if so something you should ideally (if not solely for the reason above) consider keeping in your Service Layer and accessing via JavaScript Remoting @RemoteAction.

Summary

Despite this new feature the use of Visualforce Remoting with @RemoteAction Apex should still very much factor in your decision making.  And nor despite the issue above should we necessarily let the lack of transaction management count against our use of Visualforce Remote Objects either.

For sure this will become the best and lightest way to perform rapid client side querying without impacting API limits, and I am sure the alias feature (see docs) will be welcome to those preferring more elegant AngularJS or other bindings. All very nice! Furthermore if you are developing client logic that is essentially doing simple record editing then by all means let your Apex Triggers (Domain Layer) do its job and enforce the validity of those records.

Just keep in mind when your starting to write more and more JavaScript code that is orchestrating the updating, inserting or deleting of a set of  related records, you really need to be sure be sure you and your testers understand the transaction scope differences between the two approaches described here or switch over to Apex and let the platform manage the transaction scope for you. Finally its worth keeping in mind if you don’t have a client side automated testing strategy your automatically adding manual testing overhead to your backlog.

The full source code for this blog can be found here.

UPDATE: Since publishing this blog, Salesforce documentation team have written up an excellent additional topic covering some best practices and outlining the transactional differences described above. Its really great to see this type of content appearing in the standard Salesforce developer documentation, thank you Salesforce!

Other Notes and Observations

Here are some final points of note, given this is still pilot hopefully of use to Salesforce as feedback.

  • It does buffer requests to the server like Visualforce Remoting, very cool!
  • It does not complain when you set the wrong field or get the name wrong, like SObject.put and SObject.get do, maybe to be expected, though since we gave it the field list might have been nice?
  • It differs slightly from Dynamic Apex, it uses SObject.put, here WorkOrder__c.set is used
  • Success is expressed in ‘error’ being null, not quite what i expected, and different from Visualforce Remoting.
  • It does not implement field defaulting
  • Errors need to be hanlded by callbacks (as per VF Remoting), though by default errors are not emitted to the JavaScript console like Visualforce Remoting
  • References to fields on via the apex:remoteObjectModel fields attribute do not seem to surface as dependencies on the fields, which would be good given how soft references within the JavaScript are.