Andy in the Cloud

From BBC Basic to Force.com and beyond…

Going Native with the Apex UML Tool and Tooling API!

23 Comments

Screen Shot 2013-10-14 at 22.16.58 Over the last month or so, John M DanielsJames Loghry (a fellow MVP) and myself have been collaborating together in order to accomplish two things; firstly remove the need to utilise a Heroku backend for the Apex UML tool (i first launched for Dreamforce 2013). Secondly and importantly for the first objective, create an Apex wrapper around the Tooling API.

John and I first chatted about collaborating at Dreamforce 2013 to work on the Apex UML tool and move it forward. He later put forward the suggestion that we should consider utilising the Tooling API directly from Apex and do away with the Heroku code. We quickly spun up a branch to try this out, and quite quickly re-architected it to use Visualforce Remoting and started on the plumbing!

Meanwhile James and I had also started thinking about a Apex wrapper around the Tooling API, initially independently, but ultimately decided to join forces! The combination between these two initiatives and the three of us has thus far born excellent fruit, with an initial release of the Apex UML tool running totally native, powered by an early version of the Apex Tooling API…

Apex UML Tool

As before there is a managed package version of the tool for you to install, you can actually upgrade from v1.2 (the Heroku Canvas app based version) directly. Unfortunately if you have v1.2 installed, you will have to upgrade at this stage, as i inadvertently deleted the Connected App from my development org.

  1. Install the package from the install links section here.
  2. Ensure your Apex classes are compiled
  3. Go to Apex Classes page and click the Compile all classes link
  4. Navigate to the Apex UML page
  5. You will see a Remote Site setting message, follow it and then reload the page.
  6. From this point on the functionality and usage is currently, as per the introduction given here.

NativeApexUML

If your interested in ideas we have for future versions of the tool take a look here, ideas always welcome!

Powered by the Apex Tooling API

Initially both James and I had the same thought on building out an Apex wrapper around the Tooling API’s SOAP variant, using the WSDL2Apex tool. However while this approach works quite well for the Metadata API, the Tooling API’s makes extensive use of xsd:extension aka polymorphic XML, for example in the querying of its SObject’s and Symbol Table.

Unfortunately the fact is that the XML deserialiser does not know how to deserialise into different types or how to access base class members. So we reached for the more flexible JSON deserializer. James discovered a cunning combination of manual JSON parsing and typed de-serialisation to get things moving! I’ll leave him to go into more detail on this. You can also read a little more about the strategy decision on the GitHub repository here and current discussions here.

Please note the API is still work in progress,  while the Apex UML tool has given it quite a good shake down so far, keep in mind the API design is still forming. Here are a few examples, starting with code to create an Apex Class

// Create an Apex class in 5 lines!
ToolingAPI toolingAPI = new ToolingAPI();
ToolingAPI.ApexCLass newClass = new ToolingAPI.ApexClass();
newClass.Name = 'HelloWorld';
newClass.Body = 'public class HelloWorld { }';
toolingAPI.createSObject(newClass);

This code queries the Symbol Table for a class and dumps the methods to the debug log…

		ToolingApi toolingAPI = new ToolingApi();
		List<ToolingAPI.ApexClass> apexClasses = (List<ToolingAPI.ApexClass>)
			toolingAPI.query(
				'Select Name, SymbolTable ' +
				'From ApexClass ' +
				'Where Name = \'UmlService\'').records;
		ToolingApi.SymbolTable symbolTable = apexClasses[0].symbolTable;
        for(ToolingApi.Method method : symbolTable.methods)
            System.debug(method.name);

The following is an updated version of a very early version of the API, retrieving a list of Custom Objects and Fields.

		// Constructs the Tooling API wrapper (default constructor uses user session Id)
		ToolingAPI toolingAPI = new ToolingAPI();

		// Query CustomObject object by DeveloperName (note no __c suffix required)
		List customObjects = (List)
			toolingAPI.query('Select Id, DeveloperName, NamespacePrefix From CustomObject Where DeveloperName = \'Test\'').records;

		// Query CustomField object by TableEnumOrId (use CustomObject Id not name for Custom Objects)
		ToolingAPI.CustomObject customObject = customObjects[0];
		Id customObjectId = customObject.Id;
		List customFields = (List)
			toolingAPI.query('Select Id, DeveloperName, NamespacePrefix, TableEnumOrId From CustomField Where TableEnumOrId = \'' + customObjectId + '\'').records;

		// Dump field names (reapply the __c suffix) and their Id's
		System.debug(customObject.DeveloperName + '__c : ' + customObject.Id);
		for(ToolingAPI.CustomField customField : customFields)
			System.debug(
				customObject.DeveloperName + '__c.' +
				customField.DeveloperName + '__c : ' +
				customField.Id);

This code from the Apex UML tool starts a compilation of a given Apex class to later access external references from the ApexClassMember objects SymbolTable.

		// Delete any existing MetadataContainer?
		ToolingApi tooling = new ToolingApi();
		List containers = (List)
			tooling.query(
				'SELECT Id, Name FROM MetadataContainer WHERE Name = \'ApexNavigator\'').records;
		if(containers!=null &&  containers.size()>0)
			tooling.deleteSObject(ToolingAPI.SObjectType.MetadataContainer, containers[0].Id);

		// Create MetadataContainer
		ToolingAPI.MetadataContainer container = new ToolingAPI.MetadataContainer();
		container.name = 'ApexNavigator';
		ToolingAPI.SaveResult containerSaveResult = tooling.createSObject(container);
		if(!containerSaveResult.success)
			throw makeException(containerSaveResult);
		Id containerId = containerSaveResult.id;

		// Create ApexClassMember and associate them with the MetadataContainer
		ToolingAPI.ApexClassMember apexClassMember = new ToolingAPI.ApexClassMember();
		apexClassMember.Body = classes.get(className).Body;
		apexClassMember.ContentEntityId = classes.get(className).id;
		apexClassMember.MetadataContainerId = containerId;
		ToolingAPI.SaveResult apexClassMemberSaveResult = tooling.createSObject(apexClassMember);
		if(!apexClassMemberSaveResult.success)
			throw makeException(apexClassMemberSaveResult);

		// Create ContainerAysncRequest to deploy (check only) the Apex Classes and thus obtain the SymbolTable's
		ToolingAPI.ContainerAsyncRequest asyncRequest = new ToolingAPI.ContainerAsyncRequest();
		asyncRequest.metadataContainerId = containerId;
		asyncRequest.IsCheckOnly = true;
		ToolingAPI.SaveResult asyncRequestSaveResult = tooling.createSObject(asyncRequest);
		if(!asyncRequestSaveResult.success)
			throw makeException(asyncRequestSaveResult);
		asyncRequest = ((List)
			tooling.query(
				'SELECT Id, State, MetadataContainerId, CompilerErrors ' +
				'FROM ContainerAsyncRequest ' +
				'WHERE Id = \'' + asyncRequestSaveResult.Id + '\'').records)[0];

In later blogs between us we will be extending the demo code and more cool stuff!

Next Steps

I’m looking forward to what happens next! Both initiatives have got off to a great start and we will keep incrementing. Certainly its good to have the Apex UML tool to help drive the development and testing of the Apex Tooling API. Follow my fellow collaborators here @JohnDTheMaven and @dancinllama.

23 thoughts on “Going Native with the Apex UML Tool and Tooling API!

  1. I was unable to create an Apex Class using your wrapper in a managed package setting. I added a Github issue:
    https://github.com/afawcett/apex-toolingapi/issues/11

  2. I tried to use the Tooling API to generate Apex classes from a Managed Packaged but miserably failed as soon as I installed the managed packaged in a production org. There seems to be no chance to do that. D’oh! 😦
    Would you fantastic Metadata Api Wrapper be able to do that?

    • Yep i can confirm this is not supported recently did some work in a StackEchange question on it, I think yours? As regards metadata API, yes possible, see source for my declarative rollup tool for an example

  3. Pingback: Episode 3 – Andrew Fawcett on Enterprise Development | Code Coverage

  4. Hi Andrew,

    I was going through some Tooling API exercises and came across this thread of yours. Really got a great insight reading through the posts by you. However I am facing a trouble with the query as in the post above — “Select Name, SymbolTable From ApexClass”. This query works fine from a Java client and Tooling API WSDL but fails when I execute this query in Developer Console or Workbench with the following error:

    No such column ‘SymbolTable’ on entity ‘ApexClass’

    Would like to understand if you faced any similar issue or are aware of this scenario? Any help on this is really appreciated.

    Thanks,
    Jayant

    • Thanks for your kind words, your most welcome. The SOQL only works from the Tooling API Query REST endpoint, you cannot use standard SOQL execution approaches, such as Apex with queries to Tooling API objects. The Apex Tooling API is a set of Apex wrapper classes around the REST based Tooling API. https://github.com/afawcett/apex-toolingapi. There is an example of running this query in the Apex UML tool which uses this library here, https://github.com/afawcett/apex-umlcanvas/blob/master/src/classes/UmlService.cls#L169. Hope this helps, let me know if not.

      • Thanks Andrew for the clarification and pointers. That really helps. However I was just curious about the SOQL consisting SymbolTable being passed from Tooling SOAP or REST client which worked perfectly fine and not from the Apex context. Is this in someway a restriction by Salesforce that SymbolTable field is identifiable only through SOAP or REST clients?

  5. Yeah this kind of custom field data type is not supported in Apex, which i think might be one of the reasons. The other most important one is likely the way the platform accesses the underlying ‘ApexClass’ physical Oracle table is different code paths from an Apex context than from an API context. It does appear that ApexClass objet in Apex and that in Tooling API is one and the same, but my gut feel is the similarity is only at the surface. 😉

  6. Compiled all classes and inserted a new site under remote site settings. I’m still getting a pop up error message “failed”. Using v1.5. Thanks for the tool. Helps when I’m cleaning up a system.

  7. Hi Andy

    I deployed the latest version of the Tooling API wrapper (https://github.com/afawcett/apex-toolingapi), and then tried to run the code you posted in this blog to retrieve custom objects and custom fields. I cannot get it to run, it gives a syntax error: “unexpected token: List”. When I try and comment out the offending lines, I then get this error: “Invalid type: ToolingAPI.CustomObject”. Please advise.

    • Yes this blog code only applies to the old rest based version, this has been archived in a branch in the repo, extract the library from that branch and it should be fine

  8. Thanks Andy , is there a way to get this set up on one of our developer sandboxes

  9. Hi Andy,i have installed your app on my developer login but as soon as i am selecting the apex class in the left hand side i am getting the error for remote site settings,saying unauthorised end point,i have set up a custom domain i my org as i am usng lightening.
    Kindly suggest a way to cater this

    • Have you clicked the button to create the remote side setting? Other reasons that can cause this are in the Readme file

  10. Hey Andy, just wanted to say ‘thanks’ for all your blogs and Salesforce packages, I have made use of several and learned a lot from many more.

    I was so excited to read about this package and looking forward to using it but alas my org is just too large I guess, getting ‘Collection size 3,617 exceeds maximum size of 1,000.’ on the Apex UML tab. Any ideas or am I just stuck?

    • Thank you for the kind words, most appreciated. Sadly it is a know issue. My fellow community contributor on this is working on a new version with a new architecture. His name is John Daniel. If you raise a GitHub issue on the repo you should be able to cc him and ask about this. I think there is already a know issue raised as well perhaps. Sorry I cannot be more help this time.

  11. Getting “Bad id ” when selecting a class, even after adding remote site settings : (https://raghudev-dev-ed–umlcanvas.na46.visual.force.com)

  12. I’m getting the same error as Raghu. Is there anything we can do to get around the ‘bad id’ error?

    Thanks,
    David

    • It must be a regression of some kind. Can you report it on the github repo as an issue and I can try to take a look

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s