Since I created the Apex Metadata API I get to help all sorts of folks building many different and cool things with it. One that is quite common is to automate post package installation tasks such as updating layouts or picklist values, which are not auto upgraded by the platform.
What makes these use cases a little awkward and less user friendly is that the user installing the package has to perform at least one manual post install step. That is to create the Remote Site Setting to ironically allow the native Apex code to call a native SOAP or REST platform API (see Idea here to remove this need).
What makes it more tricky, is the URL the package install user has to provide for the Remote Site Setting is quite specialised, it has to contain the orgs instance name, the package namespace (if your using the Apex Metadata API from a Visualforce page) and potentially the orgs custom domain name.
As those that follow my blog will know my Declarative Rollup Summary Tool uses the Apex Metadata API and has the same requirement. Since i have defined the Apex Exception User on my package, i get an email notification each time an uncaught exception occurs in the tool. Despite putting instructions on a Welcome page in the tool to setup the Remote Site Setting, i can see by the frequency of System.CalloutException exception i get that this manual post install step is initially being missed.
About the same time i decided to raise an enhancement to remind me to look for a solution or at least better way to indicate this step to the user. I found this question Accessing Metadata API from JavaScript on StackExchange, referencing this earlier one Dynamically set Remote Site in Apex. The infamous Mr Fox had found the answer! Which was to call the Metadata API initially via JavaScript (which is not bound by the Remote Site Setting check). Thankfully since Summer’13 Salesforce supports making API calls from the Visualforce domains, otherwise this would not have been possible due to the browsers cross domain checking! Phew!
I already had configured on the landing tab of the tool a Welcome Visualforce page, so i decided to put the code to auto create the Remote Site Setting here. Note that since Summer’14 you can now also configure a post install page on your package, this would also be a good place. Here is my implementation of the answer given, with a few tweaks, UI and error handling.
You can see that I’ve offered the user two ways, manual and pressing the button. I chose to do this, so that admins would still be aware of what the tool was doing to the configuration in the org, which i feel when using the Metadata API is quite important. Once they press the button it makes the JavaScript callout to the Metadata API and the results are passed back to an apex:actionFunction to refresh the page.
First I wrote some code to test the API connection, which basically made a callout to the Apex Metadata API and caught any exception. I chose to use the listMetadata API call, but it could have been any. Its not the response we are interested in here, it is if it throws an exception or not. Unfortunately there is no language neutral way of checking the lack of Remote Site Setting, so for now the assumption is any CalloutException is a result of this.
global static Boolean checkMetadataAPIConnection() { try { MetadataService.MetadataPort service = new MetadataService.MetadataPort(); service.SessionHeader = new MetadataService.SessionHeader_element(); service.SessionHeader.sessionId = UserInfo.getSessionId(); List<MetadataService.ListMetadataQuery> queries = new List<MetadataService.ListMetadataQuery>(); MetadataService.ListMetadataQuery remoteSites = new MetadataService.ListMetadataQuery(); remoteSites.type_x = 'RemoteSiteSetting'; queries.add(remoteSites); service.listMetadata(queries, 28); } catch (System.CalloutException e) { return false; } return true; }
Next I wrote a Visualforce Controller to call this and also calculate the URL needed by the Remote Site Setting. I did this via the Host HTTP Header, as this will include in a subscriber org all of the above attributes.
public with sharing class WelcomeController { public String Host {get;set;} public String MetadataResponse {get;set;} public Boolean MetadataConnectionWarning {get;set;} public PageReference checkMetadataAPIConnection() { // Get Host Domain Host = ApexPages.currentPage().getHeaders().get('Host'); // Attempt to connect to the Metadata API MetadataConnectionWarning = false; if(!RollupService.checkMetadataAPIConnection()) { // ... See full GitHub code ... MetadataConnectionWarning = true; } return null; } public PageReference displayMetadataResponse() { // ... See full GitHub code ... return null; } }
Finally the Visualforce page with the JavaScript callout to the Metadata API in it! The code constructs the SOAP XML, makes the call and parses the result for any errors. Before calling an apex:actionFunction to refresh the page (only the key parts of the full page are shown below).
<apex:page controller="WelcomeController" tabStyle="LookupRollupSummary__c" standardStylesheets="true" action="{!checkMetadataAPIConnection}"> <script> function createRemoteSite() { // Disable button document.getElementById('createremotesitebtn').disabled = true; // Calls the Metdata API from JavaScript to create the Remote Site Setting to permit Apex callouts var binding = new XMLHttpRequest(); var request = '<?xml version="1.0" encoding="utf-8"?>' + '<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">'+ '<env:Header>' + '<urn:SessionHeader xmlns:urn="http://soap.sforce.com/2006/04/metadata">' + '<urn:sessionId>{!$Api.Session_ID}</urn:sessionId>' + '</urn:SessionHeader>' + '</env:Header>' + '<env:Body>' + '<createMetadata xmlns="http://soap.sforce.com/2006/04/metadata">' + '<metadata xsi:type="RemoteSiteSetting">' + '<fullName>dlrs_mdapi</fullName>' + '<description>Metadata API Remote Site Setting for Declarative Rollup Tool (DLRS)</description>' + '<disableProtocolSecurity>false</disableProtocolSecurity>' + '<isActive>true</isActive>' + '<url>https://{!Host}</url>' + '</metadata>' + '</createMetadata>' + '</env:Body>' + '</env:Envelope>'; binding.open('POST', 'https://{!Host}/services/Soap/m/31.0'); binding.setRequestHeader('SOAPAction','""'); binding.setRequestHeader('Content-Type', 'text/xml'); binding.onreadystatechange = function() { if(this.readyState==4) { var parser = new DOMParser(); var doc = parser.parseFromString(this.response, 'application/xml'); var errors = doc.getElementsByTagName('errors'); var messageText = ''; for(var errorIdx = 0; errorIdx < errors.length; errorIdx++) messageText+= errors.item(errorIdx).getElementsByTagName('message').item(0).innerHTML + '\n'; displayMetadataResponse(messageText); } } binding.send(request); } </script> <body class="homeTab"> <apex:form id="myForm"> <apex:actionFunction name="displayMetadataResponse" action="{!displayMetadataResponse}" rerender="myForm"> <apex:param name="metadataResponse" assignTo="{!metadataResponse}" value="{!metadataResponse}"/> </apex:actionFunction> <apex:sectionHeader title="Declarative Rollups for Lookups" subtitle="Welcome"/> <apex:pageMessages /> <apex:outputPanel rendered="{!MetadataConnectionWarning}"> <input id="createremotesitebtn" type="button" onclick="createRemoteSite();" value="Create Remote Site Setting"/> </apex:outputPanel> </apex:form> </body> </apex:page>
Its worth noting that this solution would also apply to configuring a Remote Site for other types of Salesforce API callouts, such as Apex Tooling API, since its just the Domain part of the URL that needs to be added. Note you may need to add more Remote Sites if your also planning on calling from a Batch Apex context for example.
I’m really pleased with this implementation, but it is a little two baked into the rollup tool at present. This would make a great addition to the Apex Metadata API library (or maybe something more standalone) as a Visualforce Component for example, so that all you need to do is include the component on your welcome or post install pages to use it. Something for another day or a fellow open source developer to think about perhaps!?!
Security Review Notes: This approach has not gone through Security Review as yet. My feeling is that it should pass as there is president for calling Salesforce SOAP/REST API’s from JavaScript already and indeed was the main reason for Salesforce enabling the API endpoints from a VF domain as noted above in Summer’13. Nor does this approach bypass the CRUD, FLS or Permissions needed by such API’s, e.g. you still need to have Author Apex to permit Metadata API calls regardless of where they are called from. Finally the design approach here is user driven (e.g. they have to press abutton) rather than automated on page load (which they do generally discourage of course).
July 29, 2014 at 7:38 am
Be interesting to see if this technique passes security review, I’d really hope so since it’s informing and requesting permission from the administrator.
July 29, 2014 at 7:53 am
Yes, good point, i’ll add a note to this comment in the blog. I personally would expect so, since it is using the Salesforce API’s, its in the same domain, and the SF API’s will still require the user to be Admin (for Metadata API anyway), and there are also plenty of SF examples of using other SF SOAP/REST API’s from JavaScript (https://developer.salesforce.com/blogs/developer-relations/2013/06/calling-the-force-com-rest-api-from-visualforce-pages-revisited.html). But will update this blog if there are any issues, next time i submit the tool! 🙂
July 29, 2014 at 7:34 pm
Wondering even if salesforce would have not allowed to Call directly from Visualforce due to cross domain ,worth trying with /services/proxy and generating client side proxy and making the request .
July 29, 2014 at 8:45 pm
No need for this, the API’s are available from the VF domains.
August 28, 2014 at 2:49 am
Can you please let me know if it passes security review.
August 28, 2014 at 5:44 am
Sure will do, I am not in a position to submit my app using his yet, but will follow up here when I do. In the meantime see my thoughts on this in the blog.
September 13, 2014 at 11:59 am
Hi Andy,
This is good solution. but how can someone create components via Post-Install if somebody needs to go to the VF to create remote site for host?
Post install script would run before admin goes to this VF page. And in post-install you cannot create remote site via meta-data API, because the remote site to Host is missing. Sounds like catch 22 to me.
Does this make sense or am I missing something here?
Mitesh
September 13, 2014 at 1:00 pm
No your not missing something. Instead if a post install script you can since summer’14 define a post install page on your package (i think during upload process). I have not tried it yet but maybe this is a better place?
September 13, 2014 at 12:02 pm
Other solution might be to include all Salesforce instance in RemoteSite and package that. That is NOT at all what I want to do.
September 13, 2014 at 1:01 pm
Not not at all, good news upgrades can include such scripts assuming user has setup after initial install. In the blog I think I linked to Idea to white list Salesforce servers, please help and up vote. Thanks for your comments! 🙂
November 10, 2014 at 10:39 am
Andrew,
I’m really interested if you have pass the security review. Could you tell me?
Pablo
November 10, 2014 at 12:23 pm
I have yet to go through this process with this solution. My speculation is that it will pass, but i cannot confirm as yet. I have used this approach in my Decalrative Lookkup Rollup Summary tool, which i do plan to put through security review again in the future. I’ll update when i do.
November 18, 2014 at 4:03 pm
Brilliant! Do you have a sample code of retrieving page layouts in an installed managed package?
November 18, 2014 at 6:04 pm
See the MetadataServiceExamples.cls for a layout retrieve. The secret is in the naming of the layout it needs to include the managed package namespace in the name. Best approach is to login to org using Salesforce Developer Workbench and use the Metadata menu option to list all the layouts in the org and observe the layout names it shows here to use when calling the retrieveMetadata operation.
November 18, 2014 at 6:04 pm
There was also a question in the GitHub repo Issues list about this as well as i recall. Let me know if your struggling to find these things.
November 19, 2014 at 1:05 pm
Thanks for your help! I will give it a shot…
November 19, 2014 at 2:31 pm
I was able to read the layout, but it seems to be the old version of the layout (from the original 1.0 version of the package), not the updated layout from the latest 1.1 version of the package, and in the workbench I only see one version of the layout. I was hoping to see all versions. bummer.
What I’m trying to do is somehow read the latest 1.1 version of the layout, make a copy of the original/old 1.0 layout (because the original is part of a managed package thus not editable) and then update this copy. Doesn’t look like I’ll be able to get to the 1.1 layout…
November 20, 2014 at 8:56 am
Layouts don’t get upgraded when you install newer versions of your package. As per the the docs, Layouts are not upgradable. http://www.salesforce.com/us/developer/docs/packagingGuide/Content/packaging_packageable_components.htm. So sadly no, only in brand new installs will you see the latest layouts. This is a platform behaviour we cannot control sadly.
December 30, 2014 at 4:36 pm
This is same Mitesh harnessing you on your other blog. Thank you for your prompt reply. We have implements this workaround to create few fieldsets in subscriber org when they go to “Welcome” page. While this works in EE org, I am afraid it won;t work in PE org. if they do not have API enabled.
Have you encountered this with your PE customers?
December 30, 2014 at 7:45 pm
Andy, please see below API documentation, it clearly says PE and GE will not support Metadata API. Is there any work around to this?
So many work around just to create PricebookEntry field sets!!
http://www.salesforce.com/us/developer/docs/packagingGuide/Content/dev_packages_api_access.htm
December 30, 2014 at 8:26 pm
Sadly not, i raised a support case to double check myself.
December 30, 2014 at 8:26 pm
Yes, it has been raised before, here, unfortunately Salesforce don’t support this API in PE.
December 30, 2014 at 8:31 pm
Thank you Andy. We will have some message for PE org. Wish you a very Happy New Year!!
December 31, 2014 at 6:24 pm
Your welcome, happy new year to you as well!
Pingback: Unable to update metadata (picklist values etc) after post install script is executed | DL-UAT
July 2, 2015 at 9:22 am
Andy – Is it possible to automate 1) package installation and 2) other manual tasks such as updating layouts or picklist values, field level security using the Apex MetaData API you created in the managed package production environment? THanks in advance.
July 4, 2015 at 5:29 am
Yes for sure, take a look at MetadataServiceExamples.cls in the repo.
January 11, 2016 at 1:03 pm
Hi Andy,
I used the above solution for creation of remote site settings and it works fine in firefox and chrome but in case of IE, the above solution does not seem to work perfectly. The problem is on clicking of ‘Create Remote Site Settting’ button, method named ‘displayMetadataResponse’ is not getting called.
As I further analyzed the problem, then found that in case of IE ‘this.readyState’ is undefined and even ‘this.response’ is undefined so following code block is not working with Internet Explorer :
if(this.readyState==4) {
var parser = new DOMParser();
var doc = parser.parseFromString(this.response, ‘application/xml’);
var errors = doc.getElementsByTagName(‘errors’);
var messageText = ”;
for(var errorIdx = 0; errorIdx < errors.length; errorIdx++)
messageText+= errors.item(errorIdx).getElementsByTagName('message').item(0).innerHTML + '\n';
displayMetadataResponse(messageText);
}
can you suggest the workaround for this ?
Thanks
January 11, 2016 at 5:22 pm
I am not sure what the problem is here, I am on a Mac so cannot easily try. There ought to be general http comms examples for ie to compare though. Might also be worth asking in a more general way on stackexchange. Worth checking you don’t have any backwards compatibility settings on or page headers controlling it. This should be an industry standard approach for http comms I think.
January 18, 2016 at 11:09 am
Hi Andy,
Can this possible at the time of installation i.e. in onInstall() method of Installhandler Interface. Because giving user to set the endpoint is not great idea. If it is possible can you suggest way using your code.
Thanks,
Sandy
January 19, 2016 at 6:52 pm
Sadly not, as the remote site setting to permit the call out will not be present
April 22, 2016 at 9:59 am
Hello Andy ,
From where can i get please the Tooling api TEST classes? I went to the github repository but found only classes
Thanks for your Help
September 27, 2018 at 1:06 pm
@Andrew, the idea says it is delivered, but calling Metadata from Apex still requires Remote Site. Here is the idea you posted originally: https://success.salesforce.com/ideaView?id=08730000000l7iEAAQ
Am I misreading it? Not sure what is “delivered”?
This will be big win for all fellow developers and ISVs. Can you please cross check?
December 15, 2018 at 1:11 pm
No problem and thank you! Per the docs (whcih have been updated) “You don’t need a RemoteSiteSetting for your org to interact with the Salesforce APIs using domain URLs retrieved with this method. To bypass remote site settings, My Domain must be enabled in your org.” > https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_methods_system_url.htm (see getOrgDomainUrl method)
December 15, 2018 at 1:56 pm
Thanks Andy. That is helpful. Still not 100% remote site independent as there may be orgs which do not have “my domain” enabled. But still something is better than nothing.
December 15, 2018 at 1:58 pm
Yes agree, that was exactly my thinking as well.
October 7, 2019 at 3:24 am
Hi Andrew, thanks for a spot on blog post. We are an ISV already going though a yearly security review, so we are also curious whether your solution has gone through one, and if it passes.
December 28, 2019 at 7:16 pm
I have heard folks been successful. The key is to be open to the user what’s happening.