Andy in the Cloud

From BBC Basic to Force.com and beyond…

Tag Archives: unzipping files


by 49 Comments

Handling Office Files and Zip Files in Apex – Part 2

In part 1 of this blog I talked about using the Salesforce Metadata API, Static Resources the PageReference.getContent method to implement a native unzip. This blog completes the series by introducing two Visualforce Components (and subcomponents) that provide full zip and unzip capabilities. By wrapping the excellent JSZip library behind the scenes. I gave myself three main goals…

  • Easy of use for none JavaScript Developers
  • Abstraction of the storage / handling of file content on the server
  • Avoiding hitting the Viewstate Governor!

While this audience and future developers I hope will be the judge of the first, the rest was a matter of selecting JavaScript Remoting. As the primary means to exchange the files in the zip or to be zipped between the client and server.

Unzip Component

The examples provided in the Github repo utilise a Custom Object called Zip File and Attachments on that record to manage the files within. As mentioned above, this does not have to be the case, you could easily implement something that dynamically handles and/or generates everything if you wanted!

As Open Office files are zip files themselves, the following shows the result of using the Unzip Demo to unzip a xlsx file. The sheet1.xml Attachment is the actual Sheet in the file, in its uncompressed XML form. Thus unlocking access to the data within! From this point you can parse it and update it ready for zipping.

Screen Shot 2012-12-08 at 10.20.47

As a further example, the following screenshot shows the results of unzipping the GitHub repo zip download…

Screen Shot 2012-12-08 at 10.10.12

The c:unzipefile component renders a HTML file upload control to capture the file. Once the user selects a file, the processing starts. It then uses the HTML5 file IO support (great blog here) to read the data and pass it to the JSZip library. This is all encapsulated in the component of course! The following shows the component in action on the Unzip Demo page.

<c:unzipfile name="somezipfile" oncomplete="unzipped(state);"
 onreceive=
 "{!$RemoteAction.UnzipDemoController.receiveZipFileEntry}" />

Screen Shot 2012-12-08 at 10.09.02

As mentioned above it uses JavaScript Remoting, however as I wanted to make the component extensible in how the file entries are handled. I have allowed the page to pass in the RemoteAction to call back on. Which should look like this..

@RemoteAction
 public static String receiveZipFileEntry(
    String filename, String path, String data, String state)

In addition to the obvious parameters, the ‘state’ parameter allows for some basic state management between calls. Since of course JavaScript Remoting is stateless. Basically what this method returns is what gets sent back in the ‘state’ parameter on subsequent calls. In the demo provided, this contains the Id of the ZipFile record used to store the incoming zip files as attachments.

The other key attribute on the component is ‘oncomplete’. This can be any fragment of JavaScript you choose (the ‘state’ variable is in scope automatically). In the demo provided it calls out to an Action Function to invoke a controller method to move things along UI flow wise, in this case redirect to the Zip File record created during the process.

Zip Component

You may have noticed on the above screenshots I have placed a ‘Zip’ custom button on the Zip File objects layout. This effectively invokes the Zip Demo page. The use cases here is to take all the attachments on the record, zip them up, and produce a Document record and finally redirect to that for download.

Screen Shot 2012-12-09 at 10.16.11

Screen Shot 2012-12-09 at 10.17.02

The c:zipfile component once again wraps the JSZip library and leverages JavaScript Remoting to request the data in turn for each zip file entry. The page communicates the zip file entries via the c:zipentry component. These can be output explicitly at page rendering time (complete with inline base64 data if you wish) or empty leaving the component to request them via JS Remoting.

<c:zipfile name="someZipfile" state="{!ZipFile__c.Id}" 
  oncomplete="receiveZip(data);"
  getzipfileentry=
     "{!$RemoteAction.ZipDemoController.getZipFileEntry}">
   <apex:repeat value="{!paths}" var="path">
     <c:zipentry path="{!path}" base64="true"/>
   </apex:repeat>
</c:zipfile>

This component generates a JavaScript method on the page based on the name of the component, e.g. someZipfileGenerate. This must be called  at somepoint by the page to start the zip process.

The action method in the controller needs to look like this.

@RemoteAction
 public static String getZipFileEntry(String path, String state)

Once again the ‘state’ parameter is used. Except in this case it is only providing what was given to the c:zipfile component initially, it cannot be changed. Instead the method returns the Base64 encoded data of the requested zip file entry.

Finally the ‘oncomplete’ attributes JavaScript is executed and the resulting data is passed back (via apex:ActionFunction) to the controller for storage (not the binding in this case is transient, always good to avoid Viewstate issues when receiving large data) and redirects the user to the resulting Document page.

Summary

Neither components are currently giving realtime updates on the zip entries they are processing, so as per the messaging in the demo pages the user has to wait patiently for the next step to occur. Status / progress messages is something that can easily be implemented within the components at a future date.

These components utilise some additional components, I have not covered. Namely the c:zip and c:unzip components. If you have been following my expliots in the Apex Metdata API you may have noticed early versions of these in use in those examples, check out the Deploy and Retrieve demos in that repo.

I hope this short series on Zip file handling has been useful to some and once again want to give a big credit to the JSZip library. If you want to study the actual demo implementations more take a look at the links below. Thanks and enjoy!

Links


by 25 Comments

Handling Office Files and Zip Files in Apex – Part 1

I recently found a number of developers asking questions about Zip file handling in Apex, which as most find out pretty soon, does not exist. Statement governor concerns aside, nor is there any binary data types to make implementing support for it possible. Most understandably most want to stay on platform and not call out to some external service for this. And rightly so!

Another developer and I recently had the same need before the API provider decided to handle compression over HTTP. However while subsequently doing some work to get the Metadata API working from Apex, which also requires zip support in places. I came across a partial native solution and in addition to that I’ve also had in mind to try out some day the JSZip Javascript library.

And so I decided to write some blog entries covering these two approaches and hopefully help a few people out…

Approach A: Using Static Resources and Metadata API

A static resource is as the name suggests some that changes rarely in your code, typically used for .js, .css and images. Salesforce allows you to upload these type of files individually or within a zip file. Using its development UI and tools.

Allowing you then reference in your Visualforce pages like this

$URLFOR($Resource.myzip, '/folder/file.js').

Further more you can also do this in Apex.

PageReference somefileRef = 
   new PageReference('/resource/myzip/folder/file.js');
Blob contentAsBlob = somefileRef.getContent();
String contentAsText = contextAsBlob.toString();

So in effect Salesforce does have a built in Zip handler, at least for unzipping files anyway. The snag is, uploading a zip file dynamically once your user has provided it to you. If you review the example code that calls the Metadata API from Apex you might have spotted an example of doing just this. To create a Static Resource you can do the following.

MetadataService.MetadataPort service = createService(); 
MetadataService.StaticResource staticResource = 
    new MetadataService.StaticResource();
staticResource.fullName = 'test';
staticResource.contentType = 'text';
staticResource.cacheControl = 'public';
staticResource.content = 
   EncodingUtil.base64Encode(Blob.valueOf('Static stuff'));
MetadataService.AsyncResult[] results = 
    service.create(
      new List; { staticResource });

They key parts are the assignment of the ‘contentType’ and ‘content’ members. The ‘content’ member is a Base64 encoded string. In the example above its a static peace of text, however this could easily be the zip file the user has just uploaded for you and then Base64 encoded using the Apex EncodingUtil. You also need to set the ‘contentType’ to ‘application/zip’.

This example presents a user with the usual file upload, you will also need to ask or know what the path of the file is you want to extract. Of course if you know this or can infer it e.g. via file extension such as .docx or .xslx, which uses the Office Open XML format, then your good to go.

I’ve shared the code for this here.

Known Issues: If your zip file contains files with names that contain spaces or special characters, I have found issues with this approach. Since this also applies to URLFOR and generating links in VF pages, I plan to raise a support case to see if this is a bug or limitation. Also keep in mind the user will need to have the Author Apex permission enabled on their profile to call the Metadata API.

Summary and Part 2

This blog entry covered only unzip for known file types, but what if you wanted to have the code inspect the zip contents? Part 2 will extend some of the components I’ve been using in the apex-mdapi to use the JSZip library, examples of which are here and here. In extending those components, I’ll be looking at making them use JavaScript Remoting and local HTML5 file handling to unzip the file locally in the page and transmit the files to the server via an Apex Interface the controller implements. Likewise I want to use the same approach to prepare a zipped file by requesting zip content from the server.