[icloud - XPath] Parsing XML-response and XPath

Basic Tutorials concerning: GUI, Views, Activites, XML, Layouts, Intents, ...

[icloud - XPath] Parsing XML-response and XPath

Postby Tobias » Wed Feb 24, 2010 2:52 pm

[icloud - XPath] Parsing XML-response and XPath

What you learn: You will learn how to take care of a XML response. In this case a SOAP response. We will be using XPath and a technique that could be compared to a pull parser. We will use Ximpleware VTD-XML the API can be found here.

Why? What's next? In my first tutorial we looked into how to connect to icloud (SOAP + digest), this is the second step towards the cloud and all it's possibilities.

Difficulty: 3 of 5

Previous tutorial: Tutorial 1 - Login

Description: We have med a server request using SOAP, and now we have a SOAP-response in a byte[] variable (see first tutorial above) that we want to know what's in it.

Let's get started with the coding! :)

First of we need to know what the response XML looks like, below is an example of what it could look like

Syntax: [ Download ] [ Hide ]
Using xml Syntax Highlighting
  1. <?xml version="1.0" encoding="utf-8"?>
  2. <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
  3.         <SOAP-ENV:Body>
  4.                 <xcr:loginResponse xmlns:xcr="http://xcerion.com/xcRepository.xsd">
  5.                         <userid>128851</userid>
  6.                         <username>cafe12tutorial</username>
  7.                         <firstname>Tobias</firstname>
  8.                         <lastname>Engström</lastname>
  9.                         <locale>en_US</locale>
  10.                         <lastlogin>2010-02-21T15:52:43Z</lastlogin>
  11.                         <logins>2</logins>
  12.                         <home>2814835</home>
  13.                         <library>28148357051</library>
  14.                         <in>281483570</in>
  15.                         <out>28148357052</out>
  16.                         <contacts>1718</contacts>
  17.                         <settings></settings>
  18.                         <params></params>
  19.                         <session>OmGwd68xhZ==</session>
  20.                         <drives>
  21.                                 <drive>
  22.                                         <driveId>3006581</driveId>
  23.                                         <ownerId>128851</ownerId>
  24.                                         <folderId>2814835</folderId>
  25.                                         <sysname>xios</sysname>
  26.                                         <name></name>
  27.                                         <system>home</system>
  28.                                         <currentSize>39321</currentSize>
  29.                                         <quotaLimit>3221225472</quotaLimit>
  30.                                         <readLocked>false</readLocked>
  31.                                         <writeLocked>false</writeLocked>
  32.                                         <created>2010-02-21T15:47:10Z</created>
  33.                                 </drive>
  34.                                 <drive>
  35.                                         <driveId>3006582</driveId>
  36.                                         <ownerId>128851</ownerId>
  37.                                         <folderId>28148357051</folderId>
  38.                                         <sysname>lib</sysname>
  39.                                         <name></name>
  40.                                         <system>library</system>
  41.                                         <currentSize>301</currentSize>
  42.                                         <quotaLimit>1073741824</quotaLimit>
  43.                                         <readLocked>false</readLocked>
  44.                                         <writeLocked>false</writeLocked>
  45.                                         <created>2010-02-21T15:47:10Z</created>
  46.                                 </drive>
  47.                                 <drive>
  48.                                         <driveId>3006583</driveId>
  49.                                         <ownerId>128851</ownerId>
  50.                                         <folderId>28148357052</folderId>
  51.                                         <sysname>in</sysname>
  52.                                         <name></name>
  53.                                         <system>incoming</system>
  54.                                         <currentSize>0</currentSize>
  55.                                         <quotaLimit>1073741824</quotaLimit>
  56.                                         <readLocked>false</readLocked>
  57.                                         <writeLocked>false</writeLocked>
  58.                                         <created>2010-02-21T15:47:10Z</created>
  59.                                 </drive>
  60.                                 <drive>
  61.                                         <driveId>3006584</driveId>
  62.                                         <ownerId>128851</ownerId>
  63.                                         <folderId>28148357052</folderId>
  64.                                         <sysname>out</sysname>
  65.                                         <name></name>
  66.                                         <system>outgoing</system>
  67.                                         <currentSize>0</currentSize>
  68.                                         <quotaLimit>1073741824</quotaLimit>
  69.                                         <readLocked>false</readLocked>
  70.                                         <writeLocked>false</writeLocked>
  71.                                         <created>2010-02-21T15:47:10Z</created>
  72.                                 </drive>
  73.                         </drives>
  74.                         <group id='17179869184' name='xios'/>
  75.                 </xcr:loginResponse>
  76.         </SOAP-ENV:Body>
  77. </SOAP-ENV:Envelope>
Parsed in 0.015 seconds, using GeSHi 1.0.8.4


Now we start with initiating the classes we need and set the namespaces which are used.
Syntax: [ Download ] [ Hide ]
Using java Syntax Highlighting
  1. AutoPilot ap = new AutoPilot();
  2. VTDGen vg = new VTDGen();
  3. vg.setDoc(doc, 0 , doc.length); // doc is a byte[] which is the result from the last tutorial
  4. vg.parse(true);  // set namespace awareness to true
  5. VTDNav vn = vg.getNav();
  6. ap.bind(vn);
  7. ap.declareXPathNameSpace("SOAP-ENV","http://schemas.xmlsoap.org/soap/envelope/");
  8. ap.declareXPathNameSpace("xcr","http://xcerion.com/xcRepository.xsd");
  9.  
Parsed in 0.033 seconds, using GeSHi 1.0.8.4


Now with these objects initiated, XML-parsed and the navigator associated with the AutoPilot class we can navigate through the XML with either the classical DOM way with selectElement and first-child, next-sibling etc, the other way is as promissed XPath with the selectXPath function. Let's look into the different ways below starting with a XPath example.

Syntax: [ Download ] [ Hide ]
Using java Syntax Highlighting
  1. ap.selectXPath("/SOAP-ENV:Envelope/SOAP-ENV:Body/xcr:loginResponse/userid");
  2. String strUserid = ap.evalXPathToString();
  3.  
Parsed in 0.035 seconds, using GeSHi 1.0.8.4


Comment: The above code will in this case select my userid (128851) and evaluate it to a string (strUserid). At the moment we don't need this it was just to demonstrate how simple it is to use XPath.

The next example will continue navigating through the XML where the cursor is now (userid).
Syntax: [ Download ] [ Hide ]
Using java Syntax Highlighting
  1. vn.toElement(VTDNav.NEXT_SIBLING);
  2. String strCurrentElement = vn.toString(vn.getCurrentIndex());
  3. String strCurrentTextValue = vn.toString(vn.getText());
  4.  
Parsed in 0.035 seconds, using GeSHi 1.0.8.4


Comment: strCurrentElement equals "username" and strCurrentTextValue equals "cafe12tutorial". What we did was just to the next sibling, from <userid> to <username>.
When navigating with VTDNav it's all about indexes (positions in the document), getCurrentIndex() returns where we are now, and the toString function with the index parameter works just like you expect it to. getText() will return the index of the text-element (value of a node), and run toString on that index and you will get the textual value of the text-element (if there is any, else getText() will return -1 and you'll get a IndexOutOfBoundsException if you run toString(-1)).

Now check the documentation for a full list of available navigation methods and constants like NEXT_SIBLING, FIRST_CHILD, PARENT and more in VTDNav. My recommendation is that you try to use the VTDNav class as much as possible, and the AutoPilot only when in a real complex structure or for a initial position to start navigating from, like in the above example :)

Now to give this tutorial some more weight I'll give you a short snippet where we select the unique key for my/your folderXML (this is your folder tree, which is the heart for your online filesystem, once you have this you can start download, creating and uploading files from or to your folder structure).

If we continue from where we left in the last tutorial we can make it like this for loging in and then fetching our folderXML (Id is found in the login response).
Syntax: [ Download ] [ Hide ]
Using java Syntax Highlighting
  1. public void login() { // Comment 1
  2.                 String soapAction = "login";
  3.                 String soapBody = "";
  4.                 byte[] loginDoc = callServer(soapAction, soapBody);
  5.                
  6.                 String folderXMLId = getMyFolderXMLId(loginDoc);
  7.                 byte[] folderXML = getFolderXML(folderXMLId);
  8.                
  9.                 // Make use of the folderXML...
  10.         }
  11.  
  12. public String getMyFolderXMLId(byte[] loginResponse) { // Comment 2
  13.                 AutoPilot ap = new AutoPilot();
  14.                 try {
  15.                         VTDGen vg = new VTDGen();
  16.                         vg.setDoc(loginResponse, 0 , loginResponse.length);
  17.                         vg.parse(true);  // set namespace awareness to true
  18.                         VTDNav vn = vg.getNav();
  19.                         ap.bind(vn);
  20.                         ap.declareXPathNameSpace("SOAP-ENV","http://schemas.xmlsoap.org/soap/envelope/");
  21.                         ap.declareXPathNameSpace("xcr","http://xcerion.com/xcRepository.xsd");
  22.                        
  23.                         ap.selectXPath("/SOAP-ENV:Envelope/SOAP-ENV:Body/xcr:loginResponse/drives/drive[sysname = 'xios']/folderId");
  24.                 }
  25.             catch (ParseException e){
  26.                      System.out.println(" XML file parsing error n"+e);
  27.                      Log.e("ParsingExc", e.toString());
  28.             }
  29.             catch (XPathParseException e){
  30.                     Log.e("XPathParsingExc", e.toString());
  31.             }
  32.                         return ap.evalXPathToString();
  33.         }
  34.        
  35.         public byte[] getFolderXML(String strFolderId) {
  36.                 StringBuffer bufBody = new StringBuffer();
  37.                 bufBody.append("<folder id="");
  38.                 bufBody.append(strFolderId);
  39.                 bufBody.append(""/>");
  40.                 return callServer("getFolderXML", bufBody.toString()); // Comment 3
  41.         }
  42.  
Parsed in 0.042 seconds, using GeSHi 1.0.8.4


Comment 1: This is the login function that we created in the last tutorial with some additional rows at the end

Comment 2: The code below is the "same" as above where you can find the description of what we use it for. Note that I've added some try catch.

Comment 3: In the last tutorial I explained how the callServer function was implemented and how we build our request body. This should be pretty basic, and you should also be able to implement your own request bodies.

Below you see the request body and the response from getFolderXML (remember it won't look the exact same way when you login)

Request body:
Syntax: [ Download ] [ Hide ]
Using xml Syntax Highlighting
  1. <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema">
  2. <SOAP-ENV:Body>
  3. <getFolderXML><folder id="8589934690"/></getFolderXML>
  4. </SOAP-ENV:Body>
  5. </SOAP-ENV:Envelope>
  6.  
Parsed in 0.001 seconds, using GeSHi 1.0.8.4


Response:
Syntax: [ Download ] [ Hide ]
Using xml Syntax Highlighting
  1. <?xml version="1.0" encoding="utf-8"?>
  2. <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
  3. <SOAP-ENV:Body>
  4. <xcr:getFolderXMLResponse xmlns:xcr="http://xcerion.com/xcRepository.xsd">
  5. <fs:folder id='28148357057623' xmlns:xlink='http://www.w3.org/1999/xlink' xmlns:fs='http://xcerion.com/folders.xsd'>
  6. <fs:folder id='2814835705766543' name='Applications'>
  7. <fs:folder id='28148357057745' name='Data' type='folder'/>
  8. </fs:folder>
  9. <fs:folder id='28148357057543' name='Groups' system='groups'/>
  10. <fs:folder id='281483570576454' name='Documents'>
  11. <fs:folder id='281483570575654' name='Pictures'/>
  12. <fs:folder id='281483570577657' name='Music' type='folder'/>
  13. <fs:folder id='281483570575453' name='Movies' type='folder'>
  14. <fs:folder id='281483570579494' name='Playlists' type='folder'/>
  15. </fs:folder>
  16. </fs:folder>
  17. <fs:folder id='281483570579474' name='Trashcan' system='trashcan'/>
  18. <fs:folder id='281483570576098' name='Shared' system='shared'/>
  19. <fs:folder id='281483570570484' name='Desktop' system='desktop'/>
  20. <fs:folder id='281483570574777' name='Contacts' type='folder'/>
  21. <fs:folder id='281483570576366' name='Bookmarks' type='folder' listview='apps/system/listviews/documents_thumbnails.xsl'/>
  22. <fs:folder id='281483570576995' name='Appointments' type='folder' listview='apps/system/listviews/documents_thumbnails.xsl'/>
  23. <fs:folder id='281483570575554' name='temp' type='folder'/>
  24. <fs:folder id='281483570576454' name='Messages' type='folder' listview='custom' custom='apps/system/listviews/messages.xsl'/>
  25. <fs:folder id='281483570576949' name='Mail' type='folder'>
  26. <fs:folder id='281483570574949' name='cafe12tutorial@icloud.com' type='folder'>
  27. <fs:folder id='281483570576308' name='Inbox' type='folder' listview='custom' custom='apps/productivity/listviews/mail.xsl'/>
  28. <fs:folder id='281483570576565' name='Outbox' type='folder' listview='custom' custom='apps/productivity/listviews/mail.xsl'/>
  29. <fs:folder id='281483570576644' name='Drafts' type='folder' listview='custom' custom='apps/productivity/listviews/mail.xsl'/>
  30. <fs:folder id='281483570576333' name='Deleted' type='folder' listview='custom' custom='apps/productivity/listviews/mail.xsl'/>
  31. <fs:folder id='281483570576312' name='Spam' type='folder' listview='custom' custom='apps/productivity/listviews/mail.xsl'/>
  32. <fs:folder id='281483570576313' name='Sent' type='folder' listview='custom' custom='apps/productivity/listviews/mail.xsl'/>
  33. </fs:folder>
  34. </fs:folder>
  35. <fs:folder id='281483570576315' name='Profile' type='folder'>
  36. <fs:folder id='281483570576316' name='Guestbook' type='folder'/>
  37. <fs:folder id='281483570576317' name='Files' type='folder'/>
  38. <fs:folder id='281483570576318' name='Photos' type='folder'/>
  39. <fs:folder id='281483570576319' name='Activity' type='folder' listview='custom' custom='apps/social/profile/listviews/activities.xsl'/>
  40. <fs:folder id='281483570576320' name='Visitors' type='folder'/>
  41. </fs:folder>
  42. </fs:folder>
  43.  
  44. </xcr:getFolderXMLResponse>
  45. </SOAP-ENV:Body>
  46. </SOAP-ENV:Envelope>
  47.  
Parsed in 0.017 seconds, using GeSHi 1.0.8.4


Last but not least I will paste the full thing, this is what it looks like so far in my series of tutorials (still very basic coding).
Syntax: [ Download ] [ Hide ]
Using java Syntax Highlighting
  1. package com.xcerion.android.tutorials;
  2.  
  3. import java.io.DataInputStream;
  4.  
  5. import org.apache.http.HttpEntity;
  6. import org.apache.http.HttpResponse;
  7. import org.apache.http.auth.AuthScope;
  8. import org.apache.http.auth.UsernamePasswordCredentials;
  9. import org.apache.http.client.methods.HttpPost;
  10. import org.apache.http.entity.StringEntity;
  11. import org.apache.http.impl.client.DefaultHttpClient;
  12. import org.apache.http.params.HttpProtocolParams;
  13.  
  14. import android.util.Log;
  15.  
  16. import com.ximpleware.AutoPilot;
  17. import com.ximpleware.NavException;
  18. import com.ximpleware.ParseException;
  19. import com.ximpleware.VTDGen;
  20. import com.ximpleware.VTDNav;
  21. import com.ximpleware.XPathEvalException;
  22. import com.ximpleware.XPathParseException;
  23.  
  24. public class BaxideTiny {
  25.         // TODO: You need to edit these two lines (In the future we will make a login form and not use static data)
  26.         private String username = ""; // Edit to Your username at icloud.com
  27.         private String password = ""; // Edit to Your password associated with your icloud.com account
  28.        
  29.         public void login() {
  30.                 String soapAction = "login";
  31.                 String soapBody = "";
  32.                 byte[] loginDoc = callServer(soapAction, soapBody);
  33.                
  34.                 String folderXMLId = getMyFolderXMLId(loginDoc);
  35.                 byte[] folderXML = getFolderXML(folderXMLId);
  36.                
  37.                 // Make use of the folderXML...
  38.         }
  39.        
  40.         public byte[] callServer(String soapAction, String body) {
  41.                 byte[] result = null;
  42.                 DefaultHttpClient httpclient = new DefaultHttpClient();
  43.                
  44.                 HttpProtocolParams.setUseExpectContinue(httpclient.getParams(), false);
  45.                
  46.  
  47.             httpclient.getCredentialsProvider().setCredentials(
  48.                     new AuthScope("os.icloud.com", 80, null, "Digest"),
  49.                     new UsernamePasswordCredentials(username, password));
  50.            
  51.             HttpPost httppost = new HttpPost("http://os.icloud.com/v1/");
  52.             httppost.setHeader("soapaction", soapAction);
  53.             httppost.setHeader("Content-Type", "text/xml; charset=utf-8");
  54.            
  55.             System.out.println("executing request" + httppost.getRequestLine());
  56.  
  57.                 final StringBuffer soap = new StringBuffer();
  58.                 soap.append("<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="" xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" xmlns:xsd="http://www.w3.org/1999/XMLSchema"><SOAP-ENV:Body>");
  59.                         soap.append("<");
  60.                 soap.append(soapAction);
  61.                 soap.append(">");
  62.                 soap.append(body);
  63.                 soap.append("</");
  64.                 soap.append(soapAction);
  65.                 soap.append("></SOAP-ENV:Body></SOAP-ENV:Envelope>");
  66.                        
  67.                 try {
  68.                         HttpEntity entity = new StringEntity(soap.toString());
  69.                         httppost.setEntity(entity);
  70.                         HttpResponse response = httpclient.execute(httppost);
  71.                         HttpEntity r_entity = response.getEntity();
  72.  
  73.                 if( r_entity != null ) {
  74.                     result = new byte[(int) r_entity.getContentLength()];
  75.                 if(r_entity.isStreaming()) {
  76.                         DataInputStream is = new DataInputStream(r_entity.getContent());
  77.                         is.readFully(result);
  78.                 }
  79.                 }
  80.                 } catch(Exception E) {
  81.                        
  82.                 }
  83.                
  84.                 httpclient.getConnectionManager().shutdown();
  85.                 return result;
  86.         }
  87.        
  88.         public String getMyFolderXMLId(byte[] loginResponse) {
  89.                 AutoPilot ap = new AutoPilot();
  90.                 try {
  91.                         VTDGen vg = new VTDGen();
  92.                         vg.setDoc(loginResponse, 0 , loginResponse.length); // Comment
  93.                         vg.parse(true);  // set namespace awareness to true
  94.                         VTDNav vn = vg.getNav();
  95.                         ap.bind(vn);
  96.                         ap.declareXPathNameSpace("SOAP-ENV","http://schemas.xmlsoap.org/soap/envelope/");
  97.                         ap.declareXPathNameSpace("xcr","http://xcerion.com/xcRepository.xsd");
  98.                        
  99.                         ap.selectXPath("/SOAP-ENV:Envelope/SOAP-ENV:Body/xcr:loginResponse/drives/drive[sysname = 'xios']/folderId");
  100.                 }
  101.             catch (ParseException e){
  102.                      System.out.println(" XML file parsing error n"+e);
  103.                      Log.e("ParsingExc", e.toString());
  104.             }
  105.             catch (XPathParseException e){
  106.        
  107.             }
  108.                         return ap.evalXPathToString();
  109.         }
  110.        
  111.         public byte[] getFolderXML(String strFolderId) {
  112.                 StringBuffer bufBody = new StringBuffer();
  113.                 bufBody.append("<folder id="");
  114.                 bufBody.append(strFolderId);
  115.                 bufBody.append(""/>");
  116.                 return callServer("getFolderXML", bufBody.toString());
  117.         }
  118. }
  119.  
Parsed in 0.054 seconds, using GeSHi 1.0.8.4


Now I've written two tutorials, I would love you to send me some feedback, here or IM me, and let me know what you think, what you want more of or any comment is greatly appreciated :D !

Best regards,
Tobias
Tobias
Junior Developer
Junior Developer
 
Posts: 10
Joined: Mon Nov 23, 2009 11:29 pm

Top

Return to Novice Tutorials

Who is online

Users browsing this forum: No registered users and 9 guests