technology

JSON All The Way Down: A Story Of Betrayal

It was a quiet Thursday morning at Jana. Our weekly all-hands meeting had just ended and I had settled in at my laptop, sipping the dregs of my morning coffee. I was tweaking the UI of a new feature and I needed to test my changes on a phone with a much smaller screen—horizontal space is an important design consideration. And my monstrous Nexus 6 has plenty of screen real estate.

I started using a smaller Samsung GT-S6310L, running Android 4.1.2. Almost immediately, nothing worked as expected: the HTTP response was empty, the database was empty, and the buttons I wanted to test were nowhere to be found!

At this point, I had changed neither server nor client code. The obvious underlying variable that I knew had changed was the version of Android my test device was running. J’accuse!

I inspected the contents of the request, comparing what the server received from the Samsung phone with my Nexus. I expected the JSON body of the request to contain an array of strings. The server parsed the request from the Samsung phone as a single string: '[abcdef,z9y8x7]'. This resulted in an error and an empty response—the API endpoint needed a list, not a string. The Android M phone’s request could be correctly parsed as an array of strings: ['abcdef','z9y8x7'].

Next, I peered at how we were building the request in the Android app (in pseudo-code):

 // the request object
 public class HeisenRequest extends ApiRequest {
     public HeisenRequest(List secrets) {
         super();
         this.getParams().put("secrets", secrets); // this.getParams() returns a JSONObject
         this.setMethod("POST");
         this.setEndpoint("secrets");
     }
 }
 ...
 // creating and making the request
 List secrets = new ArrayList();
 secrets.add("secret_a");
 secrets.add("secret_b");
 HeisenRequest heisenRequest = new HeisenRequest(secrets);
 client.handleRequest(heisenRequest);
 ...
 // actually make the HTTP request
 public void handleRequest(ApiRequest request) {
     String jsonBody = request.getParams().toString(); // stringify the JSONObject
     MyVolleyRequest volleyRequest = new MyVolleyRequest();
     volleyRequest.setJsonBody(jsonBody);
     volleyRequest.setMethod(request.getMethod());
     volleyRequest.setUrl(BASE_URL + request.getEndpoint());
     volleyRequest.makeRequest();
 }

The only code in this process outside the control of Jana engineers were any of the built-in Android libraries, like Volley and JSON. So I looked another dream level deeper. Enter: Android’s JSONObject.

They're not dead. Just, you know, going another dream level deeper.

They’re not dead. Just, you know, going another dream level deeper.

Android Studio’s SDK Manager will download and make available source code for the different versions of Android. I opened the implementation of JSONObject for the version running on the Samsung test phone—API level 14.

 // I've removed a lot of boiler-plate and exception handling for brevity
 public class JSONObject {
     ...
     private final Map<String, Object> nameValuePairs;
     ...
     public JSONObject(Map copyFrom) {
         for (Map.Entry entry : copyFrom.entrySet()) {
             nameValuePairs.put(key, entry.getValue());
         }
     }
     ...
     public String toString() {
         JSONStringer stringer = new JSONStringer();
         writeTo(stringer);
         return stringer.toString();
     }
     ...
 }

(The JSONStringer stores the contents of the JSONObject inside a StringBuilder via writeTo)

When an API level 14 JSONObject is stringified, the output of each element within that object is controlled by the element’s own toString function. And there is no guarantee that any particlar Java object’s toString method will return properly formatted JSON.

Then I switched gears to see what JSONObject in API level 23 was doing right. (I did not check below API level 14 because mCent does not support older Android versions).

 public JSONObject(Map copyFrom) {
      for (Map.Entry entry : copyFrom.entrySet()) {
         String key = (String) entry.getKey();
         nameValuePairs.put(key, wrap(entry.getValue()));
     }
 }

API level 23’s implementation of JSONObject is identical except for one functional call: wrap.

 public static Object wrap(Object o) {
     if (o == null) {
         return NULL;
     }
     if (o instanceof JSONArray || o instanceof JSONObject) {
         return o;
     }
     if (o.equals(NULL)) {
         return o;
     }
     try {
         if (o instanceof Collection) {
             return new JSONArray((Collection) o);
         } else if (o.getClass().isArray()) {
             return new JSONArray(o);
         }
         if (o instanceof Map) {
             return new JSONObject((Map) o);
         }
         if (o instanceof Boolean ||
             o instanceof Byte ||
             o instanceof Character ||
             o instanceof Double ||
             o instanceof Float ||
             o instanceof Integer ||
             o instanceof Long ||
             o instanceof Short ||
             o instanceof String) {
             return o;
         }
         if (o.getClass().getPackage().getName().startsWith("java.")) {
             try {
                 return o.toString();
             }
     } catch (Exception ignored) {
     }
     return null;
 }

The purpose of wrap is to wrap an object in a “JSON safe” manner, if necessary. Basic Java types like Boolean, Float, or String are left alone because they will serialize (via their own toString methods) into valid JSON. Complex objects based on Java’s Collection or Map will not serialize into valid JSON and need to wrapped in JSONArray or JSONObject, respectively. Thus, when the JSONObject is stringified to form the request, the JSONArray-wrapped-List is guaranteed to produce a valid JSON representation of an array of strings.

In the end, I was betrayed by my assumptions about Android’s JSON libraries. I assumed that Android would use valid JSON data types and representations all the way down. I was wrong.

The solution was simple, though. Wrapping the List in a JSONArray before putting it in the request guaranteed that the List would serialize to correct JSON across Android versions and that the server would ultimately receive a list of strings: ['abcdef','z9y8x7'].

Discussion

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s