A Recipe for writing responsive REST clients on Android

Android development is easy but making the app resilient is hard.

Apps are developed under perfect conditions. Test phones with few applications running on top of them with a reliable network conditions. When the app is handed off to end users, environment change dramatically. People use their apps everywhere and network connection is very unreliable.

When developers don’t take care of such cases, it results in a sh*tty user experience. In this blog post, I will share my personal architectural choices with library references.

If you haven’t watched already, you should definitely spend an hour and carefully listen to this talk in Google IO 2010.

The most important key to provide good user experience is to drop dependency on network while handling user actions. You need to have a minimal persistent model in your client application and all of your UIs should render themselves based on this data. Changes applied by the user should be applied to local model instantly and synched to your backend in the background, whenever network is available.

It is easy to say but when application grows in size, this becomes really hard if you don’t have a proper application architecture in place.

Although there are exceptions, many applications fit well into MVC model.

  • Models: Whatever you show in your UI should be modelled accordingly and persistent. Most of the time, sqlite is your best choice in Android. Writing bare sqlite is time consuming and error prone so I highly recommend using an ORM. At Path, we use a highly customized version of GreenDao or you can use OrmLite. Although code generation is ugly, I still prefer GreenDao to avoid reflection which runs extremely slow before Honeycomb.
    UPDATE As Eugenio pointed out in Google+, looks like there is a way to avoid runtime annotation processing for ORMLite using an auto generated table config file.
    • Controllers: You should write your application logic in separate controller classes that are independent from your activities. Activities & Fragments are volatile and have short life-cycles. Responsibility of these controllers should be to handle the user request, make necessary change in data model and enqueue web requests if necessary.
    • Views and View Controllers: Views are your regular views & View holders on Android. View Controller’s depends on your state. If it is a simple layout, most of the time your Activity or Fragment. They are responsible to call your model to fetch the data and render your views. It is also their responsibility to listen for events and update views as necessary. For instance, if it is a user profile activity, the activity should:
      • Register for events related to User object (e.g. UserInfoUpdatedEvent)
      • Tell UserController to refresh user’s information (if necessary)
      • Load the data from model in an async task and render the view when it returns.
        The key rule here is that, 99% of the time, your activity should not have any method that relies on network. It should always load the data from disk and just render it.
      • Events: Events are the only contract between your components (controllers, views etc). Any component is responsible for emitting events in their scope and listening for events which may require them to take action.
        For instance, assume you have a messaging client. It is your MessageControllers responsibility to save the incoming message to database and then dispatch a NewMessageArrivedEvent with related information. Your conversation view will listen for this event and refresh itself if it is related. If your app is in background, then it will be your NotificationController‘s responsibility to show a notification. This may look like too much work but it is your key to scale the app going further. If your product people asks you to implement an unread count in user photos, then it will be that component’s responsibility to listen for the same event and update the number.

      There are many tools that will help you put this architecture in place. I’ll only mention the ones I used but they are not the only way, probably not even the best.

      • Persistence: Start with a good orm. My favorite is GreenDao but many people are happy with OrmLite.
      • Inversion of Control: Roboguice is a nice library that does more than dependency injection. Though its runtime annotation processing requirement is a bummer for Android. At Path, we use a in house written compile time dependency injection library to fit our needs but Dagger is probably the best open source solution.
      • EventBus: EventBus by GreenRobot is my favorite here. Is clean, supports getting the callback in different threads.
      • Background Activity: To have a responsive client, it is essential that you have a good architecture in place. It is very common that an application is showing a progress bar just because a more important request is waiting for a less important request to finish. This is why we’ve developed Job Queue. Once you model all of your background activity as Jobs, it will be very easy for you to control which request to run first.

      Real Life Case Study

      A Twitter client

      Writing a Twitter client is a really good example as it reflects many aspects of a regular REST client. You need to fetch different feeds, send retweets, send replies and follow requests. Lets briefly model how such a client can work.

      For now, we’ll assume that:

      • We have an Async task implementation called SafeAsyncTask that binds itself to activity lifecycle and cancels automatically if activity is stopped.
      • We have a BaseActivity class that extends Activity, registers to EventBus when activity is started and unregister when it stops.
      • Model: For simplicity, assume we have the following objects, all persistent into database:
        • Tweet
        • Feed (a list of tweet ids)
        • User
      • Our model will use a single identity scope which will guarantee that only a single instance of any tweet will exists in memory.
      • We’ll implement a TweetController to manage all tweet activity.

      Loading and displaying the feed First, we’ll write a FetchFeedJob which is responsible to fetch the feed for a given feedId. Here, feedId might be user’s own feed or any hashtag. Basically, anything that can be displayed as a feed. TweetController will have the public method which can be triggered to enqueue this job.

    Disclaimer Any codes in this section should be considered as pseudo code. I’m not writing it in an IDE so expect typos.

    File TweetController.java

    //we require the related activity if present so that Job can auto-cancel itself if activity disappears.
    public void refreshFeed(String feedId, @Nullable Activity bindToActivity) {  
        jobManager.addJobInBackground(bindToActivity == null ? Priority.MIN : Priority.UX, new FetchFeedJob(feedId, bindToActivity));  
    }
    

    Here, I assumed we have a Priority static class which holds a bunch of numbers to control priority. For simplicity, assume it has the following fields:

    public static final int MIN = 1;
    public static final int SYNC = 2;
    public static final int PREFETCH = 3;
    public static final int UX = 4;//user facing
    

    When user navigates to any feed activity, it will look like this:

    File: FeedActivity.java

    @Inject TweetController tweetController;
    @Inject FeedModel feedModel;
    public void onStart() {
        tweetController.refreshFeed(myFeedId, this);
        refreshContent();
    }
    
    private void refreshContent() {
       new SafeAsyncTask<Feed, String>(this) {
            public Feed doInBackground(String feedId) {
                return feedModel.getFeed(myFeedId);
            }
            public void onPostExecute(Feed feed) {
                if(feed.size() == 0) {
                    showLoading();
                } else {
                    hideLoading();
                    //add some code here to preserve list scroll position to avoid jumps in UI
                    listView.setAdapter(new Adapter(feed.getTweets());
                }
            }
       }.execute(myFeedId);
    }
    
    public void onEventMainThread(FetchingFeedEvent event) {
        if(event.getFeedId().equals(myFeedId)) {
            showLoadingInActionbar();
        }
    }
    
    public void onEventMainThread(UpdatedFeedEvent event) {
        if(event.getFeedId().equals(myFeedId) {
             if(event.isSuccessful()) {
                 refreshContent();
             } else {
                 showError(event.getError());
             }
        }
    }
    

    Keep in mind that event callback are NOT called if activity is stopped. It is handled by the BaseActivity. This helps us keep the code clean and not check if the view exists or activity is visible.

    The last missing piece of code is the actual Job to fetch the feed. It will look like:

    File FetchFeedJob.java

    public class FetchFeedJob extends BaseJob {
        private final String feedId;
        @Inject EventBus eventbus;
        @Inject FeedModel feedModel;
        @Inject TweetModel tweetModel;
        @Inject TwitterAPI twitterAPI;
        public FetchFeedJob(String feedId, @Nullable Activity boundToActivityLifecycle) {
            super(true, feedId, boundToActivityLifecycle);
            //require network. run in feedId group to assure that only one instance per feed runs at a given time
            //we'll assume that BaseJob can get an activity as a parameter and similar to SafeBackgroundTask, 
            //it can cancel itself silently 
            this.feedId = feedId;
        }
        public void onAdded() {
            eventBus.post(new FetchingFeedEvent(feedId);
        }
        public void onRun() {
            List<Tweet> tweets = twitterAPI.fetchFeed(feedId);
            tweets = tweetModel.saveNewTweets(tweets);//assume this returns internalized tweets which are added to the identity scope
            Feed feed = feedModel.getOrCreate(feedId);
            feed.addTweets(tweets);
            eventBus.post(new UpdatedFeedEvent(feedId, true));//true means success
        }
        public void onCancel() {
            if(super.didActivityDisappear() == false) {
                eventBus.post(new UpdatedFeedEvent(feedId, false));//false means failure
            }
        }
    }
    

    This is pretty much enough code to show a particular feed. Of course you’ll need additional functionality to paginate the feed etc but that is outside the scope of this post.

    • Re-Tweet Functionality Updating objects is a case where many developers go lazy and avoid doing local updates because preserving object state is not trivial. Most of the time, I prefer to duplicate the field with a local version to implement such functionality.

      For this example, lets assume our Tweet class has a field called didRetweet which is returned by the server as a boolean. We will add another field, called didLocalRetweet which is also a boolean but Nullable (to be specific, java.lang.Boolean). If user did any action that is pending, we use this field (while rendering, we take it into account). Since this field never arrives from server, its value is never overridden.

    File: Tweet.java

    public boolean didRetweet;
    public Boolean didLocalRetweet;
    public boolean isRetweeted() {
        return didRetweet || Boolean.TRUE.equals(didLocalRetweet);
    }
    

    Rendering code will look sth like this:

    File: TweetViewHolder.java

    if(tweet.getDidRetweet()) {
        retweetButton.setState(State.DONE);
        retweetButton.setEnabled(false);
    } else if(Boolean.TRUE.equals(tweet.getDidLocalRetweet()) {
        retweetButton.setState(State.PENDING);
        retweetButton.setEnabled(false);
    } else {
        retweetButton.setState(State.NONE);
        retweetButton.setEnabled(true);
    }
    

    When user clicks on an enabled retweet button, we call tweetController to make necessary changes.

    File: TweetViewHolder.java

    public onRetweetClick() {
        tweetController.retweet(tweet);
    }
    

    File: TweetController.java

    public void retweet(Tweet tweet) {
        if(!tweet.isRetweeted()) {
            jobManager.addJobInBackground(Priority.SYNC, new RetweetJob(tweet)); 
        }
    }
    

    And finally, RetweetJob:

    public class RetweetJob extends BaseJob {
        private final long tweetId;//we keep just the id, not the actual tweet because job is persistent. This is done to respect identity scope. Your implementation may change depending on your DAO layer.
        @Inject transient EventBus eventbus;//we mark these as transient since we don't want them to be serialized
        @Inject transient TweetModel tweetModel;
        @Inject transient TwitterAPI twitterAPI;
        public RetweetJob(Tweet tweet) {
            super(true, true);//require network & persist
            this.tweetId = tweet.getId();
        }
        @Override
        public void onAdded() {
            //job is written to disk. we can safely update the model.
            Tweet tweet = tweetModel.get(tweetId);
            tweet.setDidLocalRetweet(true);
            tweetModel.update(tweet);
            eventBus.post(new UpdatedTweetEvent(tweet);
        }
        @Override
        public void onRun() {
            Tweet tweet = tweetModel.get(tweetId);
            //maybe we've retweeted this from another client before the job could run. It is a good practice to do
            //this check before making an expensive web request.
            if(tweet.getDidRetweet() == false) {
                twitterAPI.reteweet(tweet.getId());
                //API throws an exception if it fails so if it passed this line, we know it succeeded.
                //It goes w/o saying that depending on the API, you may need to check the response.
            }
            tweet.setDidRetweet(true);
            tweet.setDidLocalRetweet(null);
            tweetModel.update(tweet);
            eventBus.post(new UpdatedTweetEvent(tweet);
        }
        @Override
        public void onCancel() {
            Tweet tweet = tweetModel.get(tweetId);
            if(tweet != null && tweet.getDidLocalRetweet() != null) {
                tweet.setDidLocalRetweet(null);
                tweetModel.update(tweet);
            }
            eventBus.post(new FailedToRtweetEvent(tweetId));
        }
        @Override
        protected boolean shouldReRunOnThrowable(Throwable throwable) {
            //handle the exception. maybe it says this tweet has already been retweeted. 
            //or maybe, API returned a 400 error so we should not retry
            //update model accordingly and dispatch necessary events
            //this may also be an internal server error or a network error in which case we should retry
            return shouldRetry;
        }
    }
    

    And finally, our FeedActivity listens for this event and updates the related row.

    File FeedActivity.java

    public void onEventMainThread(UpdatedTweetEvent tweet) { 
        TweetHolder row = findRow(tweet);
        //findRow finds the related tweet row if it is visible. if not, we just ignore it since the next 
        //time it is rendered, it will have the updated data (again, assuming that we are using a single identity
        //scope in our DAO layer. GreenDAO supports this out of the box and is very useful).
        if(row == null) return;
        row.render();
    }
    
    public void onEventMainThread(FailedToRtweetEvent event) {
        //show retweet error dialog
    }
    

    This is it. We’ve :

    • Reflected user changes instantly so they got a feedback.
    • Enqueued related persistent information to ensure we’ll retweet. Meanwhile, did show a different state in the retweet button so that user has a clue on what is going on.
    • Separated the business logic from the UI. This way, the full screen TweetActivity can also listen for the same event.
    • Since this is an important information, our Notification controller can also listen for these events and show notifications if user is not in our app.

    Final Words

    I tried to put together an example on how to write a responsive REST client on Android. Many details are skipped to keep it as short as possible but this article should give you enough idea on how to implement it.

    I have NOT invented any parts of this architecture. It is a well known way of doing things but I see many applications lack such functionality. Hope this will help you improve your user experience.

    If anything is unclear or you disagree, let me know in comments and I’ll make necessary changes or reply concerns.