Most iOS applications have to download data from the internet. Being a mobile developer, sparse resources, limited bandwidth and user-responsiveness needs make this a very interesting problem to tackle. If you are lucky, you might just have one url that you ping to download all the data for your application. But usually, one has multiple urls to download from simultaneously. Since Pulse aggregates your news from multiple sources, this is a particularly important part of the application. After a long search for an elegant and efficient solution for this problem, we discovered that NSOperations and NSOperationQueues make managing simultaneous downloads very easy. This post lists some of our learnings and gives code samples on how to implement it in your own app.

Why NSOperationQueues?

Ideally, you never want to block the main thread that handles user input. Hence, downloading data has to happen in a background thread. We have found that NSURLConnection is an excellent class to download data asynchronously. But, in order to maintain multiple connections simultaneously, one usually has to care about 3 features: (1) Throttling the number of simultaneous downloads (2) Prioritizing connections over one another (3) Easy cancellation and cleanup of such connections. These functions require a lot of bookkeeping that can get quite tedious. NSOperationQueues are extremely helpful in such situations.

An NSOperationQueue is essentially a pool of threads each of which runs a task described by NSOperation objects. It is extremely easy to wrap an asynchronous NSURLConnection in an NSOperation, as we shall see in the next section. Each NSOperation object can be given a priority and added to the queue.

[myOperation setQueuePriority:NSOperationQueuePriorityVeryHigh];
[operationQueue addOperation:myOperation];

An NSOperationQueue object allows you to specify the number of threads is should use and you can easily kill all operations.

[operationQueue setMaxConcurrentOperationCount:3];
[operationQueue cancelAllOperations];

Based on system resources and operation priorities, an NSOperationQueue runs all its operations till they finish. You can pause and resume it at any time, giving you complete control over running tasks in the background with minimal lines of code.

[operationQueue setSuspended:YES];
for (operation in newOperations) {
  [operationQueue addOperation:operation];
}
[operationQueue setSuspended:NO];

Asynchronous downloads using NSOperations

To wrap an NSURLConnection object in an NSOperation, we need to create an NSOperation subclass. When the operation starts, we can initiate an NSURLConnection and implement its delegate methods to collect downloaded data. An NSOperation uses 3 key variables to define its state: isExecuting, isFinished and isConcurrent. Since we want our downloads to run in parallel, we set isConcurrent to YES. Both isExecuting and isFinished need to be tracked in a key-value coding compliant manner. The operation is only considered finished when the isFinished property changes to YES. Check out DownloadURLOperation.h and DownloadURLOperation.m to learn more on how this is done.

New Subclass in Action

Now that we have our DownloadURLOperation subclass in place, we add its objects to operation queues and start downloading data. If an operation queue is empty, an operation starts executing as soon as it is added to the queue.

downloadOperation_ = [[DownloadURLOperation alloc] initWithURL:url];
// Add an observer to get notified when the download finishes
[downloadOperation_ addObserver:self forKeyPath:@"isFinished" options:NSKeyValueObservingOptionNew context:NULL];
[queue addOperation:downloadOperation_];

To process the downloaded data after an operation is finished, we need to observe KVO notifications.

- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)operation change:(NSDictionary *)change context:(void *)context {
  if ([operation isEqual:downloadOperation_]) {
    [downloadOperation_ removeObserver:self forKeyPath:@"isFinished"];
    [downloadOperation_ release];
    downloadOperation_ = nil;
    NSData *data = [downloadOperation_ data];
    NSError * error = [downloadOperation_ error];
    if (error != nil) {
      // handle error
    } else {
      // process data
    }
  }
}

Whenever necessary, remember to cancel the operation.

- (void)dealloc {
  if (downloadOperation_ != nil) {
    [downloadOperation_ removeObserver:self forKeyPath:@"isFinished"];
    [downloadOperation_ cancel];
    [downloadOperation_ release];
    downloadOperation_ = nil;
  }
  [super dealloc];
}

Benefits

Here is a quick summary of a some important benefits of wrapping NSURLConnections in NSOperations and using NSOperationQueues to manage simultaneous downloads.

  1. Throttling: NSOperationQueue allows you to set the maximum concurrent operations allowed to run. Thus, we can fine tune this number based on device performance and network constraints.
  2. Chained Downloads: If you need multiple downloads to happen one after the other in a sequence, using NSOperations allows you to execute chained downloads easily in a single class.
  3. Code Cleanliness: This pattern allows you to encapsulate downloading and processing data in the same class. For example: YAJLParserOperation.h and YAJLParserOperation.m parseJSON data on the fly as it is downloaded.

Although asynchronous NSURLConnection downloads happen on a background thread, its delegates are always called in the main thread in the NSDefaultRunLoopMode. This means that the delegates would never be called when the user is touching the interface (say while scrolling a tableview or tapping buttons). Even if a connection runs inside an NSOperation, this important property is maintained since we ensure that the connection starts on the main thread.

Here is some sample code that you can play around with to learn more. Please leave comments for any suggestions or improvements.