Monday, June 13, 2011

Multithreaded Renderer on iOS

Hey #iDevBlogADay,

You've probably seen this: You start a game. After the loading is complete, the game runs smooth for a brief period of time, then it suddenly starts stuttering significantly for a few seconds, culminating in a automated banner: "Welcome back to GameCenter". If it's an action game, you may just have lost a life to the stutter.

Last week, I tried to investigate this, and a potential cause was revealed to me: GameCenter is running in the same NSRunLoop as everything else, on the main thread. Hence, when it connects, performs the SSL authentication, encryption, decryption, it blocks the main thread. Apparently, this costs enough time to delay the rendering.

Not only was this cause revealed to me, but also a potential solution: Put the entire OpenGL rendering into a separate thread, so it's not tied to the temper of the NSRunLoop and it's many potential input sources. So I set out to try this.

Multithreading OpenGL Requirements

Writing a multithreaded OpenGL renderer isn't trivial. And due to my perfectionism, I wanted to do it right. That means:
  • Clean implementation
  • Fires exactly at display refresh, using CADisplayLink
  • As simple as possible, trying to avoid any low-level multithreading if possible
  • Since a single EAGLContext can only be used on one thread at a time, ideally everything should be only on the secondary thread, and no code should run on the main thread.
These requirements led me to my first approach.

Using GCD

I love Grand Central Dispatch (GCD). It's a great way to parallelize and defer code execution. And some tests I conducted showed that the overhead caused by blocks is tiny.

Hence, I created a new serial queue (as suggested by the Apple iOS OpenGL documentation), and essentially queued all calls to OpenGL in blocks on that queue, which runs on a different thread. One of the first problems that arose was that setting up CADisplayLink on that thread doesn't work, because the GCD queue threads don't have a NSRunLoop, which is what CADisplayLink uses. Hence, my display link callback wouldn't be called at all.

However, since CADisplayLink doesn't call any OpenGL code by itself, I moved it onto the main thread, and then dispatched the draw event onto the rendering queue from there. Now the callback got triggered at 60hz, as expected. But the rendering didn't work. I'm pretty sure I enforced the right EAGLContext on every draw event. And after a couple of dispatch_asyncs, the [context presentRenderbuffer:] function would stall for 1s at a time. I fiddled around a lot with this, but couldn't get it to work.

If change the rendering queue to run on the main thread (by using the main GCD queue), it magically worked well. But everything was executed on the main thread, which set me back to beginning. That's as far as I've gotten with the GCD approach.

Using a separate NSThread

My second attempt then involved an NSThread. The setup was very simple: When the GLView was created, I created the new thread, and inside I started a NSRunLoop. Then, I set up the CADisplayLink to run on this run loop. And it worked. The CADisplayLink fired reliably and the scene was rendered correctly. However, there was a small issue: There seems to be no (reliable) way to terminate the run loop. Hence, once started, I couldn't stop the rendering anymore. That's not really what I needed.

A classical approach

This is as far as I've gotten. The next thing I want to try is to create an NSThread that runs a very simple loop. It sleeps until a semaphore is fired, then renders one frame, and then sleeps again. Then, on the main thread, I run the display link, and trigger the semaphore whenever the display link fires. This is very old school, but at least I can turn it off at any time, and it appears to match all the requirements I have.


What seemed like a nice little afternoon project starts to cost a lot of time now. However, considering that moving the rendering to a separate thread seems like the best solution for the GameCenter login stuttering problem, and any other stutter that is caused by high runloop latency, it seems worth the effort.

Once I've found a good solution, I plan to make it open-source, and also include my performance monitor thingie that I described in my last post.

To close this post, I'd like to ask everyone out there: Have you written a threaded renderer on iOS? How did you make it work? Did it work reliably?



deeje said...

Apple actually recommends running your OpenGL code in a separate thread. OpenGL can run in any thread, as long as (almost) all of it runs on the same thread (the only exception is texture loading).

Daniel Blezek said...

Would it be easier to log into Game Center in a different thread?

I had to move my AudioQueue code to a different thread for the same reason.

Ken said...

I haven't used CADisplayLink, so maybe I am missing something, but if you run CADisplayLink in the main thread to ping your rendering thread semaphore, won't it still be delayed when Game Center blocks?

volcore said...

Daniel: Yeah, I've tried that too (using GCD, running the GKLocalPlayer authenticate function in a different thread), but it didn't work. It's probably dispatching to the main queue by itself.

Ken: You are right. However, since the rendering is not on the main thread anymore, the chances of missing a display link event are smaller.

In single-threaded mode, if you have one event every 16ms (60hz), and your rendering+updates take 14ms, you have only 2ms for the gamecenter stuff (and everything else) until the next event fires.

In multithreaded mode, the entire rendering is in the second thread, so you have 16ms for the GameCenter stuff, everything else, until the next displaylink event needs to fire.

In my latest testing, though, it seems that the gamecenter stuff is taking way more than 16ms, which means the only way to dodge this is to move everything off the main thread. However, I've been unsuccessful in achieving that.

Vladislav Gubarev said...

Thanks for good article!

I want to share my experience how to use NSThread with NSRunLoop and I stop it.

1. I have a class MyRenderer with with "running" property and instance of NSThread.

2. NSThread created with target "self" and selector "run":
_thread = [[NSThread alloc] initWithTarget:self selector:@selector(run) object:nil];

3. Here is "run" function with NSRunLoop which keep thread working:
- (void)run {
NSRunLoop *runLoop = [NSRunLoop currentRunLoop];
self.running = YES;
[runLoop addPort:[NSMachPort port] forMode:NSDefaultRunLoopMode];
while (self.running && [runLoop runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]]) {

4. To stop thread I have to set "running" property to "NO" in my _thread:
- (void)stop {
if ([NSThread currentThread] != _thread){
[self performSelector:@selector(stop) onThread:_thread withObject:nil waitUntilDone:NO];
[self setRunning:NO];

5. Rendering also have to be performed inside this _thread:
- (void)renderWithArgs:(MyRendererArgs *)args {
if ([NSThread currentThread] != _thread) {
[self performSelector:@selector(renderWithArgs:) onThread:_thread withObject:args waitUntilDone:NO];

... update OpenGL frame buffer here ...

Hope this may be helpful.

vignesh waran said...

Hi this is vignesh i am having 3 years of experience as a android developer and i am certified. i have knowledge on OOPS concepts in android but dont know indepth. After learning hadoop will be enough to get a good career in IT with good package? and i crossed hadoop training in chennai website where someone please help me to identity the syllabus covers everything or not??

Emi Jackson said...

Your blog is really useful for me. Thanks for sharing this informative blog. If anyone wants to get real time Oracle Training in Chennai reach FITA located at Chennai. They give professional and job oriented training for all students.

dhanamlakshmi palu said...

Nice article i was really impressed by seeing this article, it was very interesting and it is very useful for me..Android training in chennai | Android training chennai | Android course in chennai | Android course chennai

dhanalakshmi palu said...

This is extremely helpful info!! Very good work. Everything is very interesting to learn and easy to understood. Thank you for giving information. Phonegap training in chennai | Phonegap training chennai | Phonegap course in chennai | Phonegap course chennai

Sathik Ali. A said...

GREAT INFO ABOUT MOBILE APPS. I AM PLANNING TO DO iOS Training at IICT Chromepet. Please sugesst me where they cover all the required topics.

Their iOS course syllabus link is provided below:

Please revert me as soon as possible guys. I heard IICT Chromepet is the best iOS Training Institute in Chennai.