Cocos2d x 3.0 multi threaded load resource instances asynchronously

From Cocos2d-x version 2.x version to Cocos2d-x version 3.0 Final version just released last week, the engine drive core of Cocos2d-x version is still a single thread "endless loop". Once a frame encounters "big work", such as Size's large texture resource loading or network IO or a large amount of calculation, the picture will inevitably appear stuck and slow response phenomenon. Since the days of the old Win32 GUI programming, the Guru have told us: don't block the main thread (UI thread), let the Worker thread do the big work.

Mobile games, even casual games, often involve a large number of texture resources, audio and video resources, file reading and writing as well as network communication, processing a little bit less will appear picture jam, interaction is not free. Although the engine provides some support in some aspects, sometimes it is more flexible to use the Worker thread itself. Let's take the Cocos2d-x 3.0 Final version of the game as an example (for Android platform) and talk about how to load multithreaded resources.

We often see some mobile games, after the launch will first show a screen with the company Logo flash screen (Flash Screen), then will enter a game Welcome scene, click "start" to officially enter the main scene of the game. Here Flash Screen showcase, often in the background will do one thing at a time, and other that is the image resources, load the game music sound resources and configuration data is read, this is a "smoke screen", the goal is to improve with experience, so that subsequent scene rendering and scene switching use has cache directly to the data in memory, without any load.

1. Add FlashScene to your game

When the game App is initialized, we first create FlashScene to make the game display FlashScene screen as soon as possible:

// AppDelegate.cpp
bool AppDelegate::applicationDidFinishLaunching() {
     ...   ... 
    FlashScene* scene = FlashScene::create();
    pDirector->runWithScene(scene);
    return true;
}

At FlashScene init, we create one Resource Load Thread, and we use one ResourceLoadIndicator as the medium for the interaction between the rendering thread and the Worker thread.

//FlashScene.h
struct ResourceLoadIndicator {
    pthread_mutex_t mutex;
    bool load_done;
    void *context;
};
class FlashScene : public Scene
{
public:
    FlashScene(void);
    ~FlashScene(void);
    virtual bool init();
    CREATE_FUNC(FlashScene);
    bool getResourceLoadIndicator();
    void setResourceLoadIndicator(bool flag);
private:
     void updateScene(float dt);
private:
     ResourceLoadIndicator rli;
};
// FlashScene.cpp
bool FlashScene::init()
{
    bool bRet = false;
    do {
        CC_BREAK_IF(!CCScene::init());
        Size winSize = Director::getInstance()->getWinSize();
        //FlashScene Your own resources can only be loaded synchronously 
        Sprite *bg = Sprite::create("FlashSceenBg.png");
        CC_BREAK_IF(!bg);
        bg->setPosition(ccp(winSize.width/2, winSize.height/2));
        this->addChild(bg, 0);
        this->schedule(schedule_selector(FlashScene::updateScene)
                       , 0.01f);
        //start the resource loading thread
        rli.load_done = false;
        rli.context = (void*)this;
        pthread_mutex_init(&rli.mutex, NULL);
        pthread_attr_t attr;
        pthread_attr_init(&attr);
        pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
        pthread_t thread;
        pthread_create(&thread, &attr,
                    resource_load_thread_entry, &rli);
        bRet=true;
    } while(0);
    return bRet;
}
static void* resource_load_thread_entry(void* param)
{
    AppDelegate *app = (AppDelegate*)Application::getInstance();
    ResourceLoadIndicator *rli = (ResourceLoadIndicator*)param;
    FlashScene *scene = (FlashScene*)rli->context;
    //load music effect resource
     ...   ... 
    //init from config files
     ...   ... 
    //load images data in worker thread
    SpriteFrameCache::getInstance()->addSpriteFramesWithFile(
                                       "All-Sprites.plist");
     ...   ... 
    //set loading done
    scene->setResourceLoadIndicator(true);
    return NULL;
}
bool FlashScene::getResourceLoadIndicator()
{
    bool flag;
    pthread_mutex_lock(&rli.mutex);
    flag = rli.load_done;
    pthread_mutex_unlock(&rli.mutex);
    return flag;
}
void FlashScene::setResourceLoadIndicator(bool flag)
{
    pthread_mutex_lock(&rli.mutex);
    rli.load_done = flag;
    pthread_mutex_unlock(&rli.mutex);
    return;
}

We checked the indicator flag bit in the timer callback function. When we found that ok was loaded, we switched to the following game start scene:

void FlashScene::updateScene(float dt)
{
    if (getResourceLoadIndicator()) {
        Director::getInstance()->replaceScene(
                              WelcomeScene::create());
    }
}

This completes the initial design and implementation of FlashScene. Try Run1.

2. Fix breakdowns

On GenyMotion's 4.4.2 emulator, the game did not run as I expected, and the game crashed when FlashScreen appeared.

After analyzing the running logs of the game through monitor, we can see the following abnormal logs:

threadid=24: thread exiting, not yet detached (count=0)
threadid=24: thread exiting, not yet detached (count=1)
threadid=24: native thread exited without detaching

It is strange that we set PTHREAD_CREATE_DETACHED when we created the thread:

pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);

How can this problem still occur, and there are actually 3 logs. I looked at the code TextureCache::addImageAsync in the engine kernel and found nothing special in the thread creation and thread main function. Why can the kernel create threads when I would crash if I created them myself? Debug goes back and forth several times, and the problem seems to focus on the tasks performed in resource_load_thread_entry. In my code, I used SimpleAudioEngine to load sound resources and UserDefault to read some persistent data. If I removed these two tasks, the game would move to the next stage without crashing.

What do SimpleAudioEngine and UserDefault have in common? Jni calls. Yes, both of these interfaces need to be adapted to multiple platforms at the bottom, while for the Android platform, they both use the interface provided by Jni to call the methods in Java. And Jni is constrained for multithreading. The Android developer website has this quote:

All threads are Linux threads, scheduled by the kernel. They're usually started from managed code (using Thread.start), but they can also be created elsewhere and then attached to the JavaVM. For example, a thread started with pthread_create can be attached with the JNI AttachCurrentThread or AttachCurrentThreadAsDaemon functions. Until a thread is attached, it has no JNIEnv, and cannot make JNI calls.

It seems that the new thread created by pthread_create cannot make Jni interface calls by default, unless Attach goes to Vm, obtains an JniEnv object, and requires Detach Vm before exit. Well, let's try 1. The Cocos2d-x engine provides some JniHelper methods for Jni operations.

#if (CC_TARGET_PLATFORM == CC_PLATFORM_ANDROID)
#include "platform/android/jni/JniHelper.h"
#include <jni.h>
#endif
static void* resource_load_thread_entry(void* param)
{
     ...   ... 
    JavaVM *vm;
    JNIEnv *env;
    vm = JniHelper::getJavaVM();
    JavaVMAttachArgs thread_args;
    thread_args.name = "Resource Load";
    thread_args.version = JNI_VERSION_1_4;
    thread_args.group = NULL;
    vm->AttachCurrentThread(&env, &thread_args);
     ...   ... 
    //Your Jni Calls
     ...   ... 
    vm->DetachCurrentThread();
     ...   ... 
    return NULL;
}

About what is JavaVM, what is JniEnv, Android Developer official documents describe as follows:

The JavaVM provides the "invocation interface" functions, which allow you to create and destroy a JavaVM. In theory you can have multiple JavaVMs per process, but Android only allows one.
The JNIEnv provides most of the JNI functions. Your native functions all receive a JNIEnv as the first argument.
The JNIEnv is used for thread-local storage. For this reason, you cannot share a JNIEnv between threads.

3. Solve the black screen problem

The code above successfully resolved the thread crash problem, but that's not the end of the story, because we then had another "black screen" event. The so-called "black screen" is not completely black. However, when entering the game WelcomScene, only instances of LabelTTF in Scene can be displayed, while the rest of Sprite cannot be displayed. Obviously it must have something to do with us loading texture resources in Worker thread:

SpriteFrameCache::getInstance()->addSpriteFramesWithFile("All-Sprites.plist");

We built SpriteFrame by compressing the image into a large texture, which is recommended by Cocos2d-x. But to get to the root of the problem, look no further than the monitor journal. We did find some unusual logs:

libEGL: call to OpenGL ES API with no current context (logged once per thread)

According to Google, only Renderer Thread can be called egl, because context of egl is created in Renderer Thread, Worker Thread does not have context of EGL. context cannot be found when egl is operated, so the operation fails and the texture cannot be displayed. To solve this problem, take a look at how TextureCache::addImageAsync does it.

TextureCache::addImageAsync only loads image data in the worker thread, while the texture object Texture2D instance is created in addImageAsyncCallBack. This means that the texture is still created in the Renderer thread, so there won't be the "black screen" problem we had above. Imitating addImageAsync, let's modify the following code:

static void* resource_load_thread_entry(void* param)
{
     ...   ... 
    allSpritesImage = new Image();
    allSpritesImage->initWithImageFile("All-Sprites.png");
     ...   ... 
}
void FlashScene::updateScene(float dt)
{
    if (getResourceLoadIndicator()) {
        // construct texture with preloaded images
        Texture2D *allSpritesTexture = TextureCache::getInstance()->
                           addImage(allSpritesImage, "All-Sprites.png");
        allSpritesImage->release();
        SpriteFrameCache::getInstance()->addSpriteFramesWithFile(
                           "All-Sprites.plist", allSpritesTexture);

        Director::getInstance()->replaceScene(WelcomeScene::create());
    }
}

After finishing this 1 modification, the game screen becomes 1 cut normal, and the multi-threaded resource loading mechanism officially takes effect.