Controller lag and inaccuracy

goodhustle · January 2013

@Ayrik, that's a pretty cool solution, basically just wrapping the ODK OuyaController class directly.

Optimizing it further, one could imagine using badlogic's suggestion to fetch all axis/button values for a controller at once. From reading the JNI docs, it seems to me that the stack frames you're pushing and popping are for global/local refs and don't have anything to do with floats and ints being used in static calls, so it shouldn't trigger GC in theory.

In the interest of trying out easier solutions, I'm going to take a look at UnityPlayerNativeActivity subclassing with event forwarding: http://docs.unity3d.com/Documentation/Manual/PluginsForAndroid.html

tgraupmann · January 2013

When I tried UnityPlayerNativeActivity, I wasn't seeing all the controller events and keycodes for every controller.

goodhustle · January 2013

@tgraupmann Doh. I only have the OUYA controllers, so it probably wouldn't be useful for me to try that if it means it wouldn't work with other setups. Did you file a bug with the Unity guys like @AngryAnt? It's probably something they would want to know about.

tgraupmann · January 2013

Two potential problems with a bug report. 1. Unity QA/Dev needs access to the hardware. 2. OUYA folks have been using the ODK and ramped up using it, I would think QA would see it and hide under a desk in horror. They'd have to some video just to understand the bug and maybe the repro steps.

That said, maybe we could get Erik at Unity an OUYA.

Or... we just handle it at the Java level...

goodhustle · January 2013

Okay, so I actually went about calling into JNI every frame with Unity's AndroidJavaClass / AndroidJavaObject helper classes, and it was actually really straightforward with a few hiccups.

First off, ODK OuyaController.getButton is broken for everything but O,U,Y,A buttons, so you can't just call it to get the state of any button on a controller. I ended up keeping track of button state and updating it in onKeyUp/onKeyDown events. Then I can call a static method to fetch the controller state for a player number and use that data on the Unity side.

If you're feeling adventurous, here's some example code to use at your own risk: http://pastebin.com/JmhLT1iV

I don't know if it's any more performant than the SendMessage approach. I'll just take @Ayrik's word for it that it worked much better. There are no JNI refs created either, so I don't think it will trigger GC. Time will tell, I guess.

A few extra notes about this:

* I think relying on OuyaController entirely would be nicer, and I'll switch to that when the ODK is fixed. Then you wouldn't need the static array and those big switch statements in onKeyUp/Down.

* In my game, I'm calling this every frame and it works fine. Polling through JNI vs. SendMessage perhaps twice a frame is anyone's guess.

* The Unity SDK should import OuyaController and use its getPlayerNumForDeviceId to assign the device playerId. Right now it just increments monotonically in the order it gets back from InputDevice.getDeviceIds(), which is bad.

tgraupmann · January 2013

Wow nice info.

Right now it just assigns the player id in order which is just a placeholder for when the player id is linked to a controller profile.

Litteratus · January 2013

goodhustle: In your revised onKeyDown and onKeyUp methods you call -

int playerId = OuyaController.getPlayerNumByDeviceId(event.getDeviceId());

However OuyaController doesn't have that function.. at least in my version it doesn't. Is this something you wrote? I've tried a couple things, but failed pretty good so far.

goodhustle · January 2013

@Litteratus, check out the ODK javadoc documentation that comes along with the full ODK. I think that is accurate. But you need to import the OuyaController class to get to all of it too, which involves an import statement at the top of OuyaUnityApplication.java and also some messing around with OuyaPanel or OuyaMenuAdmin to get the ouya sdk in the classpath. Like I said, for those feeling adventurous. :)

Litteratus · January 2013

@goodhustle: ;Wow.. believe it or not I have been working off of ODK 0.0.4 this whole time... no wonder I didn't see that function. I wonder what's in store for me now...

Thanks again!

yankijp · February 2013

Hi y'all! Is there a package of something newer for fast input? I have permission to setup my Ouya at a gaming event in a week and it would be great to show off things running on actual hardware instead of my Slate machine with XBOX controllers.

Thanks!

FelineArts · February 2013

having the lag issue too :(

tgraupmann · February 2013

The next package doesn't have the faster updates yet. I'm focusing on the purchase encryption updates from 0.0.6.

FelineArts · February 2013

just a little coment, i changed the batteries and the lag was reduced

tgraupmann · February 2013

Is that the "turbo" brand of batteries?

FelineArts · February 2013

i still have the duracell what came with the dev kit, there is still lag, but it was significally reduced

yankijp · February 2013

Ok so I followed goodhustle's code from pastbin. I managed to get it to compile after adding the classpath and do not get any errors back. Now in Unity I just created a script and attached it to my camera as a test:

AndroidJavaClass jc = new AndroidJavaClass("tv.ouya.demo.OuyaUnityApplication"); AndroidJavaObject data = jc.CallStatic<AndroidJavaObject>("GetControllerState", 1); float axisLSX = data.Get<float>("AxisLSX");

It doesn't look like that AndroidJavaClass can find the class. This is what I get in logcat:

W/System.err( 1771): java.lang.ClassNotFoundException: tv.ouya.demo.OuyaUnityApp

lication

I am a bit sleepy so I will pass out and continue tomorrow. Any input appreciated.

Thanks!

goodhustle · February 2013

@yankijp, are you running on the actual device? None of the java-side stuff runs in the editor, so I usually wrap it with #if UNITY_OUYA && !UNITY_EDITOR and provide fallback to regular Unity Input in my own code.

yankijp · February 2013

@goodhustle, Yes I am running on the actual device. The class not found exception I am getting in logcat when it is deployed.

I did your changes on top of a clean newest ODK release with the latest firmware on my Ouya.

The first challenge was getting the classpath in there to get it to compile the java but that is all good now along with the import statement for the sdk. So I am not getting any errors on compile/deploy.

Thanks!

goodhustle · February 2013

@yankijp My next step would be to make sure all the bundle identifiers match. Did you change it away from tv.ouya.demo.OuyaUnityApplication (which I think is the default)? If so, did you get it changed in AndroidManifest.xml, OuyaUnityApplication's package declaration, and the bundle identifier in the build settings?

yankijp · February 2013

They are set to the default and I haven't changed them. I will double check when I get home though to make sure they match.

Thanks!

yankijp · February 2013

Yay! Almost there! Ok So it compiles fine and I am getting button states back from the controller... the only issue is that for anything calling getAxisValue it is returning 0. So I am not getting thumbstick directions at all. I code from Ayrik and goodhustle but it isn't working. In Unity I am calling:

AndroidJavaObject data = jc.CallStatic<AndroidJavaObject>("GetControllerState", 0);

float axisLSY = data.Get<float>("AxisLSY");

float axisLSX = data.Get<float>("AxisLSX");

bool oButtonState = data.Get<bool>("ButtonO");

//testing Arvik's code here

float xt = jc.CallStatic<float>("GetLeftX", 0);

I am getting correct feedback from the button but the floats are zero.

Any feedback on this? I am hoping to have it working by tomorrow night as I am going to let regular people play my build at a festival. :)

tgraupmann · February 2013

You can use the same ODK input overrides to save the state information in Java. And then pull the states with your JNI interface instead of sending across with JSON.

Ayrik · February 2013

I JUST FIXED ALL THIS FOR GOOD (I'm really excited!). I discovered what "pinned memory" truly means! OK, so I just got home from work, but tomorrow, if I have time, I'd like to post something a little easier so you don't have to piece it all together. I will be referring to this document: http://pastebin.com/JmhLT1iV that @goodhustle created. (I apologize for the formatting. I really don't know how to do anything in this forum)

Step 1:

Remove lines 15-19 since they're duplicates.

Step 2:

Replace Line 176. ControllerState data = new ControllerState();

With: ControllerState data = playerStates[playerNum];

Step 3:

Remove lines 178-209

Step 4:

Insert the following lines int onGenericMotionEvent() between Line 142 and 143:

int playerNum = OuyaController.getPlayerNumByDeviceId(event.getDeviceId());

ControllerState data = playerStates[playerNum];

OuyaController c = OuyaController.getControllerByPlayer(playerNum);

data.AxisLSX = c.getAxisValue(OuyaController.AXIS_LS_X);

data.AxisLSY = c.getAxisValue(OuyaController.AXIS_LS_Y);

data.AxisRSX = c.getAxisValue(OuyaController.AXIS_RS_X);

data.AxisRSY = c.getAxisValue(OuyaController.AXIS_RS_Y);

data.AxisLT = c.getAxisValue(OuyaController.AXIS_L2);

data.AxisRT = c.getAxisValue(OuyaController.AXIS_R2);

Ayrik · February 2013

Step 5:

Now, on the Unity side, the most important thing is to remove the using block, and store both the AndroidJavaClass jc, and the AndroidJavaObject in some sort of object. I made a MyOuyaInput.cs which I will share hopefully tomorrow, but PM me if I haven't yet. It stores an array of AndroidJavaObjects, one for each controller.

Step 6:

NOW, ONLY CALL jc.CallStatic<AndroidJavaObject>("GetControllerState", i) ONCE, EVER! Since it's pointing to memory stored in Java, it will continue to point to it and update as necessary automatically. You will still need to call data.Get<float>("AxisLSX"); for every controller state you want to keep track of, which is the next step.

Step 7:

In some Update function, loop through all the controllers and save the states of everything in some storage class/structs. I store the current state, and the previous frame's state, so I can detect down states.

Step 8:

It is VERY important that you Change the script execution order of your new ouya input script to the very FIRST slot if you plan to detect down states based on previous button states.

---------

Hopefully that is everything! Following these steps will alleviate all the GC calls you get every frame, and your game will run smooth again.

This method also simplifies the code, making it almost identical to Unity's built in input system, so the wrapper is really simple. I believe, also, that all the crazy switch statements in OuyaInputManager are no longer necessary either, since OuyaController in Java takes care of all that for you. I still use all that for the editor though, with several of my own changes.

Sorry guys I couldn't sleep until I posted this tonight! Time for bed.

goodhustle · February 2013

@Ayrik, cool! That's pretty good, sorry about the dup code, I think I probably just double-pasted it by mistake. I didn't know that it would keep the reference to the static var "live". However, if it's live access instead of making copies, don't we still need to implement something like what badlogic suggested in terms of either double buffering or locking? In any case, I'll be making these changes and checking them out tomorrow too. :)

tgraupmann · February 2013

Don't forget about the OnPause and OnResume java event. Who knows what happens to those JNI references after you exit the game. And if they are static they may survive across scene loads.

goodhustle · February 2013

@tgraupmann The docs say that AndroidJavaClass/AndroidJavaObject are the highest-level JNI API that unity offers, and is more or less a black box. They implement a lot of caching, so if we want finer-grained control over persistence we probably need to drop into lower-level Unity JNI API calls (or just talk to Unity if we run into problems).

I will say though that they both implement IDisposable, so I would imagine we could call Dispose on them when necessary (such as in OnApplicationQuit or pause/resume listeners) since it would forgo the using() blocks.

yankijp · February 2013

Holy crud I think it's working now. I did Ayrik's changes and some other joyousness. At the basic level it looks like I am getting input. :D

p.s. THANK YOU! :)

Ayrik · February 2013

tgraupmann said:

Don't forget about the OnPause and OnResume java event. Who knows what happens to those JNI references after you exit the game. And if they are static they may survive across scene loads.

Pausing and resuming all seems to work fine for me now after adding the m_UnityPlayer.pause() and resume() in the correct places. All except for SkinnedMeshRenderers that are currently loaded get all their UVs reset to 0,0 for some reason (I think that's what's happening).

goodhustle said:

@tgraupmann The docs say that AndroidJavaClass/AndroidJavaObject are the highest-level JNI API that unity offers, and is more or less a black box. They implement a lot of caching, so if we want finer-grained control over persistence we probably need to drop into lower-level Unity JNI API calls (or just talk to Unity if we run into problems).

I will say though that they both implement IDisposable, so I would imagine we could call Dispose on them when necessary (such as in OnApplicationQuit or pause/resume listeners) since it would forgo the using() blocks.

Now that I understand what's going on a bit more and am less afraid to dive into Java, I will continue to do some research to improve this. I am, in fact, calling Dispose() on both objects in my OnDestroy method of MyOuyaInput.cs, and I imagine the OS is disposing these properly on app shutdown anyway, but better safe than sorry.

yankijp said:

Holy crud I think it's working now. I did Ayrik's changes and some other joyousness. At the basic level it looks like I am getting input. :D

p.s. THANK YOU! :)

Glad to help. Couldn't have done it without the other awesome people in this community like @tgraupmann @goodhustle and a few others especially.

I'm still curious about other methods like the native player activity, and the lower level JNI code Unity provides, but I'm satisfied with the results so far.

tgraupmann · February 2013

My experience with native player activity was that using it caused Unity to eat a bunch of input that should have gone to Java. That's why the ODK implements Activity and loads the Unity player within the main layout. You'll notice after using a bunch of controllers that not all the events were recognized unless I derived from Activity.

Some of the Create Contest Entries wanted to use a Quit button to completely exit their OUYA application.

Which way would you recommend?

Would you just call Application.Quit in Unity which might kill the Unity player and leave the Java running? Or would you add a JNI interface to kill the application on the Java side? And how would you go about that?

Howdy, Stranger!

Categories

Controller lag and inaccuracy

Comments