Controller lag and inaccuracy

Shiels · February 2013

One thing I'm very confused about... what import statement(s) for the ouyasdk are required for OuyaUnityApplication.java exactly?

yankijp · February 2013

Shiels:

import tv.ouya.console.api.OuyaController;

@Arik,@goodhustle cool! Thanks tons! I am mostly there now! :) One tiny thing. My triggers are returning 0. I might have something wrong as I am tired but it looks correct on my end.

On unity side:

.Get<float>("AxisRT");

.Get<float>("AxisLT");

I am sooooooo close as the rest of it is working and my input has been moved over. It seems smoother/faster now with 4 inputs at once. Its hard for me to test as I have to push on 4 controllers myself. :D

Ayrik · February 2013

I was planning on looking into the triggers tonight.

goodhustle · February 2013

tgraupmann said:
Some of the Create Contest Entries wanted to use a Quit button to completely exit their OUYA application.

Which way would you recommend?

Would you just call Application.Quit in Unity which might kill the Unity player and leave the Java running? Or would you add a JNI interface to kill the application on the Java side? And how would you go about that?

I've been using Application.Quit on Beast Boxing Turbo, and ever since implementing the Unity-standard calls in the Activity I haven't noticed any zombie app processes or audio bugs - it all runs in the same process, and I think the UnityPlayer actually handles saving playerprefs and tearing things down in Application.Quit or through its UnityPlayer.Quit method in java. So JNI shouldn't be necessary AFAIK. I'm also not using NativeActivity for this, this is going through your existing custom Activity.

tgraupmann · February 2013

Beast Boxing Turbo is awesome!

goodhustle · February 2013

@tgraupmann Thanks! :) It was actually really dependent on the Unity-standard Input idioms, so I had to write my own input abstraction layer on top of OuyaGameObject/OuyaSDK and fork the java side to keep everything working cross-platform in a single codebase. If you want, I can send you my latest code with these JNI changes, and you and @HashbangGames can see if you want to incorporate anything from in there to accomodate people like me with existing large projects. It was all based on 0.4, but I've been keeping the ouya-sdk.jar up to date and patching in your changes from time to time.

tgraupmann · February 2013

The core team should be sending me 4 controllers so that I can test the lag. I'll ping you when I can reproduce. I follow what you've done here, and it's pretty much how I planned. With a few tests we can get the optimal performance.

yankijp · February 2013

@Ayrik thanks tons! :) Looking forward to it.

tgraupmann · February 2013

I saw a comment about understanding pinned memory. Let me show what I meant by that.

Pinned memory is like sharing the same exact memory data between C# and C++ without marshalling it back and fourth.

For example say you have a Texture2D and you want to edit the pixels in C++ without passing buffers around.

First you need the texture:
Texture = new Texture2D((int)m_captureRect.width, (int)m_captureRect.height, TextureFormat.ARGB32, false);

And then you get the pixel data:
m_pixels = Texture.GetPixels32(0);

And then you pin the handle:
m_pixelsHandle = GCHandle.Alloc(m_pixels, GCHandleType.Pinned);

You get a pointer to the pinned handle:
m_pinnedPointer = m_pixelsHandle.AddrOfPinnedObject();

And if the pointer is not null:
if (m_pinnedPointer != IntPtr.Zero)

You can pass the pinnedPointer to C++
m_captureThreadId = SendToCPlusPlus(m_pinnedPointer);

To directly access the array data.

I wanted to do something like this across Java, which is somewhat like the JNI calls.

This would be for the fastest < 1ms controller input.

Anyway just a little background to think about.

yankijp · February 2013

Thanks Tim for that info! :) Just a quick thank you to all. It looks like my input lag is pretty much gone using 4 controllers. I am using right and left button for now until the triggers get worked out but that means I am bringing my Ouya dev kit out to the wild on Saturday and Sunday for regular gamers to play my Ouya Create entry Space Operetta!

Thank you all so much!! :)

tgraupmann · February 2013

Ah I didn't try that one yet. DropBox link us when you are ready. :P

nimble_gorilla · February 2013

Is there a semi-official spot for the latest unity stuff? I see code snippets in this and other threads, but I'm not sure what the latest status is.

tgraupmann · February 2013

The ODK 0.0.6 contains "1C" the last stable release. I have "1D" which has the OnPause fixes that you can PM me for.

Ayrik · February 2013

Just thought I'd post my research. I've been doing everything I can to eliminate controller latency, but now I'm just not so sure it's possible. If I plug in an Xbox 360 controller (4, actually), then it works perfectly, but the OUYA controllers cause all sorts of lag. I'm getting several onGenericMotionEvent messages every frame. It is fine when I don't do anything in that function, but I have to with my new changes to fix the frame rate. Anyway, I'm convinced this is a hardware problem...I may try to disconnect the triggers and see what happens.

tgraupmann · February 2013

What if you did something crazy like override the button events only to return that they've been handled while keeping track of when they happened, without sending the information to Unity?

And then you just have Unity poll for the history information like once per second?

Litteratus · February 2013

@tgraupmann

I don't think it's a matter of how often they are being polled.. I am seeing the same issues, when I am using one Ouya controller, everything is silky smooth. Two controllers introduces a strange lag. If I were to unpair, and plug Xbox 360 controllers in, the lag is completely gone, even during the same session.

Polling for input once per second and reacting to the history would not work. When I press the joystick forward, there is a chance that it would be the very first command in the history and it would take 1 second for my character to respond in game.

At this point, we have basically given up on developing for a 4 player local mode and focused on 1 player per Ouya online multiplayer... in hopes that new hardware will solve our issues.

Arakade · February 2013

tgraupmann said:

I saw a comment about understanding pinned memory. Let me show what I meant by that.
...
To directly access the array data.

I wanted to do something like this across Java, which is somewhat like the JNI calls.

I believe you're looking for NIO's ByteBuffer in Direct Mode:

http://docs.oracle.com/javase/6/docs/api/java/nio/ByteBuffer.html

For a clearer explanation, see the bottom section titled "6: Direct Buffers" in:

http://onjava.com/pub/a/onjava/2002/10/02/javanio.html?page=2

A quick check suggests it's at least available on Android:

http://developer.android.com/reference/java/nio/ByteBuffer.html#wrap(byte[])

But I haven't validated that this definitely actually works in Android's Dakvik yet ... But it definitely looks possible!

I don't have a devkit but, as an ex-Java guru, I'm happy to assist if I can?

Otherwise I'm cheering along with others here for you good folk getting us a great gamedev environment :-)

HTH, Rupert

tgraupmann · February 2013

Wow @Arakade that looks awesome. We will start doing regular G+ hangouts and maybe we can cover this topic. If you want to run any APK tests, send a DropBox link, and I'll run it.

Arakade · February 2013

tgraupmann said:

Wow @Arakade that looks awesome.

Happy to help :-)

We will start doing regular G+ hangouts and maybe we can cover this topic.

Great! Hit me up if/when! (/ How will we hear about them?)

If you want to run any APK tests, send a DropBox link, and I'll run it.

Wow! That's a superawesome kind offer! Any and all feedback ultra welcome! (I am gonna be ready for backer release day! I AM! :-) )

tgraupmann · February 2013

I'll start a thread about the next hangout.
http://forums.ouya.tv/discussion/710/g-hangout-scheduling

Arakade · February 2013

Brilliant, starred! Looking forward to it! (Hope I can make it - tomorrow at midnight GMT! Even then there is no WAY I'll manage to watch 10 hours of video tomorrow! o_O :-) )

Thanks again! G'Night!

Arakade · February 2013

Hey, guys

Per my previous post and last night's G+ Hangout, I put together a little example of 'memory pinning' between C and Java. Since I was rather low on time today, I decided it'd be fastest to start with a simple ANSI C <-> Java base. It...

starts as Java (like Ouya)
makes a JNI call to C which...
allocates some memory (calloc()'d) then...
'pins' that in memory for Java and returns.
Java then updates a byte (for now) and notifies C via JNI
C prints the byte
Java tells C to tidyup (which it does)
program done

Per ByteBuffer doc, there are various methods (bulk and single, typed or byte-based) for modifying and reading this buffer. Obviously the choice of which is/are best comes down to what the specific needs are and measuring performance. If representing everything as raw bytes (e.g. +/-128 for joystick axes and maybe bitfields for sets of buttons) suffices, we can probably just do mass byte writes? Otherwise there are obviously float methods, etc.

The code's here: https://dl.dropbox.com/u/26410379/JNINativeMemory1.tar.gz

All feedback welcome :-) (man, it's been a while since I did C!)

Is this enough base for one of you smart chaps to work from?

If I'm still feeling enthusiastic tomorrow, I might see about integrating this into either (a) Tim's C# sample or better (b) directly into the Ouya plugin (possibly requiring a Windows build environment setup -- for convenience, I used Linux).

If the latter, this leads to the questions...

What's the most up-to-date code to work from? (Ayrik's Feb 8'th post?)

I'll split the rest of my questions into another post since they're kinda involved (aka long :-D )

TBC!

Arakade · February 2013

The questions/discussion on implementation approach...

It seems people will want both event-based and polling. It occurs these might have different efficiency patterns. Should we allow enabling one per-project?

I envision the most efficient polling pathway is duplicating the OuyaController's state-querying facility in C# but having it work from a shared memory segment (shm) updated by Java. Fair?

Ideally Java directly modifies the shm atomically (if such can be done in a C#-compatible way?) That way Java can write directly into the shared memory (assuming that's fastest). This probably requires using only single byte updates which is likely a non-stater, fair?

Next best probably involves a fast poll from C#->C->Java which bulk copies latest input state (maintained separately) into the shm then returns. We can then call this from Update() (/ FixedUpdate()?) once for all controllers before querying the C# OuyaController individually. Fair?

If no contraindication, I might investigate this next.

What benefits might shm offer to event-based version? I suspect event-based probably needs to avoid excess JNI calls since this could preoccupy the CPU too much...? Thoughts welcome.

Anyway, more than enough from me!

Cheers, Rupert.

P.s. just at the end of writing all this, I came across references to reading joysticks from NativeActivity using AMotionEvent_getAxisValue() ! Seems it's not fully supported yet. Still, interesting stuff that perhaps we can lobby Ouya to improve?

Unity discussion:

http://forums.ouya.tv/discussion/396/nativeactivity-right-analog-input

Vote for issue fix in Android source:

http://code.google.com/p/android/issues/detail?id=47150

newmessage · February 2013

Litteratus said:

Ayrik said:
OuyaUnityApplication.java:

@Override
public boolean onGenericMotionEvent(MotionEvent event) {
boolean handled = OuyaController.onGenericMotionEvent(event);
return handled || super.onGenericMotionEvent(event);
}

Ayrik - I attempted to implement this, and it's quite clear that I am not up to speed on my Java..
I believe the relevant error that I am getting is:

[Results] elapsedTime: 0.6164112 errors: D:\AirborneDynamo\ABD_Multiplayer\Assets\Plugins\Android\src\OuyaUnityApplication.java:368: cannot find symbol
symbol : class OuyaController
location: class com.Litteratus.AirborneDynamo.OuyaUnityApplication
OuyaController c = OuyaController.getControllerByPlayer(playerNum);
^

I know the OuyaController lives in the ouya-sdk.jar, and I was pretty sure import tv.ouya.sdk.*; would cover the entire library. What could I be missing?

Thanks
Shane

well, I think U missed to include ouya sdk on your package

Arakade · February 2013

newmessage said:

Litteratus said:

Ayrik said:
OuyaUnityApplication.java:

@Override
public boolean onGenericMotionEvent(MotionEvent event) {
boolean handled = OuyaController.onGenericMotionEvent(event);
return handled || super.onGenericMotionEvent(event);
}
<snip>
I know the OuyaController lives in the ouya-sdk.jar, and I was pretty sure import tv.ouya.sdk.*; would cover the entire library. What could I be missing?

well, I think U missed to include ouya sdk on your package

...and, helpfully, it's tv.ouya.console.api.OuyaController :-)

Arakade · February 2013

Just as headsup, I'm working on the as-near-zero-latency-as-possible version proposed above using 'pinned memory'. Initial version will cover polling controls. If there's sufficient interest, an event-based version might follow (the choice is settable).

To give you an idea of the difference, JNI is generally considered slow by Java gamedevs hence something to be avoided where possible (but also a sometimes necessary evil).

For a Unity game polling all basic controls (2 sticks * 2 axes, 2 triggers, all 6 regular buttons + 1 system control = 13), @goodhustle's much improved version still likely makes 13 JNI calls each frame per controller (so that's 52 JNI calls per frame). These each involve JNI marshalling and unmarshalling the response (unknown cost -- maybe zero, maybe not?). And it generates Java-side garbage each call.

My version aims to make 1 JNI call regardless of number of controls, zero marshalling (it's transferred through shared memory for C# to process) and near-zero garbage. I've started by working with 1 byte for everything (including the analogue sticks meaning +/- 128 bit resolution limitation) so it might even get to zero JNI calls if the limitations are acceptable.

Will keep you apprised of progress. Current version is recording events into a byte[] on the Java side.

Cheers, Rupert.

P.s. I might need help with the C# side since I'm new there!

goodhustle · February 2013

@Arakade, the code in my pastebin is actually very old (not to mention it triggers a weird NPE inside of OuyaController before the first axis event, reported as a OUYA bug already), so let me clarify where I've gotten in case it helps you out.

My current solution is a philosophical fork from the official unity plugin and follows @Ayrik's method by only keeping a reference to java side's static input buffers around, and then appears to work over a shm-like JNI magic bridge and unmarshalls the static member data through the AndroidJavaObject.Get<T> generic. The docs say they are pretty high level, and I'm not sure exactly what they do under the hood other than "caching". :)

This is all on Github now at https://github.com/getluky/OuyaUnityBridge. You can find the Java (https://github.com/getluky/OuyaUnityBridge/blob/master/ExampleProject/Assets/Plugins/Android/src/OuyaUnityActivity.java) and C# (https://github.com/getluky/OuyaUnityBridge/blob/master/ExampleProject/Assets/Plugins/OuyaUnityBridge/OuyaInput.cs) sides that do exactly this, which is what @Ayrik awesomely instigated earlier in the thread. The other major difference is that I rely upon the OuyaController class entirely, to get the benefits of automatic third party controller mapping that the OUYA staff makes, whereas (AFAIK, correct me if I'm wrong at this point) the official unity plugin processes all input manually.

As for your other questions, it definitely does not generate Java-side GC anymore, as the AndroidJavaObject is used for the lifespan of the C# bridge class, which then just calls AndroidJavaObject.Get<T> for each different value every frame (see https://github.com/getluky/OuyaUnityBridge/blob/master/ExampleProject/Assets/Plugins/OuyaUnityBridge/OuyaInput.cs#L474). This does not appear to cause the large amounts of GC I was seeing when I was creating new copies every time a JNI request came through. Also, there are 6 axes and 14 buttons to track on each controller.

I am not sure whether switching to byte buffers would result in large marginal improvements. I suspect by avoiding JNI calls, we're just left with marshalling/unmarshalling which I don't see as a huge cost (as you say, maybe not?). I'm certainly satisfied with the low input latency and high framerates i'm seeing with my current implementation - both Beast Boxing Turbo (1p) and 2 Hando Commando (2p) on the OUYA testing store are using it.

There are also some gremlins to be aware of - I have heard a lot of anecdotal reports that batteries with low charge may be one of the culprits of laggy wireless communication, and exactly how the wireless controller drivers work would require a deep dive into Android. Personally I doubt the goog designed it with four simultaneous dual stick controllers in mind. There's also wireless lag, HDTV response time lag because of video frame buffer adjustment (like running 720p on 1080i), and maybe some HDMI decode lag (just guessing here).

What I don't know is whether any of this makes a difference in respect to input latency and not pure CPU or communication load. If Unity receives a SendMessage, it's executed synchronously. Whatever happens between java UnityPlayer.UnitySendMessage and the Unity Engine is a black box. The shm approach is probably fastest, but as I said, I don't know if simple UnityEngine.AndroidJavaObject creates a shm already, as they seem to have direct access into their java-side static members.

In any case, on your C# / Unity side, you're going to need to collect those inputs and set them up for polling. You will want to make sure that it gets called first in the Script Execution Order (under project settings) to ensure you don't have a one-frame lag on the Unity side as well.

VAMflax · February 2013

Okay, I've got back on this as I have an OUYA event to go to on the 25th.

On Mac, Unity 4.0, I have just got to reading the O button. And there is no GC output at all!! High five!!

I had a few hiccups along the way...

For a start, in OuyaPanel.cs, I added the ouya-sdk.jar to the start of the jars string:

string OUYAsdkJarPlusJars = string.Format("\"{0}:{1}:{2}:{3}:{4}:{5}\"", OUYAsdkJar, pathToolsJar, GetPathAndroidJar(), pathGsonJar, pathUnityJar, pathOuyaUnityPluginJar);

string command = string.Format("-source 1.6 -target 1.6 {0} -classpath {1} -bootclasspath {2} -d \"{3}\"", includeFiles, OUYAsdkJarPlusJars, OUYAsdkJarPlusJars, pathClasses);

RunProcess(pathJavaC, command );

It took me a while to see the OSX version!!

Before that was working I got "failed building DEX" every time.

Then I had general fun with java - I'm a c++ person really.

Then, the killer was in OuyaGameObject.cs...

//m_javaClass = new AndroidJavaClass("tv.ouya.demo.OuyaUnityApplication");
m_javaClass = new AndroidJavaClass("com.VAMflax001.CMTest02.OuyaUnityApplication");

I had left tv.ouya.demo where my bundle id should be!!

But yeah, the O button is now working!!

Thanks x 1,000,000 to all involved! :D

VAMflax · February 2013

This:

Step 4:

Insert the following lines int onGenericMotionEvent() between Line 142 and 143:

int playerNum = OuyaController.getPlayerNumByDeviceId(event.getDeviceId());

ControllerState data = playerStates[playerNum];

OuyaController c = OuyaController.getControllerByPlayer(playerNum);

data.AxisLSX = c.getAxisValue(OuyaController.AXIS_LS_X);

data.AxisLSY = c.getAxisValue(OuyaController.AXIS_LS_Y);

data.AxisRSX = c.getAxisValue(OuyaController.AXIS_RS_X);

data.AxisRSY = c.getAxisValue(OuyaController.AXIS_RS_Y);

data.AxisLT = c.getAxisValue(OuyaController.AXIS_L2);

data.AxisRT = c.getAxisValue(OuyaController.AXIS_R2);

Does not work for me :(

Game dies complaining it can't find OuyaController.getAxisValue. Odd that normally the event has getAxisValue...?

Anyone have any ideas?

VAMflax · February 2013

It's working fine by changing the c. to event. BTW.

Howdy, Stranger!

Categories

Controller lag and inaccuracy

Comments