Changelog

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[4.2.0] - 2019-12-01

Added

Now based on NativeArray<T>, minimum version now 2018.1

Native Touch was using managed array together with GC pinning for the native side to write to. This release improve the performance of everything overall by moving to NativeArray<T>. At the same time, this bumps the minimum version to 2018.1 because it is the first version to introduce this.

Theoretically this is a completely free performance boost over old versions.

See Jackson Dunstan's article for more details how operating on NativeArray<T> generates more efficient, more direct compiled code.

(EXPERIMENTAL) NativeTouchTracker : iOS performance improved

Previously I tried to match input in 2 frames by comparing coordinates, without realizing that connected inputs are in the same index as the previous frame. The code changed to utilize this fact now. Thank you for e-mailing me about this!

Unity Package Manager compatible Samples

Demo folder is now renamed to Samples~, which is now hidden from Unity importer until you need it. Various demo audio inside now properly not imported to your game, saving you import time on switching platforms. In this new convention, samples are imported on demand to your project by a button in the Package Manager instead of being there from the start. You can also just copy to your project out of ~ suffixed folder.

NativeTouchTracker and NativeTouchAndroidStudio project moved to Samples~

They are not exactly a samples, but an additional features for the user to opt in and use. It goes well with this unimported folder so only the essentials are imported automatically.

Other changes

Folder structure in Unity's UPM convention

• Managed folder renamed to Runtime. .asmdef moved into the Runtime folder as well.
• Extras folder merged into the new Samples~, you can get its content by sample importing or just copy it out.
• Documentation~ hidden folder added to the package. It contains a Markdown version of the official website https://exceed7.com/native-touch.
• HowToUse offline zipped website documentation removed in favor of Markdown documentation in Documentation~ folder.
• Documentation~ and Samples~ included in .zip form until Asset Store support publishing unimported folder. Please unzip it manually when you need them. (Either zipped or unzipped to ___~, the content won't be unnecessarily imported to your game.)

Discontinued support for 2017.1.5f1.

Now the lowest supported version is 2017.4.34f1. (The lowest possible LTS in Unity Hub.)

Trivia : 2017.1.5f1 didn't have Assembly Definition Files, supporting that version make it very difficult to utilize internal keyword together with [InternalsVisibleTo] as it doesn't work with Editor magic folder.

Others

• New website design : https://exceed7.com/native-touch. It is a big deal since now the website linked to Documentation~ hidden folder in this package rather than having to handcraft it separately. Even this CHANGELOG.md turns into a webpage, along with API documentation that is the same as code documentation in this package. Now you can read documentation in their Markdown form in Documentation~ or in the website. It will also reduce my maintenance burden, things in the web will stay up to date with the package 100%. Awesome!
• The web has better explanation why audio could get faster.
• Rewrite many method's code documentation. (Also reflected in the API page in the web.)

[4.1.0] - 2019-10-01

Changed

Dropped support for x86 architecture in the prebuilt AAR.

Due to recent Google Play policy change to accept only 64-bit build and Unity deprecating x86 build (and you cannot even submit a build with x86 lingering in there if you opt in to Google's new Android App Bundle), I have rebuild the AAR with only armeabi-v7a (32-bit) and arm64-v8a (64-bit).

I had received a report that some phone mistakenly pick x86 in the AAR previously and crash on start even though it should have picked arm64-v8a, so the best way now that Unity is also removing x86, is to just remove x86 from the AAR.

If you are using pre 2019.2 Unity, don't check x86 in the Android build configuration. Doing so will only drop support for about 2 devices in the world according to Google Play console. No one made x86 devices to the market anymore. If you really want x86 compatibility for a very specific circumstance, please use the included Android Studio project in Extra folder and compile a new AAR with x86.

Fixed

• [Android] The AAR is compiled with targetSdkVersion 27 instead of previously 28, to support developers that use Android SDK of lower version with Unity.
• [Android] The project that compiled the AAR is upgraded to Android Gradle plugin version 3.5.0 and Gradle Version 5.4.1.

[4.0.1] - 2019-06-06

Fixed

• A rounding error in NativeTouchRecognizer.mm causing "plus" iOS device which have 3x native coordinate scaling to return a coordinate error by -1 from an intended place. If you was trying to write a touch tracker that connect up points and wondering why they didn't connect properly on plus devices, this is the reason. I am incredibly sorry.

[4.0.0] - 2019-03-20

Added

New underlying interop mechanism : ring buffer

The talk back from native to C# is improved. Relying less on delegate and more "communication via memory area". It is a straight upgrade and you have nothing to lose from user's perspective regardless of modes of operation. (It's a new backend of sorts) The callback based API you use is still the same.

How it was working previously :

• C# send a delegate to static method to native.
• Native wait for touch callback from OS.
• Native call that delegate with the touch data. Its argument is a single touch struct, and to be allocated from platform-specific touch structure to allow cross platform. (MotionEvent for Android, and UITouch for iOS) Plus it is not easy to send the whole object from Java or Objective C that C# can understand so I must make a struct.
• So you receive tons of callback when you mash the screen.

This is mostly not a problem on iOS IL2CPP and Android Mono, the callback is as fast as "just a method call". However on Android IL2CPP, the Java -> C# method call is very slow, as slow as 16ms (1 frame of 60 FPS). This is not acceptable. Be able to handle touch in a "callback style" in Unity is the core function of Native Touch. But it is no use if the callback is so slow that it arrives later than Unity's touch.

The new way it is working right now :

• C# allocates a fixed length array of touches as a ring buffer.
• This is pinned and its address is sent to native side to use.
• C# send a delegate to static method to native also, but this time its argument is no longer touch struct, but just 2 int specifying where in the ring buffer to check out the data and how long from that point. (Wraps around to beginning, as per ring buffer definition.)
• Native wait for touch callback from OS.
• The native side writes those touches to the ring buffer boundary. It knows the fixed length allocated at managed side. Along the way it remembers where it start writing and how much it had written.
• When finished one set of touches, native side use the delegate to tell C# to "check out" the ring buffer at where and how long.
• C# check the ring buffer it owned and found that there are new touch data written from the native side.
• C# iterates through them and present you each touch as the same as before. You still get to see NativeTouchData given to you one by one, but the iteration has been moved from native side to managed C#. Each touch is no longer gated by native to managed delegate callback.

All C# side still remains in safe code. It uses IntPtr which is usable in safe context to give native side the address.

This approach aims to solve several things :

• Previously IL2CPP on Android (but not iOS, and not on Mono Android) had abysmal delegate callback performance. This patch greatly reduces amount of callbacks needed. (As high as 5x lower when the screen is really busy.) On Android by compiling with Mono backend the delegate performance improved instantly. This might be because native to managed delegate in Android is assisted by JIT mechanism. When on AOT only backend like IL2CPP something may have become more difficult to do. I suspect that Java View call to invoke that remembered C# delegate is not passed through IL2CPP, and so it is not "just a function pointer call" anymore. Something heavy is added. Still, even with a mere 1 callback on Android IL2CPP it is unacceptable that it takes 1 frame or more. But ring buffer allows a new API that could be used instead of callback style. More in its own section.
• Previously each callback is weighted by a full touch struct on the parameter. Now it is just 2 integers. Lighter is always better.
• Previously there is a touch struct allocation at native side. Now it wrote directly to the ring buffer which C# owns using a struct-mapped pointer which is maintained to match the C# side. There is no allocation at all not counting the native touch struct coming from the OS's callback. (MotionEvent and UITouch)

New StartOption added : noCallback

Be able to handle touch in a "callback style" in Unity is the core function of Native Touch. I am adding one more approach more similar to Input.touches you used to do.

To recap, the previous "callback based" API sends C# static method to be "reverse p-invoked" by Java's View or iOS's UIGestureRecognizer when it receive a touch. With ring buffer shared memory upgrade in this version, the reverse p-invoke is just telling C# to look at specific spot in those memory areas. The native had written something new (touch data) to it. C# see new data and understand that those are new touches. Greatly improve interop speed by talking via memory content plus lesser callback, rather than purely via callbacks with a lot of parameters.

But still, platform like Android IL2CPP is very bad at calling even one Java -> C# delegate. And this call is blocking the Java View's touch handler. Though, if I tell C# to tell C to call delegate back to C#, this is fast, in fact 2x faster than Mono. Likely IL2CPP optimized these cross platform complexity into "just a function call".

Native Touch is special because the originator of action is Java's view. Java is JNI-ing to C to invoke the delegate because Java could not do direct memory write (required for the ring buffer optimization) and Java is too coward to do direct method pointer call. This C invoking C# works, but probably IL2CPP knows nothing about it. So when it jumbles into IL2CPP'ed Android, something heavy must be in place to make it work. (Probably something heavy like C#'s reflection?)

Native Touch 4.0.0 allows opting out completely from receiving callbacks with noCallback on the StartOption used when you call NativeTouch.Start(). Previously Native Touch will not allow you to start without registering any callback. Now even with callback registered, it will let you start and completely ignores the callbacks.

What's the point? How to receive touches then? This bring us to the next feature...

New touch handling style added : Ring buffer iteration API NativeTouch.touches.

Instead of relying on callbacks, you wait for your touches in Update() as usual. BUT instead of checking Input.touches, you now have an access to NativeTouch.touches.

This NativeTouch.touches is essentially that ring buffer but with a wrapper to help you read from the correct place on it. It contains .TryGetAndMoveNext(out NativeTouchData), an easy interface that you could while until you have catch up with the latest data on the ring buffer.

One advantage over the callback based is that you have no worry about thread. Remember that the callback way is initiated by native side. The thread in that callback scope is depending on what thread the native side is using. On Android, it is not in the same thread as Unity's main thread and is completely not in sync. This make it a bit difficult to migrate and use those NativeTouchData for things waiting in the main thread. Potentially involving some mutex locks to make it safe.

When you use NativeTouch.touches API, you are reading from the same ring buffer that native side is writing new touches to, which might be in parallel. But not to worry as I have made sufficient mutex locks that it properly waits or avoiding each other. (This lock is not enabled on iOS, as the write is not in parallel unlike in Android)

When you was using the callback way, you are likely doing some kind of caching of those touches in order to make the main thread's Update() to be able to use them. Now you can think that this caching and making it main thread compatible is already done for you. have Except you didn't pay the cost of native callback with noCallback start option! On platform like Android IL2CPP where callback is expensive, using noCallback with waiting to do ring buffer iteration in the next Update() is often faster than the usual callback way.

Also, did we completely defeat the point of Native Touch? No, for several reasons :

• Still could be faster : The data may still appear earlier or equal to Input.touches. I had a report that some phone have Input.touches data appears as late as 3 frames from the touch. You may see data appearing in NativeTouch.touches earlier. It cannot be later, the code is instructed to write to ring buffer area even before handing the touch to Unity.
• Still more data : The touches waiting in the ring buffer of NativeTouch.touches have touch timestamps. Even if they arrive at the same time as Input.touches, you have some missing information that Unity discarded.
• Still customizable and source available : The touches is "native". We can modify the plugin to include more from the native side that you fear Unity is discarding or processed out to fit the Touch data structure. Touch timestamp is one such thing that we added back. Unity's touch processing is in closed source. We could do nothing about it. The "native" in the name is guaranteeing you can do whatever native Android or iOS can do about those events equally. But Native Touch make them appears in Unity easier without any processing.

What we lose by using ring buffer iteration-based API :

• Previously it is possible to make it in time to use data from the callback for moving object, that is still earlier than rendering submission in the game loop. This make it visibly faster that Native Touch could speed up things. With this the chance of being able to do so still exists but lower, since we wait until the next main thread Update() to use the touches.
• Previously it is possible to do a thread safe thing in that callback. Everything there is 100% faster than waiting for Unity main thread's Update(). For example, if you want to make an app that plays sound instantly you touch the screen without any other logic (don't care where you touch the screen), nothing could beat placing native audio playing code in the Native Touch static callback. (No, AudioSource is not thread safe. The audio playing must be main thread independent for platform like Android which the callback comes in an another thread.)

Changed

ClearCallbacks() now cannot be used while in Start() state.

• It is now not possible to use NativeTouch.ClearCallbacks() while you are in Native Touch enabled state. Stopping Native Touch on platform like iOS can cause a remaining queued touches to be dispatched for the last time. If you clear the callbacks before stopping it could lead to null reference error. (Because Native Touch is intentionally not doing null check, for speed.)

Fixed

• Better error throwing in the code.
• Demo scene is fixed to show how to do ring buffer iteration based API together with noCallback start option.
• [iOS] No longer replays tons of touches on calling NativeTouch.Stop(). The fix for this is achieved by never removing the touch recognizer but just temporarily turn it off. The strange touch replay through UIView was caused when removing the recognizer.

[3.0.0] - 2019-02-28

Added

NativeTouch.RealScreenResolution

Native Touch's coordinate is unscaled, if your game uses Resolution Scaling (Or dynamic resolution scaling) then Screen.width/height/resolution/etc will be scaled down too. If you use that to calculate something with Native Touch's returned coordinate then it is wrong. (Unity's Input.___ API will be scaled down too)

Since native side has no idea how small Unity is scaling things, the approach is to use this method to return resolution of Screen.__ API as if it wasn't scaled. It got the same bounds as what coordinate that is returning from Native Touch. Then you could calculate the touch coordinate into the scaled coordinate.

Changed

Native Touch now returns "Native Scale" instead of just (virtual) scale.

This is a breaking change and a fix at the same time. Previously there is a mistake that returns just scale in NativeTouchRecognizer.mm line 92. Now it is instead nativeScale.

The reason is Unity's coordinate is going by the native scale. See this table, the -Plus device have a smaller multiplier from 3 to 2.608.

That means the previous version of Native Touch returns coordinate higher than Unity's bound when it goes to 3x where actually it is just 2.608x, for example. This update brings them down to the same bound.

Fixed

• Demo is updated to work correctly in the case of using Resolution Scaling.
• NativeTouchTracker demo is updated to work correctly in the case of using Resolution Scaling.
• Fixed extern methods intended for iOS side mistakenly getting into Android build and cause linker error.
• Better XML code documentation.

[2.1.0] - 2018-10-10

Added

Bonus content added : NativeTouchTracker

It is zipped in the Extra folder. Feed it a lot of NativeTouchData in sequence and you would be able to poll data from it just like Unity's Input.touches. There are tons of preprocessor directives inside that make the whole thing works magically on both Android and iOS.

This struct is an example of how my take of interpreting native touches would go. There are several discarded data still, based on the need of my own game, you can take a look at it as a guideline for creating your own wrapper.

Requirements (And why it stays in a zip file)

• A reference to Unity.Collections and Unity.Mathematics package, making it goes well with C# Jobs, ECS, and Burst compilation.
• An understanding of how native collection works, this is a struct full of them. You also have to Dispose it.
• Incremental Compiler package for C#7.0 syntax.

New demo scene

It now has a colored square following your finger's movement.

Fixed

• The previous version's Android touch sends wrong touch phase in the case of 3 or more touches. It is corrected now.
• Previously iOS sends duplicated touch events and it is possible to get Stationary phase because I wrongly used [event allTouches] instead of touches in the native side.
• Previously Android sends Cancelled phase only for the "main" touch in the pack and other touches fixed as "ambiguous Moved" (Might be Moved or implied Stationary). Now only for Cancelled phase there is a special rule so that all touches in the pack are considered Cancelled instead of Moved. Otherwise there's no way to tell which touch came together with the Cancelled touch.

[2.0.0] - 2018-09-08

Added

Android support

If you put #if to get Android out of the way before, now it is not needed.

Changed

Static callback signature changed to just one struct

It is much easier than before that you have to declare 5 ints or something and that was very error prone. This struct has a nice C# property so that each platform access only its valid fields.

Also because of this Unity's Touch support has been dropped. The mode is just full mode or not-full mode. (minimal mode)

Minimal or full mode specified via StartOption instead

There is a new overload for Start() which accepts some options, Start(startOption).

The touch continues to Unity

Previous version completely disable Unity touch. This version to get the same behaviour you have to use the new StartOption when calling Start(startOption).

Fixed

iOS now correctly returns stationary phase touch on multitouch scenario

The previous version has a bug. When you have multiple fingers down and move one finger you only get one move phase in the callback. Now in addition, you get other fingers as stationary phase as well. Outside of multitouch scenario you can never receive a stationary phase.

In Android there is no stationary phase at all. Please read Callback Details in the website.

[1.0.0] - 2018-04-13

The first release. Supports only iOS.

Unreleased

iPad Pro and Apple Pencil 2 support

I am getting an iPad Pro this year. After that I could confirm if we can fully utilize the ProMotion 120Hz input fully or not, and if there are any missing information from the new Apple Pencil that Unity discards but we could recover or not.

However, I am suspecting that it is already supported currently, but without able to confirm by myself I can't guarantee the support status of these.

ECS Support

The touch could optionally produce an Entity to any world you want at any BarrierSystem you like in the case of Android, or immediately with EntityManager of your choice the case of iOS. The touch struct will be implementing IComponentData interface and attached to that Entity.

Then any system wish to use the touch can inject them and attach ISystemStateComponentData which signify processed touch to it. A utility method to clean up all processed touch will be provided as well to save memory space.

I will wait for Unity.Entities package to be out of preview and be more widespread first or else the asmdef would have to refer to preview assembly.

At first to make the onboarding optional, the support might come in the form of preprocessor directive like having NATIVE_TOUCH_ECS and it changes the using in the code to use Unity ECS library and adds : IComponentData to the touch struct.

[Android] Historical values

Many values from Android has variable amount of "historical values". Currently Native Touch discards them. I have to find a nice way to send variable values from native to managed and make it usable with the current workflow.