Vuforia Flutter SDK

[imaNNeo]

We would be happy if we could use Vuforia in our Flutter applications.

As far as I know, we already have the Android SDK and iOS SDK. Having a Flutter SDK should not be that much challenging. We can just have a wrapper around those two SDKs and create the Flutter SDK (for mobile).

I know there is a solution (such as using Unity and importing it in the Flutter application), but it is super complicated and heavy to just have AR in the app. Also in our use case, there are some battery usage issues when we use Vuforia through the Unity SDK in our Flutter app.
Also, when we use Unity, we have to take care of pricing, so it adds another cost and we count it as a cost that we pay for Vuforia (our AR solution).
But if we have a Flutter SDK (wrapper), we can just pay for Vuforia and it decreases our costs (both in financial and technical)

Also, I think the community can help to create the wrapper. (I personally don't have time to help)

Let me know what you think.

Cheers!
Iman

[ptc-thesbyro]

Hi @imaNNeo,

Thanks for reaching out.

While we do not have any current plans to develop a Flutter SDK, I encourage others reading this thread to contribute with experiences and resources for using Flutter and Vuforia Engine.

[Ahmadre]

Hi @imaNNeo,

Thanks for reaching out.

While we do not have any current plans to develop a Flutter SDK, I encourage others reading this thread to contribute with experiences and resources for using Flutter and Vuforia Engine.

Hi, contributor of Flutter-Unity Plugin here and Creator of the Web-Solution for that Plugin.

I already discussed with my collegue @imaNNeo about this.

There're several issues with Vuforia and I already fixed them in the Flutter-Unity Plugin.

But Unity has crucial Battery Issues and iOS issues with controlling the camera.

I will start developing a Vuforia Community Plugin for Flutter and update you guys about this.

It will be published on pub.dev.

It will take some time, but I want to integrate all features! Image and Object detection has priority, but we need all features in long-term.

Best regards
4hm4dr3

[ptc-emaggio]

Hi @imaNNeo and @Ahmadre thanks for raising this. I would be interested to know more about the Unity battery issues and also camera controls on iOS. Possibly there is something we can do to help you.

[Ahmadre]

Hi @imaNNeo and @Ahmadre thanks for raising this. I would be interested to know more about the Unity battery issues and also camera controls on iOS. Possibly there is something we can do to help you.

At first: thank you for your interests on this.

I try to answer you as detailed as I can: I am working on 3D software solutions since Flutter exists (late 2018).

Unity-as-a-Library existed before and was already successfully used in other Frontend-SDKs like Angular or React (See for example: https://gilchr.ist/blog/two-way-communication-between-angular-components-and-unity-webgl-build ).

To mention out: we don't have any issues with Vuforia SDK itself! We're very happy with what Vuforia offers and all solutions are working perfectly (Image, Object detection and Cloud Image Recognition tested on my side here).

If you work with Unity-as-a-Library it's like if you stack on your Flutter Framrwork another Framework: Unity. So You got a Stacked Framework just to detect images and render 3D objects. The problem is: Unity is a Game Engine which comes on the fly with a lot engines we don't even need + Unity itself is a Thread running in the Flutter process which controls the Camera (bi-directional communication between the 2 Threads). Now there's a way you can pause or stop the Unity instance. Then the camera is turning off, if you leave the Unity Screen inside your Flutter app....if you're on Android 👀, but not on iOS! You may think: hey this is maybe just a misconfiguration or any native function is missing for Swift...nope! This is a kind of firm binding from iOS platform to the thread accessing the camera only in Unity! So: this is a known bug from Unity as a Library itself and has nothing to do with Vuforia or Flutter. We tested this and we can successfully disable/enable the camera with any kind of Plugins in Flutter. You can "disable" the Main Camera (or the Vuforia Camera) in the Unity Scene, BUT iOS still detects that there's some kind of camera still "recording"! This is a huge issue because the Apple App-Store (obviously) declines such apps in the App Store Review (justifiably)! This issue was reported multiple times (regardless if you use Flutter or Angular or React-Native). And geuess what: there's a Unity Asset to buy for $5 if you want this fix for your Unity project (but not for Unity-as-aLibrary or Unity embedded as a Framework somewhere else!).

Camera issue for ios aside, the issue with battery and too much cpu/gpu usage:

Unity is a Game Engine. That answers it xD. For sure you can setup your graphics quality levels and provide presets for them (Low, Medium, High Quality, or in other words: Performance Mode or Quality Mode) and change them in your Flutter App as an option for your users (and also detect the device and already set Low quality mode for example to save battery life). BUT still Unity is consuming too much battery and is not efficient to the cpu and gpu of your device. We also tested this on different devices and especially on Android the ARCore is a mess! We tested this also with multiple application (Car configurator and Window configurator accessing only native AR Tools from Unity).

All-in-all: it's a bad decision and bad practice to use a Stacked Framework where you have 2 engines conflicting with each other and I develop with Unity since a decade and Unity's mobile support is terrible! Unity introduces bugs over bugs with every stable release! Sometimes you have to compile your iOS app with a lower version and sometimes you have to compile your Android app with a newer version! You never know: sometimes you wake up and your app is not working with your unity version, even if you have your OS and XCode not updated yet. There's also the huge configuration and russian roulette when you setup Flutter Unity. I've written myself a lot of docs just to provide other Flutter Developers a guide how to setup this Stacked Framework 👀 .

I developed these Apps with Flutter-Unity:

1. fenzta HOME (window configurator without Vuforia): https://fenzta.com/en/
   -> iOS: https://apps.apple.com/de/app/fenzta-home/id1595613045
   -> Android: https://play.google.com/store/apps/details?id=de.fenzta.app
   -> Web: https://app.fenzta.com/

2. Andreas Belzek Art (art app with Vuforia):
   -> iOS: https://apps.apple.com/de/app/andreas-belzek-art/id6503223911
   -> Android: https://play.google.com/store/apps/details?id=de.applogik.hommage

What I wanted to point out here: The Vuforia implementation of AR is MUCH better then Unity's native built-in version! Vuforia implemeted AR even better then Unity!

It really doesn't make any sense to use Unity, if you already have Flutter! Flutter is by far the best Framework nowadays to create corss-platform solutions and when you already use Unity as your main Cross-Platform Framework why using Flutter? Correct! It doesn't makes sense, because Unity itself can also build for all platforms but is a Game Engine. And we need simple 3D solutions (like ThreeJS for web).

So because we want to keep it simple and have full control to camera and not relying on Unity to fix it's own platform bugs, it's better to have our own Flutter Vuforia SDK.

For your info: I had vacation since 2 weeks, but I will start soon to develop a Federated Plugin for this :) (see: https://docs.flutter.dev/packages-and-plugins/developing-packages#federated-plugins)

[ptc-emaggio]

Thanks @Ahmadre for the detailed answers. I have shared your comments with our product team.