Skip to content

Add GenAI inference task #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
May 12, 2024
Merged
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
10990cf
Adds mediapipe_core package (#11)
craiglabenz Nov 13, 2023
7850d32
Add utility to collect headers from google/mediapipe (#10)
craiglabenz Nov 13, 2023
6a7dacb
[FFI] MediaPipe SDKs finder automation (#16)
craiglabenz Jan 12, 2024
8980132
Adds mediapipe_text package (#12)
craiglabenz Mar 11, 2024
4c159a7
Native Assets CI fix (#20)
craiglabenz Mar 12, 2024
4304801
Text Embedding task (#21)
craiglabenz Apr 3, 2024
3fc2d11
updated and re-ran generators
craiglabenz Mar 11, 2024
e01f26b
fixed embedding header file and bindings
craiglabenz Mar 11, 2024
ce42365
adds text embedding classes to text pkg
craiglabenz Mar 11, 2024
d36976b
moved worker dispose method to base class
craiglabenz Mar 25, 2024
f0344b9
class hierarchy improvements
craiglabenz Apr 1, 2024
992332b
cleaned up dispose methods
craiglabenz Apr 1, 2024
4ac6d20
initial commit of language detection task
craiglabenz Apr 1, 2024
b9b6c89
finishes language detection impl
craiglabenz Apr 2, 2024
bed398a
adds language detection demo
craiglabenz Apr 2, 2024
73bada3
backfilling improvements to classification and embedding
craiglabenz Apr 2, 2024
a392f83
adds language detection tests
craiglabenz Apr 2, 2024
4b742a3
add new model download to CI script
craiglabenz Apr 3, 2024
65cc969
fixes stale classification widget test, adds language detection widge…
craiglabenz Apr 3, 2024
daa2eb3
initial inference commit of flutter create -t package
craiglabenz Apr 15, 2024
8a63461
rename inference folder
craiglabenz Apr 15, 2024
a1992b6
adds build tooling for inference headers and ffigen
craiglabenz Apr 15, 2024
e5e76c6
rename "inference" to "genai"
craiglabenz Apr 16, 2024
fdd63eb
adds initial inference impl
craiglabenz Apr 17, 2024
904ee92
flutter creat example for genai
craiglabenz Apr 17, 2024
3dc77e5
sdks crawling update
craiglabenz Apr 19, 2024
9ab9a95
adds android project to genai example
craiglabenz Apr 25, 2024
74275cc
adds ios project to genai example
craiglabenz Apr 25, 2024
6542360
updates to macos project in genai example
craiglabenz Apr 25, 2024
143d6dc
inference impl improvements
craiglabenz Apr 25, 2024
30e813a
inference example - nearly working
craiglabenz Apr 25, 2024
ce5f887
Merge branch 'main' into genai-inference
craiglabenz Apr 25, 2024
40f93a7
various cleanup
craiglabenz Apr 25, 2024
f1b6f39
inference demo updates
craiglabenz Apr 29, 2024
6e39594
model path debugging
craiglabenz Apr 29, 2024
881114e
inference demo improvements
craiglabenz May 1, 2024
a73e36a
improvements to inference demo
craiglabenz May 1, 2024
e3d7c67
genai inference example improvements
craiglabenz May 6, 2024
b1005dc
updates to inference and example
craiglabenz May 10, 2024
d6cad7c
fixed issue with chat screens sometimes not refreshing while the LLM …
craiglabenz May 11, 2024
b05d708
Merge branch 'main' into genai-inference
craiglabenz May 12, 2024
0c648a0
updated SDK manifests
craiglabenz May 12, 2024
bf5bcd8
removed unused ffi utils copy param
craiglabenz May 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/main.yaml
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ permissions: read-all

on:
pull_request:
branches: [ main ]
branches: [main]
push:
branches: [main, ffi-wrapper, ffi-wrapper-text-pkg]
schedule:
15 changes: 14 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -57,4 +57,17 @@ test_text:
cd packages/mediapipe-task-text/example && flutter test

example_text:
cd packages/mediapipe-task-text/example && flutter run -d macos
cd packages/mediapipe-task-text/example && flutter run -d macos

# GenAI ---
generate_genai:
cd packages/mediapipe-task-genai && dart --enable-experiment=native-assets run ffigen --config=ffigen.yaml

# Example genai invocation.
# Note that `GEMMA_4B_CPU_URI` can either be a local path or web URL. Similar values exist for
# 8B and GPU variants.
#
# For desktop development, standard environment variables like this work great.
# $ GEMMA_4B_CPU_URI=/path/to/gemma-2b-it-cpu-int4.bin flutter run -d [macos, windows, linux]
# For emulator or attached device testing, use `--dart-define` for the same values.
# $ flutter run -d [<device_id>] --dart-define=GEMMA_4B_CPU_URI=https://url/to.com/gemma-2b-it-cpu-int4.bin
9 changes: 6 additions & 3 deletions packages/mediapipe-core/lib/src/interface/task_options.dart
Original file line number Diff line number Diff line change
@@ -6,9 +6,12 @@ import 'dart:typed_data';

import 'package:equatable/equatable.dart';

/// {@template TaskOptions}
/// {@template BaseOptions}
/// Root class for options classes for MediaPipe tasks.
///
/// {@endtemplate}
abstract class Options extends Equatable {}

/// {@template TaskOptions}
/// Implementing classes will contain two [BaseInnerTaskOptions] subclasses,
/// including a descendent of the universal options struct, [BaseBaseOptions].
/// The second field will be task-specific.
@@ -17,7 +20,7 @@ import 'package:equatable/equatable.dart';
/// This implementation is not immutable to track whether `dispose` has been
/// called. All values used by pkg:equatable are in fact immutable.
// ignore: must_be_immutable
abstract class BaseTaskOptions extends Equatable {
abstract class BaseTaskOptions extends Options {
/// {@macro TaskOptions}
BaseTaskOptions();

4 changes: 2 additions & 2 deletions packages/mediapipe-core/lib/src/io/ffi_utils.dart
Original file line number Diff line number Diff line change
@@ -90,11 +90,11 @@ extension DartAwarePointerChars on Pointer<Pointer<Char>> {
///
/// See also:
/// * [toDartString], for a non-list equivalent.
List<String?> toDartStrings(int length) {
List<String> toDartStrings(int length) {
if (isNullPointer) {
throw Exception('Unexpectedly called `toDartStrings` on nullptr');
}
final dartStrings = <String?>[];
final dartStrings = <String>[];
int counter = 0;
while (counter < length) {
dartStrings.add(this[counter].toDartString());
2 changes: 1 addition & 1 deletion packages/mediapipe-core/lib/src/io/task_options.dart
Original file line number Diff line number Diff line change
@@ -15,7 +15,7 @@ import 'third_party/mediapipe/generated/mediapipe_common_bindings.dart'
/// should manage their [InnerTaskOptions] fields. The two suggested methods are
/// [copyToNative] and [dispose].
/// {@endtemplate}
mixin TaskOptions<T extends Struct> on BaseTaskOptions {
mixin TaskOptions<T extends Struct> on Options {
/// {@template TaskOptions.copyToNative}
/// Copies these task options into native memory. Any fields of type
/// [InnerTaskOptions] should have their `assignToStruct` method called.
8 changes: 6 additions & 2 deletions packages/mediapipe-core/test/io/task_options_test.dart
Original file line number Diff line number Diff line change
@@ -86,13 +86,17 @@ void main() {
expect(ptr.ref.score_threshold, lessThan(0.90001));
expect(ptr.ref.category_allowlist_count, 3);
expect(
ptr.ref.category_allowlist.toDartStrings(3),
ptr.ref.category_allowlist.toDartStrings(
3,
),
['good', 'great', 'best'],
);

expect(ptr.ref.category_denylist_count, 4);
expect(
ptr.ref.category_denylist.toDartStrings(4),
ptr.ref.category_denylist.toDartStrings(
4,
),
['bad', 'terrible', 'worst', 'honestly come on'],
);
});
4 changes: 3 additions & 1 deletion packages/mediapipe-task-audio/sdk_downloads.dart
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
// Generated file. Do not manually edit.
final Map<String, Map<String, String>> sdkDownloadUrls = {};
// Used by the flutter toolchain (via build.dart) during compilation of any
// Flutter app using this package.
final Map<String, Map<String, Map<String, String>>> sdkDownloadUrls = {};
29 changes: 29 additions & 0 deletions packages/mediapipe-task-genai/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Miscellaneous
*.class
*.log
*.pyc
*.swp
.DS_Store
.atom/
.buildlog/
.history
.svn/
migrate_working_dir/

# IntelliJ related
*.iml
*.ipr
*.iws
.idea/

# The .vscode folder contains launch configuration and tasks you configure in
# VS Code which you may wish to be included in version control, so this line
# is commented out by default.
#.vscode/

# Flutter/Dart/Pub related
# Libraries should not include pubspec.lock, per https://dart.dev/guides/libraries/private-files#pubspeclock.
/pubspec.lock
**/doc/api/
.dart_tool/
build/
10 changes: 10 additions & 0 deletions packages/mediapipe-task-genai/.metadata
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# This file tracks properties of this Flutter project.
# Used by Flutter tool to assess capabilities and perform upgrades etc.
#
# This file should be version controlled and should not be manually edited.

version:
revision: "bd909542a33ab1d5249363a2434ae50ee468094f"
channel: "master"

project_type: package
3 changes: 3 additions & 0 deletions packages/mediapipe-task-genai/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
## 0.0.1

* TODO: Describe initial release.
1 change: 1 addition & 0 deletions packages/mediapipe-task-genai/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TODO: Add your license here.
39 changes: 39 additions & 0 deletions packages/mediapipe-task-genai/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<!--
This README describes the package. If you publish this package to pub.dev,
this README's contents appear on the landing page for your package.
For information about how to write a good package README, see the guide for
[writing package pages](https://dart.dev/guides/libraries/writing-package-pages).
For general information about developing packages, see the Dart guide for
[creating packages](https://dart.dev/guides/libraries/create-library-packages)
and the Flutter guide for
[developing packages and plugins](https://flutter.dev/developing-packages).
-->

TODO: Put a short description of the package here that helps potential users
know whether this package might be useful for them.

## Features

TODO: List what your package can do. Maybe include images, gifs, or videos.

## Getting started

TODO: List prerequisites and provide or point to information on how to
start using the package.

## Usage

TODO: Include short and useful examples for package users. Add longer examples
to `/example` folder.

```dart
const like = 'sample';
```

## Additional information

TODO: Tell users more about the package: where to find more information, how to
contribute to the package, how to file issues, what response they can expect
from the package authors, and more.
9 changes: 9 additions & 0 deletions packages/mediapipe-task-genai/analysis_options.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
include: ../analysis_options.yaml

linter:
rules:
- public_member_api_docs # see https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#documentation-dartdocs-javadocs-etc

analyzer:
exclude:
- "**/mediapipe_genai_bindings.dart"
122 changes: 122 additions & 0 deletions packages/mediapipe-task-genai/build.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
import 'dart:io';
import 'package:native_assets_cli/native_assets_cli.dart';
import 'package:http/http.dart' as http;
import 'package:path/path.dart' as path;

import 'sdk_downloads.dart';

late File logFile;

final logs = <(DateTime, String)>[];
void log(String msg) {
logs.add((DateTime.now(), msg));
if (!logFile.parent.existsSync()) {
logFile.parent.createSync();
}

if (logFile.existsSync()) {
logFile.deleteSync();
}
logFile.createSync();
logFile.writeAsStringSync(logs
.map<String>((rec) => '[${rec.$1.toIso8601String()}] ${rec.$2}')
.toList()
.join('\n\n'));
}

Future<void> main(List<String> args) async {
final buildConfig = await BuildConfig.fromArgs(args);
logFile = File(
path.joinAll([
Directory.current.path, // root dir of app using `mediapipe-task-xyz`
'build/${buildConfig.dryRun ? "dryrun" : "live-run"}-build-log.txt',
]),
);

log(args.join(' '));
final String targetOs = buildConfig.targetOs.toString();

log('dir.current: ${Directory.current.absolute.path}');

// Throw if target runtime is unsupported.
if (!sdkDownloadUrls.containsKey(targetOs)) {
throw Exception('Unsupported target OS: $targetOs. '
'Supported values are: ${sdkDownloadUrls.keys.toSet()}');
}

final buildOutput = BuildOutput();
buildOutput.dependencies.dependencies
.add(buildConfig.packageRoot.resolve('build.dart'));
buildOutput.dependencies.dependencies
.add(buildConfig.packageRoot.resolve('sdk_downloads.dart'));

final modelName = 'libllm_inference_engine';
final Iterable<String> archKeys;
if (buildConfig.dryRun) {
archKeys = sdkDownloadUrls[targetOs]![modelName]!.keys;
} else {
archKeys = [buildConfig.targetArchitecture.toString()];
}
for (String arch in archKeys) {
arch = getArchAlias(arch);
log("arch: $arch");
log("sdkDownloadUrls[targetOs]: ${sdkDownloadUrls[targetOs]}");
log("sdkDownloadUrls[targetOs]['$modelName']: ${sdkDownloadUrls[targetOs]![modelName]}");
log("sdkDownloadUrls[targetOs]['$modelName'][$arch]: ${sdkDownloadUrls[targetOs]![modelName]![arch]}");

if (!sdkDownloadUrls[targetOs]!['libllm_inference_engine']!
.containsKey(arch)) {
continue;
}
final assetUrl =
sdkDownloadUrls[targetOs]!['libllm_inference_engine']![arch]!;
final downloadFileLocation = buildConfig.outDir.resolve(
'${arch}_${assetUrl.split('/').last}',
);
log('downloadFileLocation: $downloadFileLocation');
buildOutput.assets.add(
Asset(
id: 'package:mediapipe_genai/src/io/third_party/mediapipe/generated/mediapipe_genai_bindings.dart',
linkMode: LinkMode.dynamic,
target: Target.fromArchitectureAndOs(
Architecture.fromString(arch), buildConfig.targetOs),
path: AssetAbsolutePath(downloadFileLocation),
),
);
if (!buildConfig.dryRun) {
downloadAsset(assetUrl, downloadFileLocation);
}
}

await buildOutput.writeToFile(outDir: buildConfig.outDir);
}

Future<void> downloadAsset(String assetUrl, Uri destinationFile) async {
final downloadUri = Uri.parse(assetUrl);
final downloadedFile = File(destinationFile.toFilePath());
log('Saving file to ${downloadedFile.absolute.path}');

final downloadResponse = await http.get(downloadUri);
log('Download response: ${downloadResponse.statusCode}');

if (downloadResponse.statusCode == 200) {
if (downloadedFile.existsSync()) {
downloadedFile.deleteSync();
}
downloadedFile.createSync();
log('Saved file to ${downloadedFile.absolute.path}\n');
downloadedFile.writeAsBytes(downloadResponse.bodyBytes);
} else {
log('${downloadResponse.statusCode} :: ${downloadResponse.body}');
throw Exception(
'${downloadResponse.statusCode} :: ${downloadResponse.body}');
}
}

/// Translates native-assets architecture names into MediaPipe architecture names
String getArchAlias(String arch) {
return <String, String>{
'arm': 'arm64',
}[arch] ??
arch;
}
43 changes: 43 additions & 0 deletions packages/mediapipe-task-genai/example/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Miscellaneous
*.class
*.log
*.pyc
*.swp
.DS_Store
.atom/
.buildlog/
.history
.svn/
migrate_working_dir/

# IntelliJ related
*.iml
*.ipr
*.iws
.idea/

# The .vscode folder contains launch configuration and tasks you configure in
# VS Code which you may wish to be included in version control, so this line
# is commented out by default.
#.vscode/

# Flutter/Dart/Pub related
**/doc/api/
**/ios/Flutter/.last_build_id
.dart_tool/
.flutter-plugins
.flutter-plugins-dependencies
.pub-cache/
.pub/
/build/

# Symbolication related
app.*.symbols

# Obfuscation related
app.*.map.json

# Android Studio will place build artifacts here
/android/app/debug
/android/app/profile
/android/app/release
30 changes: 30 additions & 0 deletions packages/mediapipe-task-genai/example/.metadata
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# This file tracks properties of this Flutter project.
# Used by Flutter tool to assess capabilities and perform upgrades etc.
#
# This file should be version controlled and should not be manually edited.

version:
revision: "86135b7774e32fa7b0ad0d116511471f36048579"
channel: "master"

project_type: app

# Tracks metadata for the flutter migrate command
migration:
platforms:
- platform: root
create_revision: 86135b7774e32fa7b0ad0d116511471f36048579
base_revision: 86135b7774e32fa7b0ad0d116511471f36048579
- platform: ios
create_revision: 86135b7774e32fa7b0ad0d116511471f36048579
base_revision: 86135b7774e32fa7b0ad0d116511471f36048579

# User provided section

# List of Local paths (relative to this file) that should be
# ignored by the migrate tool.
#
# Files that are not part of the templates will be ignored by default.
unmanaged_files:
- 'lib/main.dart'
- 'ios/Runner.xcodeproj/project.pbxproj'
16 changes: 16 additions & 0 deletions packages/mediapipe-task-genai/example/.vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"configurations": [
{
"name": "Flutter",
"request": "launch",
"type": "dart",
"env": {
"GEMMA_4B_CPU_URI": "https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-cpu-int4.bin",
"GEMMA_4B_GPU_URI": "https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-gpu-int4.bin",
"GEMMA_8B_CPU_URI": "https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-cpu-int8.bin",
"GEMMA_8B_GPU_URI": "https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-gpu-int8.bin",
},
"toolArgs": []
}
]
}
16 changes: 16 additions & 0 deletions packages/mediapipe-task-genai/example/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# example

A new Flutter project.

## Getting Started

This project is a starting point for a Flutter application.

A few resources to get you started if this is your first Flutter project:

- [Lab: Write your first Flutter app](https://docs.flutter.dev/get-started/codelab)
- [Cookbook: Useful Flutter samples](https://docs.flutter.dev/cookbook)

For help getting started with Flutter development, view the
[online documentation](https://docs.flutter.dev/), which offers tutorials,
samples, guidance on mobile development, and a full API reference.
28 changes: 28 additions & 0 deletions packages/mediapipe-task-genai/example/analysis_options.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# This file configures the analyzer, which statically analyzes Dart code to
# check for errors, warnings, and lints.
#
# The issues identified by the analyzer are surfaced in the UI of Dart-enabled
# IDEs (https://dart.dev/tools#ides-and-editors). The analyzer can also be
# invoked from the command line by running `flutter analyze`.

# The following line activates a set of recommended lints for Flutter apps,
# packages, and plugins designed to encourage good coding practices.
include: package:flutter_lints/flutter.yaml

linter:
# The lint rules applied to this project can be customized in the
# section below to disable rules from the `package:flutter_lints/flutter.yaml`
# included above or to enable additional rules. A list of all available lints
# and their documentation is published at https://dart.dev/lints.
#
# Instead of disabling a lint rule for the entire project in the
# section below, it can also be suppressed for a single line of code
# or a specific dart file by using the `// ignore: name_of_lint` and
# `// ignore_for_file: name_of_lint` syntax on the line or in the file
# producing the lint.
rules:
# avoid_print: false # Uncomment to disable the `avoid_print` rule
# prefer_single_quotes: true # Uncomment to enable the `prefer_single_quotes` rule

# Additional information about this file can be found at
# https://dart.dev/guides/language/analysis-options
13 changes: 13 additions & 0 deletions packages/mediapipe-task-genai/example/android/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
gradle-wrapper.jar
/.gradle
/captures/
/gradlew
/gradlew.bat
/local.properties
GeneratedPluginRegistrant.java

# Remember to never publicly share your keystore.
# See https://flutter.dev/docs/deployment/android#reference-the-keystore-from-the-app
key.properties
**/*.keystore
**/*.jks
40 changes: 40 additions & 0 deletions packages/mediapipe-task-genai/example/android/app/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
plugins {
id "com.android.application"
id "kotlin-android"
// The Flutter Gradle Plugin must be applied after the Android and Kotlin Gradle plugins.
id "dev.flutter.flutter-gradle-plugin"
}

android {
namespace = "com.example.example"
compileSdk = flutter.compileSdkVersion
ndkVersion = flutter.ndkVersion

compileOptions {
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
}

defaultConfig {
// TODO: Specify your own unique Application ID (https://developer.android.com/studio/build/application-id.html).
applicationId = "com.example.example"
// You can update the following values to match your application needs.
// For more information, see: https://docs.flutter.dev/deployment/android#reviewing-the-gradle-build-configuration.
minSdk = flutter.minSdkVersion
targetSdk = flutter.targetSdkVersion
versionCode = flutter.versionCode
versionName = flutter.versionName
}

buildTypes {
release {
// TODO: Add your own signing config for the release build.
// Signing with the debug keys for now, so `flutter run --release` works.
signingConfig = signingConfigs.debug
}
}
}

flutter {
source = "../.."
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<!-- The INTERNET permission is required for development. Specifically,
the Flutter tool needs it to communicate with the running application
to allow setting breakpoints, to provide hot reload, etc.
-->
<uses-permission android:name="android.permission.INTERNET"/>
</manifest>
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<application
android:label="example"
android:name="${applicationName}"
android:icon="@mipmap/ic_launcher">
<activity
android:name=".MainActivity"
android:exported="true"
android:launchMode="singleTop"
android:taskAffinity=""
android:theme="@style/LaunchTheme"
android:configChanges="orientation|keyboardHidden|keyboard|screenSize|smallestScreenSize|locale|layoutDirection|fontScale|screenLayout|density|uiMode"
android:hardwareAccelerated="true"
android:windowSoftInputMode="adjustResize">
<!-- Specifies an Android theme to apply to this Activity as soon as
the Android process has started. This theme is visible to the user
while the Flutter UI initializes. After that, this theme continues
to determine the Window background behind the Flutter UI. -->
<meta-data
android:name="io.flutter.embedding.android.NormalTheme"
android:resource="@style/NormalTheme"
/>
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
<!-- Don't delete the meta-data below.
This is used by the Flutter tool to generate GeneratedPluginRegistrant.java -->
<meta-data
android:name="flutterEmbedding"
android:value="2" />
</application>
<!-- Required to query activities that can process text, see:
https://developer.android.com/training/package-visibility and
https://developer.android.com/reference/android/content/Intent#ACTION_PROCESS_TEXT.
In particular, this is used by the Flutter engine in io.flutter.plugin.text.ProcessTextPlugin. -->
<queries>
<intent>
<action android:name="android.intent.action.PROCESS_TEXT"/>
<data android:mimeType="text/plain"/>
</intent>
</queries>
</manifest>
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
package com.example.example

import io.flutter.embedding.android.FlutterActivity

class MainActivity: FlutterActivity()
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Modify this file to customize your launch splash screen -->
<layer-list xmlns:android="http://schemas.android.com/apk/res/android">
<item android:drawable="?android:colorBackground" />

<!-- You can insert your own image assets here -->
<!-- <item>
<bitmap
android:gravity="center"
android:src="@mipmap/launch_image" />
</item> -->
</layer-list>
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Modify this file to customize your launch splash screen -->
<layer-list xmlns:android="http://schemas.android.com/apk/res/android">
<item android:drawable="@android:color/white" />

<!-- You can insert your own image assets here -->
<!-- <item>
<bitmap
android:gravity="center"
android:src="@mipmap/launch_image" />
</item> -->
</layer-list>
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<!-- Theme applied to the Android Window while the process is starting when the OS's Dark Mode setting is on -->
<style name="LaunchTheme" parent="@android:style/Theme.Black.NoTitleBar">
<!-- Show a splash screen on the activity. Automatically removed when
the Flutter engine draws its first frame -->
<item name="android:windowBackground">@drawable/launch_background</item>
</style>
<!-- Theme applied to the Android Window as soon as the process has started.
This theme determines the color of the Android Window while your
Flutter UI initializes, as well as behind your Flutter UI while its
running.
This Theme is only used starting with V2 of Flutter's Android embedding. -->
<style name="NormalTheme" parent="@android:style/Theme.Black.NoTitleBar">
<item name="android:windowBackground">?android:colorBackground</item>
</style>
</resources>
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<!-- Theme applied to the Android Window while the process is starting when the OS's Dark Mode setting is off -->
<style name="LaunchTheme" parent="@android:style/Theme.Light.NoTitleBar">
<!-- Show a splash screen on the activity. Automatically removed when
the Flutter engine draws its first frame -->
<item name="android:windowBackground">@drawable/launch_background</item>
</style>
<!-- Theme applied to the Android Window as soon as the process has started.
This theme determines the color of the Android Window while your
Flutter UI initializes, as well as behind your Flutter UI while its
running.
This Theme is only used starting with V2 of Flutter's Android embedding. -->
<style name="NormalTheme" parent="@android:style/Theme.Light.NoTitleBar">
<item name="android:windowBackground">?android:colorBackground</item>
</style>
</resources>
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<!-- The INTERNET permission is required for development. Specifically,
the Flutter tool needs it to communicate with the running application
to allow setting breakpoints, to provide hot reload, etc.
-->
<uses-permission android:name="android.permission.INTERNET"/>
</manifest>
18 changes: 18 additions & 0 deletions packages/mediapipe-task-genai/example/android/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
allprojects {
repositories {
google()
mavenCentral()
}
}

rootProject.buildDir = "../build"
subprojects {
project.buildDir = "${rootProject.buildDir}/${project.name}"
}
subprojects {
project.evaluationDependsOn(":app")
}

tasks.register("clean", Delete) {
delete rootProject.buildDir
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
org.gradle.jvmargs=-Xmx4G -XX:MaxMetaspaceSize=2G -XX:+HeapDumpOnOutOfMemoryError
android.useAndroidX=true
android.enableJetifier=true
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6.3-all.zip
25 changes: 25 additions & 0 deletions packages/mediapipe-task-genai/example/android/settings.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
pluginManagement {
def flutterSdkPath = {
def properties = new Properties()
file("local.properties").withInputStream { properties.load(it) }
def flutterSdkPath = properties.getProperty("flutter.sdk")
assert flutterSdkPath != null, "flutter.sdk not set in local.properties"
return flutterSdkPath
}()

includeBuild("$flutterSdkPath/packages/flutter_tools/gradle")

repositories {
google()
mavenCentral()
gradlePluginPortal()
}
}

plugins {
id "dev.flutter.flutter-plugin-loader" version "1.0.0"
id "com.android.application" version "7.3.0" apply false
id "org.jetbrains.kotlin.android" version "1.7.10" apply false
}

include ":app"
34 changes: 34 additions & 0 deletions packages/mediapipe-task-genai/example/ios/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
**/dgph
*.mode1v3
*.mode2v3
*.moved-aside
*.pbxuser
*.perspectivev3
**/*sync/
.sconsign.dblite
.tags*
**/.vagrant/
**/DerivedData/
Icon?
**/Pods/
**/.symlinks/
profile
xcuserdata
**/.generated/
Flutter/App.framework
Flutter/Flutter.framework
Flutter/Flutter.podspec
Flutter/Generated.xcconfig
Flutter/ephemeral/
Flutter/app.flx
Flutter/app.zip
Flutter/flutter_assets/
Flutter/flutter_export_environment.sh
ServiceDefinitions.json
Runner/GeneratedPluginRegistrant.*

# Exceptions to above rules.
!default.mode1v3
!default.mode2v3
!default.pbxuser
!default.perspectivev3
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>en</string>
<key>CFBundleExecutable</key>
<string>App</string>
<key>CFBundleIdentifier</key>
<string>io.flutter.flutter.app</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>App</string>
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>1.0</string>
<key>MinimumOSVersion</key>
<string>12.0</string>
</dict>
</plist>
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#include? "Pods/Target Support Files/Pods-Runner/Pods-Runner.debug.xcconfig"
#include "Generated.xcconfig"
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#include? "Pods/Target Support Files/Pods-Runner/Pods-Runner.release.xcconfig"
#include "Generated.xcconfig"
44 changes: 44 additions & 0 deletions packages/mediapipe-task-genai/example/ios/Podfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Uncomment this line to define a global platform for your project
# platform :ios, '12.0'

# CocoaPods analytics sends network stats synchronously affecting flutter build latency.
ENV['COCOAPODS_DISABLE_STATS'] = 'true'

project 'Runner', {
'Debug' => :debug,
'Profile' => :release,
'Release' => :release,
}

def flutter_root
generated_xcode_build_settings_path = File.expand_path(File.join('..', 'Flutter', 'Generated.xcconfig'), __FILE__)
unless File.exist?(generated_xcode_build_settings_path)
raise "#{generated_xcode_build_settings_path} must exist. If you're running pod install manually, make sure flutter pub get is executed first"
end

File.foreach(generated_xcode_build_settings_path) do |line|
matches = line.match(/FLUTTER_ROOT\=(.*)/)
return matches[1].strip if matches
end
raise "FLUTTER_ROOT not found in #{generated_xcode_build_settings_path}. Try deleting Generated.xcconfig, then run flutter pub get"
end

require File.expand_path(File.join('packages', 'flutter_tools', 'bin', 'podhelper'), flutter_root)

flutter_ios_podfile_setup

target 'Runner' do
use_frameworks!
use_modular_headers!

flutter_install_all_ios_pods File.dirname(File.realpath(__FILE__))
target 'RunnerTests' do
inherit! :search_paths
end
end

post_install do |installer|
installer.pods_project.targets.each do |target|
flutter_additional_ios_build_settings(target)
end
end

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>IDEDidComputeMac32BitWarning</key>
<true/>
</dict>
</plist>
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>PreviewsEnabled</key>
<false/>
</dict>
</plist>
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
<?xml version="1.0" encoding="UTF-8"?>
<Scheme
LastUpgradeVersion = "1510"
version = "1.3">
<BuildAction
parallelizeBuildables = "YES"
buildImplicitDependencies = "YES">
<BuildActionEntries>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "97C146ED1CF9000F007C117D"
BuildableName = "Runner.app"
BlueprintName = "Runner"
ReferencedContainer = "container:Runner.xcodeproj">
</BuildableReference>
</BuildActionEntry>
</BuildActionEntries>
</BuildAction>
<TestAction
buildConfiguration = "Debug"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
shouldUseLaunchSchemeArgsEnv = "YES">
<MacroExpansion>
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "97C146ED1CF9000F007C117D"
BuildableName = "Runner.app"
BlueprintName = "Runner"
ReferencedContainer = "container:Runner.xcodeproj">
</BuildableReference>
</MacroExpansion>
<Testables>
<TestableReference
skipped = "NO"
parallelizable = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "331C8080294A63A400263BE5"
BuildableName = "RunnerTests.xctest"
BlueprintName = "RunnerTests"
ReferencedContainer = "container:Runner.xcodeproj">
</BuildableReference>
</TestableReference>
</Testables>
</TestAction>
<LaunchAction
buildConfiguration = "Debug"
selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
launchStyle = "0"
useCustomWorkingDirectory = "NO"
ignoresPersistentStateOnLaunch = "NO"
debugDocumentVersioning = "YES"
debugServiceExtension = "internal"
allowLocationSimulation = "YES">
<BuildableProductRunnable
runnableDebuggingMode = "0">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "97C146ED1CF9000F007C117D"
BuildableName = "Runner.app"
BlueprintName = "Runner"
ReferencedContainer = "container:Runner.xcodeproj">
</BuildableReference>
</BuildableProductRunnable>
</LaunchAction>
<ProfileAction
buildConfiguration = "Profile"
shouldUseLaunchSchemeArgsEnv = "YES"
savedToolIdentifier = ""
useCustomWorkingDirectory = "NO"
debugDocumentVersioning = "YES">
<BuildableProductRunnable
runnableDebuggingMode = "0">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "97C146ED1CF9000F007C117D"
BuildableName = "Runner.app"
BlueprintName = "Runner"
ReferencedContainer = "container:Runner.xcodeproj">
</BuildableReference>
</BuildableProductRunnable>
</ProfileAction>
<AnalyzeAction
buildConfiguration = "Debug">
</AnalyzeAction>
<ArchiveAction
buildConfiguration = "Release"
revealArchiveInOrganizer = "YES">
</ArchiveAction>
</Scheme>
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>IDEDidComputeMac32BitWarning</key>
<true/>
</dict>
</plist>
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>PreviewsEnabled</key>
<false/>
</dict>
</plist>
13 changes: 13 additions & 0 deletions packages/mediapipe-task-genai/example/ios/Runner/AppDelegate.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import Flutter
import UIKit

@main
@objc class AppDelegate: FlutterAppDelegate {
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
GeneratedPluginRegistrant.register(with: self)
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
{
"images" : [
{
"size" : "20x20",
"idiom" : "iphone",
"filename" : "Icon-App-20x20@2x.png",
"scale" : "2x"
},
{
"size" : "20x20",
"idiom" : "iphone",
"filename" : "Icon-App-20x20@3x.png",
"scale" : "3x"
},
{
"size" : "29x29",
"idiom" : "iphone",
"filename" : "Icon-App-29x29@1x.png",
"scale" : "1x"
},
{
"size" : "29x29",
"idiom" : "iphone",
"filename" : "Icon-App-29x29@2x.png",
"scale" : "2x"
},
{
"size" : "29x29",
"idiom" : "iphone",
"filename" : "Icon-App-29x29@3x.png",
"scale" : "3x"
},
{
"size" : "40x40",
"idiom" : "iphone",
"filename" : "Icon-App-40x40@2x.png",
"scale" : "2x"
},
{
"size" : "40x40",
"idiom" : "iphone",
"filename" : "Icon-App-40x40@3x.png",
"scale" : "3x"
},
{
"size" : "60x60",
"idiom" : "iphone",
"filename" : "Icon-App-60x60@2x.png",
"scale" : "2x"
},
{
"size" : "60x60",
"idiom" : "iphone",
"filename" : "Icon-App-60x60@3x.png",
"scale" : "3x"
},
{
"size" : "20x20",
"idiom" : "ipad",
"filename" : "Icon-App-20x20@1x.png",
"scale" : "1x"
},
{
"size" : "20x20",
"idiom" : "ipad",
"filename" : "Icon-App-20x20@2x.png",
"scale" : "2x"
},
{
"size" : "29x29",
"idiom" : "ipad",
"filename" : "Icon-App-29x29@1x.png",
"scale" : "1x"
},
{
"size" : "29x29",
"idiom" : "ipad",
"filename" : "Icon-App-29x29@2x.png",
"scale" : "2x"
},
{
"size" : "40x40",
"idiom" : "ipad",
"filename" : "Icon-App-40x40@1x.png",
"scale" : "1x"
},
{
"size" : "40x40",
"idiom" : "ipad",
"filename" : "Icon-App-40x40@2x.png",
"scale" : "2x"
},
{
"size" : "76x76",
"idiom" : "ipad",
"filename" : "Icon-App-76x76@1x.png",
"scale" : "1x"
},
{
"size" : "76x76",
"idiom" : "ipad",
"filename" : "Icon-App-76x76@2x.png",
"scale" : "2x"
},
{
"size" : "83.5x83.5",
"idiom" : "ipad",
"filename" : "Icon-App-83.5x83.5@2x.png",
"scale" : "2x"
},
{
"size" : "1024x1024",
"idiom" : "ios-marketing",
"filename" : "Icon-App-1024x1024@1x.png",
"scale" : "1x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"images" : [
{
"idiom" : "universal",
"filename" : "LaunchImage.png",
"scale" : "1x"
},
{
"idiom" : "universal",
"filename" : "LaunchImage@2x.png",
"scale" : "2x"
},
{
"idiom" : "universal",
"filename" : "LaunchImage@3x.png",
"scale" : "3x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Launch Screen Assets

You can customize the launch screen with your own desired assets by replacing the image files in this directory.

You can also do it by opening your Flutter project's Xcode project with `open ios/Runner.xcworkspace`, selecting `Runner/Assets.xcassets` in the Project Navigator and dropping in the desired images.
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<document type="com.apple.InterfaceBuilder3.CocoaTouch.Storyboard.XIB" version="3.0" toolsVersion="12121" systemVersion="16G29" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" launchScreen="YES" colorMatched="YES" initialViewController="01J-lp-oVM">
<dependencies>
<deployment identifier="iOS"/>
<plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="12089"/>
</dependencies>
<scenes>
<!--View Controller-->
<scene sceneID="EHf-IW-A2E">
<objects>
<viewController id="01J-lp-oVM" sceneMemberID="viewController">
<layoutGuides>
<viewControllerLayoutGuide type="top" id="Ydg-fD-yQy"/>
<viewControllerLayoutGuide type="bottom" id="xbc-2k-c8Z"/>
</layoutGuides>
<view key="view" contentMode="scaleToFill" id="Ze5-6b-2t3">
<autoresizingMask key="autoresizingMask" widthSizable="YES" heightSizable="YES"/>
<subviews>
<imageView opaque="NO" clipsSubviews="YES" multipleTouchEnabled="YES" contentMode="center" image="LaunchImage" translatesAutoresizingMaskIntoConstraints="NO" id="YRO-k0-Ey4">
</imageView>
</subviews>
<color key="backgroundColor" red="1" green="1" blue="1" alpha="1" colorSpace="custom" customColorSpace="sRGB"/>
<constraints>
<constraint firstItem="YRO-k0-Ey4" firstAttribute="centerX" secondItem="Ze5-6b-2t3" secondAttribute="centerX" id="1a2-6s-vTC"/>
<constraint firstItem="YRO-k0-Ey4" firstAttribute="centerY" secondItem="Ze5-6b-2t3" secondAttribute="centerY" id="4X2-HB-R7a"/>
</constraints>
</view>
</viewController>
<placeholder placeholderIdentifier="IBFirstResponder" id="iYj-Kq-Ea1" userLabel="First Responder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="53" y="375"/>
</scene>
</scenes>
<resources>
<image name="LaunchImage" width="168" height="185"/>
</resources>
</document>
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<document type="com.apple.InterfaceBuilder3.CocoaTouch.Storyboard.XIB" version="3.0" toolsVersion="10117" systemVersion="15F34" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" useTraitCollections="YES" initialViewController="BYZ-38-t0r">
<dependencies>
<deployment identifier="iOS"/>
<plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="10085"/>
</dependencies>
<scenes>
<!--Flutter View Controller-->
<scene sceneID="tne-QT-ifu">
<objects>
<viewController id="BYZ-38-t0r" customClass="FlutterViewController" sceneMemberID="viewController">
<layoutGuides>
<viewControllerLayoutGuide type="top" id="y3c-jy-aDJ"/>
<viewControllerLayoutGuide type="bottom" id="wfy-db-euE"/>
</layoutGuides>
<view key="view" contentMode="scaleToFill" id="8bC-Xf-vdC">
<rect key="frame" x="0.0" y="0.0" width="600" height="600"/>
<autoresizingMask key="autoresizingMask" widthSizable="YES" heightSizable="YES"/>
<color key="backgroundColor" white="1" alpha="1" colorSpace="custom" customColorSpace="calibratedWhite"/>
</view>
</viewController>
<placeholder placeholderIdentifier="IBFirstResponder" id="dkx-z0-nzr" sceneMemberID="firstResponder"/>
</objects>
</scene>
</scenes>
</document>
49 changes: 49 additions & 0 deletions packages/mediapipe-task-genai/example/ios/Runner/Info.plist
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleDisplayName</key>
<string>Example</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>example</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>$(FLUTTER_BUILD_NAME)</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>$(FLUTTER_BUILD_NUMBER)</string>
<key>LSRequiresIPhoneOS</key>
<true/>
<key>UIApplicationSupportsIndirectInputEvents</key>
<true/>
<key>UILaunchStoryboardName</key>
<string>LaunchScreen</string>
<key>UIMainStoryboardFile</key>
<string>Main</string>
<key>UISupportedInterfaceOrientations</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
<key>UISupportedInterfaceOrientations~ipad</key>
<array>
<string>UIInterfaceOrientationPortrait</string>
<string>UIInterfaceOrientationPortraitUpsideDown</string>
<string>UIInterfaceOrientationLandscapeLeft</string>
<string>UIInterfaceOrientationLandscapeRight</string>
</array>
</dict>
</plist>
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#import "GeneratedPluginRegistrant.h"
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import Flutter
import UIKit
import XCTest

class RunnerTests: XCTestCase {

func testExample() {
// If you add code to the Runner application, consider adding tests here.
// See https://developer.apple.com/documentation/xctest for more information about using XCTest.
}

}
510 changes: 510 additions & 0 deletions packages/mediapipe-task-genai/example/lib/bloc.dart

Large diffs are not rendered by default.

3,012 changes: 3,012 additions & 0 deletions packages/mediapipe-task-genai/example/lib/bloc.freezed.dart

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
// // Copyright 2014 The Flutter Authors. All rights reserved.
// // Use of this source code is governed by a BSD-style license that can be
// // found in the LICENSE file.

// import 'dart:async';
// import 'dart:ffi';
// import 'dart:math';

// import 'package:logging/logging.dart';
// import 'package:mediapipe_genai/mediapipe_genai.dart';

// final _log = Logger('FakeInferenceEngine');

// class FakeInferenceEngine implements LlmInferenceEngine {
// FakeInferenceEngine(LlmInferenceOptions options) {
// _log.info('Initializing FakeInferenceEngine with $options');
// }

// @override
// Stream<String> generateResponse(String text) {
// final controller = StreamController<String>.broadcast();

// final rnd = Random();
// // Delay between 500 and 1000ms
// Future.delayed(Duration(milliseconds: rnd.nextInt(500) + 500)).then(
// (_) async {
// final List<String> message =
// _genericResponses[rnd.nextInt(_genericResponses.length)];

// for (final chunk in message) {
// // Delay between 500 and 1000ms
// await Future.delayed(Duration(milliseconds: rnd.nextInt(500) + 500));
// controller.add(chunk);
// }
// controller.close();
// },
// );

// return controller.stream;
// }

// @override
// Future<int> sizeInTokens(String text) async =>
// (text.split(' ').length * 1.5).toInt();

// // These three methods, `cancel`, `dispose`, and `handleErrorMessage` serve
// // no purpose whatsoever for this fake implementation and should not be
// // necessary to implement here; but surprisingly the type system won't compile
// // if they aren't present. Strangely, the analysis server does not report any
// // issue, so the (false positive, I believe) error only arises when you run
// // the actual app.
// void cancel() => throw UnimplementedError();

// @override
// void dispose() => throw UnimplementedError();

// void handleErrorMessage(Pointer<Pointer<Char>> errorMessage, [int? status]) =>
// throw UnimplementedError();
// }

// const _genericResponses = <List<String>>[
// ['Hello', 'my good', 'friend!', 'How are you doing?', 'Are you well?'],
// ['Well now, don\'t', 'get so hasty! There are still reasonable', 'options.'],
// ['From what I\'m hearing,', 'it sounds like you need to dance more.'],
// [
// 'I too enjoy a nice sandwich,',
// 'but you\'ve gotta',
// 'toast the bread - come',
// 'on!',
// ],
// ['I don\'t appreciate that slander and', 'I own\'t', 'stand for it!'],
// ];
94 changes: 94 additions & 0 deletions packages/mediapipe-task-genai/example/lib/llm_inference_demo.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:example/bloc.dart';
import 'package:example/model_selection_screen.dart';
import 'package:example/models/models.dart';
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:mediapipe_genai/mediapipe_genai.dart';
// import 'fake_inference_engine.dart';
import 'widgets/widgets.dart';

class LlmInferenceDemo extends StatefulWidget {
const LlmInferenceDemo({super.key});

@override
State<LlmInferenceDemo> createState() => _LlmInferenceDemoState();
}

class _LlmInferenceDemoState extends State<LlmInferenceDemo>
with AutomaticKeepAliveClientMixin<LlmInferenceDemo> {
final results = <Widget>[];

late TranscriptBloc bloc;

@override
void initState() {
super.initState();
bloc = TranscriptBloc(
engineBuilder: LlmInferenceEngine.new,
// engineBuilder: FakeInferenceEngine.new,
);
}

@override
Widget build(BuildContext context) {
super.build(context);

return MultiBlocListener(
listeners: [
// Error listener
BlocListener<TranscriptBloc, TranscriptState>(
bloc: bloc,
listener: (context, TranscriptState state) {
if (state.error != null) {
ScaffoldMessenger.of(context).showSnackBar(
SnackBar(content: Text(state.error!)),
);
}
},
),
],
child: BlocBuilder(
bloc: bloc,
builder: (context, TranscriptState state) {
return Scaffold(
appBar: AppBar(title: const Text('Inference')),
body: SafeArea(
child: ModelSelectionScreen(
modelsReady: state.modelsReady,
deleteModel: (LlmModel model) => bloc.add(DeleteModel(model)),
downloadModel: (LlmModel model) =>
bloc.add(DownloadModel(model)),
modelInfoMap: state.modelInfoMap,
selectModel: (LlmModel model) {
Navigator.push(
context,
MaterialPageRoute(
builder: (context) => ChatScreen(
bloc,
model: model,
key: ValueKey(model.displayName),
),
),
);
},
),
),
);
},
),
);
}

@override
bool get wantKeepAlive => true;

@override
void dispose() {
bloc.close();
super.dispose();
}
}
21 changes: 21 additions & 0 deletions packages/mediapipe-task-genai/example/lib/logging.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:io' as io;
import 'package:logging/logging.dart';

final log = Logger('Genai');

void initLogging() {
Logger.root.level = Level.FINEST;
Logger.root.onRecord.listen((record) {
io.stdout.writeln('${record.level.name} [${record.loggerName}]'
'['
'${record.time.hour.toString()}:'
'${record.time.minute.toString().padLeft(2, "0")}:'
'${record.time.second.toString().padLeft(2, "0")}.'
'${record.time.millisecond.toString().padRight(3, "0")}'
'] ${record.message}');
});
}
119 changes: 119 additions & 0 deletions packages/mediapipe-task-genai/example/lib/main.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:flutter_localizations/flutter_localizations.dart';
import 'package:logging/logging.dart';
import 'llm_inference_demo.dart';
import 'logging.dart';

/// {@template AppBlocObserver}
/// Logger of all things related to `pkg:flutter_bloc`.
/// {@endtemplate}
class AppBlocObserver extends BlocObserver {
/// {@macro AppBlocObserver}
const AppBlocObserver();

static final _log = Logger('AppBlocObserver');

@override
void onChange(BlocBase<dynamic> bloc, Change<dynamic> change) {
_log.finer('onChange(${bloc.runtimeType}, $change)');
super.onChange(bloc, change);
}

@override
void onEvent(Bloc<dynamic, dynamic> bloc, Object? event) {
_log.finer('onEvent($event)');
super.onEvent(bloc, event);
}

@override
void onError(BlocBase<dynamic> bloc, Object error, StackTrace stackTrace) {
// print('onError(${bloc.runtimeType}, $error, $stackTrace)');
super.onError(bloc, error, stackTrace);
}
}

void main() {
initLogging();
Bloc.observer = const AppBlocObserver();
runApp(
const MaterialApp(
localizationsDelegates: [
GlobalMaterialLocalizations.delegate,
GlobalWidgetsLocalizations.delegate,
GlobalCupertinoLocalizations.delegate, // This is required
],
home: MainApp(),
),
);
}

class MainApp extends StatefulWidget {
const MainApp({super.key});

@override
State<MainApp> createState() => _MainAppState();
}

class _MainAppState extends State<MainApp> {
final PageController controller = PageController();

final titles = <String>['Inference'];
int titleIndex = 0;

void switchToPage(int index) {
controller.animateToPage(
index,
duration: const Duration(milliseconds: 200),
curve: Curves.easeOut,
);
setState(() {
titleIndex = index;
});
}

@override
Widget build(BuildContext context) {
const activeTextStyle = TextStyle(
fontWeight: FontWeight.bold,
color: Colors.orange,
);
const inactiveTextStyle = TextStyle(
color: Colors.white,
);
return Scaffold(
body: PageView(
controller: controller,
children: const <Widget>[
LlmInferenceDemo(),
],
),
bottomNavigationBar: SizedBox(
height: 50 + MediaQuery.of(context).viewPadding.bottom / 2,
child: ColoredBox(
color: Colors.blueGrey,
child: Column(
children: [
Row(
mainAxisAlignment: MainAxisAlignment.spaceEvenly,
children: <Widget>[
TextButton(
onPressed: () => switchToPage(0),
child: Text(
'Inference',
style:
titleIndex == 0 ? activeTextStyle : inactiveTextStyle,
),
),
],
),
SizedBox(
height: MediaQuery.of(context).viewPadding.bottom / 2,
),
],
),
),
),
);
}
}
243 changes: 243 additions & 0 deletions packages/mediapipe-task-genai/example/lib/model_location_provider.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
import 'dart:async';
import 'dart:io';
import 'package:example/models/models.dart';
import 'package:example/model_storage/model_storage.dart';
import 'package:http/http.dart' as http;
import 'package:logging/logging.dart';

final _log = Logger('ModelLocationProvider');

/// Provides the locations of a given [LlmModel].
///
/// There are two constructors, but [fromEnvironment] is expected to be the most
/// useful assuming the Flutter application was build with the
/// `--dart-define=GEMMA_8B_GPU_PATH=<location>` flags, or similar, depending
/// on which models the developer and user intend to use.
///
/// Usage:
/// ```dart
/// final provider = ModelLocationProvider.fromEnvironment();
/// (Future<String> locationFuture, Stream<int> downloadProgress,) =
/// await provider.getModelLocation(LlmModel.gemma8bGpu);
/// if (downloadProgress != null) {
/// await for (final percent in downloadProgress) {
/// showDownloadPercentage(percent);
/// }
/// }
/// // Should complete instantly
/// final location = await locationFuture;
/// ```
///
/// See also:
/// * [LlmModel], the enum which tracks each available LLM.
class ModelLocationProvider {
ModelLocationProvider._({required ModelPaths modelLocations}) {
storage = ModelStorage()
..setInitialModelLocations(modelLocations).then(
(_) {
_ready.complete();
},
);
}
factory ModelLocationProvider.fromEnvironment() {
return ModelLocationProvider._(
modelLocations: ModelLocationProvider._getModelLocationsFromEnvironment(),
);
}

late final ModelStorageInterface storage;

final _ready = Completer<void>();
Future<void> get ready => _ready.future;

// Useful for quick development if there is any friction around passing
// environment variables
static const hardcodedLocations = <LlmModel, String>{
LlmModel.gemma4bCpu:
'https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-cpu-int4.bin',
LlmModel.gemma4bGpu:
'https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-gpu-int4.bin',
LlmModel.gemma8bCpu:
'https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-cpu-int8.bin',
LlmModel.gemma8bGpu:
'https://storage.googleapis.com/random-storage-asdf/gemma/gemma-2b-it-gpu-int8.bin',
};

static ModelPaths _getModelLocationsFromEnvironment() {
final locations = <LlmModel, String>{};
for (final model in LlmModel.values) {
String location = hardcodedLocations[model] ??
Platform.environment[model.environmentVariableUriName] ??
model.dartDefine;

// `model.dartDefine` has an empty state of an empty string, not null,
// which is why `location` is a `String` and not a `String?`
if (location.isNotEmpty) {
locations[model] = location;
}
}
return locations;
}

/// {@macro downloadExists}
Future<bool> downloadExists(String downloadLocation) =>
storage.downloadExists(downloadLocation);

Future<bool> downloadExistsForModel(LlmModel model) async {
String? path = storage.pathFor(model);
return path == null ? false : storage.downloadExists(path);
}

Future<void> delete(LlmModel model) async => await storage.delete(model);

/// {@macro binarySize}
int binarySize(String location) => storage.binarySize(location);

/// Accepts a model and returns the size on disk for said model if it is
/// already downloaded and fully available.
int? binarySizeForModel(LlmModel model) {
final location = pathFor(model);
return location != null ? storage.binarySize(location) : null;
}

/// {@macro pathFor}
String? pathFor(LlmModel model) => storage.pathFor(model);

/// {@macro urlFor}
Uri? urlFor(LlmModel model) => storage.urlFor(model);

/// Storage for the controllers tied to in-progress downloads, each of which
/// should emit download progress updates as a percentage.
final _downloadControllers = <LlmModel, StreamController<int>>{};

/// Asychronously returns the String for the location of a locally available
/// copy of the requested model and an optional download progress stream if
/// the model is downloading.
Future<(Future<String>, Stream<int>?)> getModelLocation(
LlmModel model,
) async {
await ready;
final path = storage.pathFor(model);
if (path != null) {
if (!await storage.downloadExists(path)) {
throw Exception(
'Location $path for $model was expected to have a file, but it is '
'missing.',
);
}

final binarySize = storage.binarySize(path);
if (binarySize > 0) {
_log.finer(
'Returning already downloaded model for $model of '
'$binarySize bytes',
);
return (Future.value(path), null);
}
_log.finer('Deleting existing $model of 0 bytes');
storage.delete(model);
}

final url = storage.urlFor(model);
if (url != null) {
return _getOrStartDownload(model, url);
}
throw Exception(
'ModelLocationProvider does not know where to find a '
'${model.name} model, either on disk or elsewhere. Did you include a '
'clause like `${model.environmentVariableUriName}=<location>` before '
'`flutter run`? See the example README for more details.',
);
}

/// Either initiates a fresh download or returns an ongoing download.
Future<(Future<String>, Stream<int>?)> _getOrStartDownload(
LlmModel model,
Uri url,
) async {
final downloadDestination = await storage.urlToDownloadDestination(url);

// This is the way to figure out if the file is still being downloaded.
// If there is a download controller for the model, we must await its
// completion.
if (!_downloadControllers.containsKey(model)) {
return (
Future.value(downloadDestination),
await _downloadFile(model, url, downloadDestination)
);
}
return (
Future.value(downloadDestination),
_downloadControllers[model]!.stream
);
}

/// Downloads the file, creating and updating a stream with progress.
///
/// This method does not return the location of the file, and in fact should
/// not be called directly. Call [getModelLocation], which makes use of
/// [_getOrStartDownload] if the file is not yet available on disk.
Future<Stream<int>> _downloadFile(
LlmModel model,
Uri location,
String downloadDestination,
) async {
// Prepare a place for the file to be downloaded.
if (await storage.downloadExists(downloadDestination)) {
throw Exception('File exists at LLM model location in _downloadFile, '
'which expects to only be called when said model location is empty. '
'Unexpectedly occupied file location was ${location.path}');
}
final downloadSink = await storage.create(downloadDestination);

// Setup the request itself and read preliminary headers.
final request = http.Request('GET', location);
final response = await request.send();
final contentLength = int.parse(response.headers['content-length']!);
_log.finer('File download: $contentLength bytes');
final downloadCompleter = Completer<bool>();
int downloadedBytes = 0;
int lastPercentEmitted = 0;

// Setup our reporting stream. Use a broadcast stream because calls to
// `_getOrStartDownload` will need to resubscribe if the file is already
// being downloaded.
_downloadControllers[model] = StreamController<int>.broadcast();

// Actually begin downloading
response.stream.listen(
(List<int> bytes) {
downloadSink.add(bytes);
downloadedBytes += bytes.length;
int percent = ((downloadedBytes / contentLength) * 100).toInt();
if ((percent > lastPercentEmitted)) {
_log.finest('ModelLocationProvider :: $percent%');
_downloadControllers[model]!.add(percent);
lastPercentEmitted = percent;
}
},
onDone: () => downloadCompleter.complete(true),
onError: (error, stacktrace) {
_log.shout('error: $error');
_log.shout('stacktrace: $stacktrace');
downloadCompleter.complete(false);
},
);
downloadCompleter.future.then((downloadSuccessful) async {
if (downloadSuccessful) {
// Setting this value marks the model binary as fully downloaded.
storage.setPathCache(model, downloadDestination);
storage.close(downloadDestination);
} else {
// If the download did not complete, remove a partial download
storage.abort(downloadDestination);
}
// Clean-up resources
// Closing this controller is what signals to outside code that the file
// download is complete.
_downloadControllers[model]?.close();
_downloadControllers.remove(model);
});
return _downloadControllers[model]!.stream;
}
}
223 changes: 223 additions & 0 deletions packages/mediapipe-task-genai/example/lib/model_selection_screen.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
// import 'package:example/bloc.dart';
import 'dart:math';

import 'package:adaptive_dialog/adaptive_dialog.dart';
import 'package:example/models/llm_model.dart';
import 'package:flutter/material.dart';
import 'package:getwidget/getwidget.dart';

class ModelSelectionScreen extends StatelessWidget {
const ModelSelectionScreen({
required this.deleteModel,
required this.downloadModel,
required this.modelInfoMap,
required this.modelsReady,
required this.selectModel,
super.key,
});

// final TranscriptBloc bloc;
final Map<LlmModel, ModelInfo> modelInfoMap;
final bool modelsReady;

/// Handler to delete the selected model.
final Function(LlmModel) deleteModel;

/// Handler to downloaded the selected model.
final Function(LlmModel) downloadModel;

/// Handler to change the selected model.
final Function(LlmModel) selectModel;

@override
Widget build(BuildContext context) {
return Padding(
padding: const EdgeInsets.all(8.0),
child: ListView(
// scrollDirection: Axis.horizontal,
children: <Widget>[
_modelSelectionTile(LlmModel.gemma4bCpu),
_modelSelectionTile(LlmModel.gemma4bGpu),
_modelSelectionTile(LlmModel.gemma8bCpu),
_modelSelectionTile(LlmModel.gemma8bGpu),
],
),
);
}

Widget _modelSelectionTile(LlmModel model) {
return ModelSelectionTile(
model,
ready: modelsReady,
modelInfo: modelInfoMap[model]!,
selectModel: () => selectModel(model),
delete: () => deleteModel(model),
download: () => downloadModel(model),
);
}
}

class ModelSelectionTile extends StatelessWidget {
const ModelSelectionTile(
this.model, {
required this.delete,
required this.download,
required this.modelInfo,
required this.ready,
required this.selectModel,
super.key,
});

final LlmModel model;
final ModelInfo modelInfo;

/// False until the initial state of models is fully loaded. Do not render
/// warnings until this is true.
final bool ready;

/// Handler to change the selected model to [model].
final VoidCallback selectModel;

/// Handler to downloaded [model].
final VoidCallback download;

/// Handler to delete [model].
final VoidCallback delete;

@override
Widget build(BuildContext context) {
return GestureDetector(
onTap: _handleTap,
child: Card(
child: ListTile(
title: Text(
model.displayName,
// style: const TextStyle(fontSize: 12),
),
leading: switch (modelInfo.state) {
ModelState.downloaded =>
const Icon(Icons.check, color: Colors.green),
ModelState.downloading => const Icon(Icons.downloading),
ModelState.empty => SizedBox.fromSize(size: const Size.square(1)),
},
subtitle: switch (modelInfo.state) {
ModelState.downloaded => Text(
humanize(modelInfo.downloadedBytes!),
style: TextStyle(fontSize: 11, color: Colors.grey[600]!),
),
ModelState.downloading =>
_DownloadingBar(downloadPercent: modelInfo.downloadPercent!),
ModelState.empty => ready && modelInfo.remoteLocation == null
? const Text('Configure URL to download',
style: TextStyle(color: Colors.red))
: Container(), // const Icon(Icons.download),
},
trailing: GestureDetector(
onTap: () => _launchModal(context),
child: switch (modelInfo.state) {
ModelState.downloaded => const Icon(Icons.delete),
ModelState.downloading =>
SizedBox.fromSize(size: const Size.square(1)),
ModelState.empty => modelInfo.remoteLocation == null
? SizedBox.fromSize(size: const Size.square(1))
: const Icon(Icons.download),
},
),
),
),
);
}

// Responds to taps anywhere on the `ModelSelectionTile` except for the
// trailing action icon.
void _handleTap() {
final VoidCallback? actionHandler = switch (modelInfo.state) {
ModelState.downloaded => selectModel,
ModelState.downloading => null,
ModelState.empty => null,
};
actionHandler?.call();
}

// Responds to taps on the trailing action icon, which should always either
// do nothing or launch a modal.
void _launchModal(BuildContext context) {
final VoidCallback? actionHandler = switch (modelInfo.state) {
ModelState.downloaded => delete,
ModelState.downloading => null,
ModelState.empty => download,
};
if (actionHandler == null) {
return;
}
final String title = switch (modelInfo.state) {
ModelState.downloaded => 'Delete',
ModelState.downloading => '',
ModelState.empty => 'Download',
};

final String? message = switch (modelInfo.state) {
ModelState.downloaded =>
'Would you like to delete this downloaded model?',
ModelState.downloading => null,
ModelState.empty => 'Would you like to download this model?',
};

showAlertDialog<bool>(
context: context,
title: title,
message: message,
actions: <AlertDialogAction<bool>>[
AlertDialogAction<bool>(
key: false,
label: 'Cancel',
isDefaultAction: true,
isDestructiveAction: modelInfo.state == ModelState.downloaded,
),
AlertDialogAction<bool>(
key: true,
label: title,
isDefaultAction: true,
isDestructiveAction: modelInfo.state == ModelState.downloaded,
),
],
).then((result) {
if (result == true) {
actionHandler();
}
});
}
}

class _DownloadingBar extends StatelessWidget {
const _DownloadingBar({required this.downloadPercent});

final int downloadPercent;

@override
Widget build(BuildContext context) {
return GFProgressBar(
percentage: (downloadPercent / 100).toDouble().clamp(0, 1.0),
lineHeight: 8,
backgroundColor: Colors.green[100]!,
progressBarColor: Colors.green[700]!,
);
}
}

final gigaByte = pow(2, 30);
final megaByte = pow(2, 20);

String humanize(int fileSize) {
if (fileSize > gigaByte) {
return '${(fileSize / gigaByte).roundTo(2)} GB';
} else if (fileSize > megaByte) {
return '${(fileSize / megaByte).roundTo(2)} MB';
}
return '$fileSize bytes';
}

extension RoundableDouble on double {
double roundTo(int decimalPlaces) =>
double.parse(toStringAsFixed(decimalPlaces));
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import 'dart:async';
import 'dart:io';

import 'package:example/models/models.dart';
import 'package:path_provider/path_provider.dart';
import 'package:path/path.dart' as path;

import 'model_storage_interface.dart';

class ModelStorage extends ModelStorageInterface {
@override
int binarySize(String location) {
final file = File(location);
if (!file.existsSync()) {
throw Exception('Unexpectedly asked for binary size of non-existent '
'file at $location');
}
return file.lengthSync();
}

@override
Future<bool> downloadExists(String downloadLocation) =>
File(downloadLocation).exists();

@override
Future<String> urlToDownloadDestination(Uri location) async => path.join(
(await _getDownloadFolder()).absolute.path,
location.pathSegments.last,
);

Directory? _downloadFolder;
Future<Directory> _getDownloadFolder() async {
_downloadFolder ??= await getApplicationCacheDirectory();
return Future.value(_downloadFolder);
}

@override
Future<void> abort(String location) async {
if (!_downloadCache.containsKey(location)) {
throw Exception('Abort called for location $location, which is not the '
'site of an ongoing donwload.');
}
final file = File(location);
if (await file.exists()) {
await file.delete();
}
_downloadCache[location]!.close();
_downloadCache.remove(location);
}

@override
Future<void> close(String location) async {
if (!_downloadCache.containsKey(location)) {
throw Exception('Abort called for location $location, which is not the '
'site of an ongoing donwload.');
}
_downloadCache[location]!.close();
_downloadCache.remove(location);
}

final _downloadCache = <String, StreamSink<List<int>>>{};

@override
Future<StreamSink<List<int>>> create(String location) async {
final file = File(location);
if (await file.exists()) {
throw Exception('Attempted to download on top of existing file at '
'$location. Delete that file before proceeding.');
}
_downloadCache[location] = file.openWrite();
return _downloadCache[location]!;
}

@override
Future<void> delete(LlmModel model) async {
final path = pathFor(model);
if (path != null) {
clearPathCache(model);
final file = File(path);
if (await file.exists()) {
await file.delete();
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
export 'model_storage_interface.dart';

export 'universal_model_storage.dart'
if (dart.library.html) 'web_model_storage.dart'
if (dart.library.io) 'io_model_storage.dart';
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
import 'dart:async';

import 'package:example/models/models.dart';
import 'package:logging/logging.dart';

typedef ModelPaths = Map<LlmModel, String>;

final _log = Logger('ModelLocationStorage');

abstract class ModelStorageInterface {
Future<void> setInitialModelLocations(
ModelPaths initialModelLocations,
) async {
for (final (model, location) in initialModelLocations.items) {
_log.finer('${model.name} :: $location');
if (location.isEmpty) continue;
final uri = Uri.parse(location);

late final String downloadDestination;
switch (uri.scheme.startsWith('http')) {
case (true):
{
downloadDestination = await urlToDownloadDestination(uri);
if (await downloadExists(downloadDestination)) {
setPathCache(model, downloadDestination);
}
setUriCache(model, uri);
}
case (false):
{
// If the location provided does not start with "http", it must be
// something immediately accessible to the current runtime.
downloadDestination = location;
if (await downloadExists(downloadDestination)) {
setPathCache(model, downloadDestination);
} else {
_log.warning(
'Bad specification. Model for ${model.name} not found '
'at at $location',
);
}
}
}
}
}

/// Deletes the file at the given location.
Future<void> delete(LlmModel model);

/// Opens the resources to save an [LlmModel] binary and returns a stream to
/// which bytes may be written.
Future<StreamSink<List<int>>> create(String location);

/// Marks a download initiated by [create] as successfully completed. Throws
/// an exception if no such download is in progress.
Future<void> close(String location);

/// Terminates a download and deletes all progress. Throws an exception if no
/// such download is in progress.
Future<void> abort(String location);

/// Determine to where in on-device storage this remote Url should be downloaded.
Future<String> urlToDownloadDestination(Uri location);

/// {@template downloadExists}
/// Returns true if the model at the given file is completely downloaded, or
/// false if it does not exist at all or is still downloading.
/// {@endtemplate}
Future<bool> downloadExists(String downloadLocation);

/// {@template binarySize}
/// Returns the size of the binary at the given on-device location. Throws an
/// exception if the file does not exist, so before calling this method,
/// verify that the model is downloaded with [downloadExists].
/// {@endtemplate}
int binarySize(String location);

/// Returns the on-disk location of a downloaded model.
String? pathFor(LlmModel model) => _pathCache[model];

void clearPathCache(LlmModel model) => _pathCache.remove(model);

/// Returns the remote location of a model.
Uri? urlFor(LlmModel model) => _urlCache[model];

/// On disk locations for the model. This could either have been supplied
/// at compile-time via environment variable, or at runtime due to a web url
/// location (also defined at compile-time) that was since been downloaded.
final _pathCache = <LlmModel, String>{};

/// Off-disk locations for models that could be downloaded to disk. Doing so
/// will creating an entry in [_pathCache] for where the downloaded model
/// resides on disk.
///
/// This is stored (if known) even if the storage system believes the model is
/// already downloaded, because the user could need to delete and redownload
/// said model.
final _urlCache = <LlmModel, Uri>{};

void setUriCache(LlmModel model, Uri location) {
_log.fine('Registered remote ${model.name} location at $location');
_urlCache[model] = location;
}

void setPathCache(LlmModel model, String location) {
_log.fine(
'Detected downloaded binary for ${model.name} of size '
'${binarySize(location)} bytes',
);
_pathCache[model] = location;
}
}

extension ItemsMap<K, V> on Map<K, V> {
Iterable<(K, V)> get items sync* {
for (final K key in keys) {
yield (key, this[key] as V);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import 'dart:async';

import 'package:example/models/models.dart';

import 'model_storage_interface.dart';

class ModelStorage extends ModelStorageInterface {
@override
int binarySize(String location) => throw UnimplementedError();

@override
Future<bool> downloadExists(String downloadLocation) =>
throw UnimplementedError();

@override
Future<String> urlToDownloadDestination(Uri location) async =>
throw UnimplementedError();

@override
Future<void> abort(String location) => throw UnimplementedError();

@override
Future<void> close(String location) => throw UnimplementedError();

@override
Future<void> delete(LlmModel model) => throw UnimplementedError();

@override
Future<StreamSink<List<int>>> create(String location) =>
throw UnimplementedError();
}
94 changes: 94 additions & 0 deletions packages/mediapipe-task-genai/example/lib/models/chat_message.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:flutter/material.dart';
import 'package:freezed_annotation/freezed_annotation.dart';
import 'package:uuid/uuid.dart';

part 'chat_message.freezed.dart';

@Freezed()
class ChatMessage with _$ChatMessage {
const ChatMessage._();
const factory ChatMessage({
required String id,
required String body,
required MessageOrigin origin,
required int cursorPosition,

/// Always true for a user's message, but only true for the LLM once it has
/// finished composing its reply.
required bool isComplete,
}) = _ChatMessage;

factory ChatMessage.origin(String body, MessageOrigin origin) =>
origin == MessageOrigin.user
? ChatMessage.user(body)
: ChatMessage.llm(body);

factory ChatMessage.llm(String body, {int? cursorPosition}) => ChatMessage(
// Sometimes the LLM starts a response with multiple empty newlines
body: body,
origin: MessageOrigin.llm,
cursorPosition: cursorPosition ?? 0,
isComplete: false,
id: const Uuid().v4(),
);

factory ChatMessage.user(String body) => ChatMessage(
body: body,
origin: MessageOrigin.user,
cursorPosition: body.length,
isComplete: true,
id: const Uuid().v4(),
);

ChatMessage complete() {
assert(() {
if (origin.isUser) {
throw Exception(
'Only expected to complete messages from the LLM. '
'Did you call complete() on the wrong ChatMessage?',
);
}
return true;
}());
return copyWith(isComplete: true);
}

bool get displayingFullString => cursorPosition == body.length;

// String get displayString => body.substring(0, cursorPosition);
String get displayString => body;

ChatMessage advanceCursor() => copyWith(cursorPosition: cursorPosition + 1);
}

enum MessageOrigin {
user,
llm;

bool get isUser => switch (this) {
MessageOrigin.user => true,
MessageOrigin.llm => false,
};

bool get isLlm => switch (this) {
MessageOrigin.user => false,
MessageOrigin.llm => true,
};

String get transcriptName => switch (this) {
MessageOrigin.user => 'USER',
MessageOrigin.llm => 'LLM',
};

Alignment alignmentFromTextDirection(TextDirection textDirection) =>
switch (textDirection) {
TextDirection.ltr =>
isUser ? Alignment.centerRight : Alignment.centerLeft,
TextDirection.rtl =>
isUser ? Alignment.centerLeft : Alignment.centerRight,
};
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
// coverage:ignore-file
// GENERATED CODE - DO NOT MODIFY BY HAND
// ignore_for_file: type=lint
// ignore_for_file: unused_element, deprecated_member_use, deprecated_member_use_from_same_package, use_function_type_syntax_for_parameters, unnecessary_const, avoid_init_to_null, invalid_override_different_default_values_named, prefer_expression_function_bodies, annotate_overrides, invalid_annotation_target, unnecessary_question_mark

part of 'chat_message.dart';

// **************************************************************************
// FreezedGenerator
// **************************************************************************

T _$identity<T>(T value) => value;

final _privateConstructorUsedError = UnsupportedError(
'It seems like you constructed your class using `MyClass._()`. This constructor is only meant to be used by freezed and you are not supposed to need it nor use it.\nPlease check the documentation here for more information: https://github.com/rrousselGit/freezed#adding-getters-and-methods-to-our-models');

/// @nodoc
mixin _$ChatMessage {
String get id => throw _privateConstructorUsedError;
String get body => throw _privateConstructorUsedError;
MessageOrigin get origin => throw _privateConstructorUsedError;
int get cursorPosition => throw _privateConstructorUsedError;

/// Always true for a user's message, but only true for the LLM once it has
/// finished composing its reply.
bool get isComplete => throw _privateConstructorUsedError;

@JsonKey(ignore: true)
$ChatMessageCopyWith<ChatMessage> get copyWith =>
throw _privateConstructorUsedError;
}

/// @nodoc
abstract class $ChatMessageCopyWith<$Res> {
factory $ChatMessageCopyWith(
ChatMessage value, $Res Function(ChatMessage) then) =
_$ChatMessageCopyWithImpl<$Res, ChatMessage>;
@useResult
$Res call(
{String id,
String body,
MessageOrigin origin,
int cursorPosition,
bool isComplete});
}

/// @nodoc
class _$ChatMessageCopyWithImpl<$Res, $Val extends ChatMessage>
implements $ChatMessageCopyWith<$Res> {
_$ChatMessageCopyWithImpl(this._value, this._then);

// ignore: unused_field
final $Val _value;
// ignore: unused_field
final $Res Function($Val) _then;

@pragma('vm:prefer-inline')
@override
$Res call({
Object? id = null,
Object? body = null,
Object? origin = null,
Object? cursorPosition = null,
Object? isComplete = null,
}) {
return _then(_value.copyWith(
id: null == id
? _value.id
: id // ignore: cast_nullable_to_non_nullable
as String,
body: null == body
? _value.body
: body // ignore: cast_nullable_to_non_nullable
as String,
origin: null == origin
? _value.origin
: origin // ignore: cast_nullable_to_non_nullable
as MessageOrigin,
cursorPosition: null == cursorPosition
? _value.cursorPosition
: cursorPosition // ignore: cast_nullable_to_non_nullable
as int,
isComplete: null == isComplete
? _value.isComplete
: isComplete // ignore: cast_nullable_to_non_nullable
as bool,
) as $Val);
}
}

/// @nodoc
abstract class _$$ChatMessageImplCopyWith<$Res>
implements $ChatMessageCopyWith<$Res> {
factory _$$ChatMessageImplCopyWith(
_$ChatMessageImpl value, $Res Function(_$ChatMessageImpl) then) =
__$$ChatMessageImplCopyWithImpl<$Res>;
@override
@useResult
$Res call(
{String id,
String body,
MessageOrigin origin,
int cursorPosition,
bool isComplete});
}

/// @nodoc
class __$$ChatMessageImplCopyWithImpl<$Res>
extends _$ChatMessageCopyWithImpl<$Res, _$ChatMessageImpl>
implements _$$ChatMessageImplCopyWith<$Res> {
__$$ChatMessageImplCopyWithImpl(
_$ChatMessageImpl _value, $Res Function(_$ChatMessageImpl) _then)
: super(_value, _then);

@pragma('vm:prefer-inline')
@override
$Res call({
Object? id = null,
Object? body = null,
Object? origin = null,
Object? cursorPosition = null,
Object? isComplete = null,
}) {
return _then(_$ChatMessageImpl(
id: null == id
? _value.id
: id // ignore: cast_nullable_to_non_nullable
as String,
body: null == body
? _value.body
: body // ignore: cast_nullable_to_non_nullable
as String,
origin: null == origin
? _value.origin
: origin // ignore: cast_nullable_to_non_nullable
as MessageOrigin,
cursorPosition: null == cursorPosition
? _value.cursorPosition
: cursorPosition // ignore: cast_nullable_to_non_nullable
as int,
isComplete: null == isComplete
? _value.isComplete
: isComplete // ignore: cast_nullable_to_non_nullable
as bool,
));
}
}

/// @nodoc
class _$ChatMessageImpl extends _ChatMessage {
const _$ChatMessageImpl(
{required this.id,
required this.body,
required this.origin,
required this.cursorPosition,
required this.isComplete})
: super._();

@override
final String id;
@override
final String body;
@override
final MessageOrigin origin;
@override
final int cursorPosition;

/// Always true for a user's message, but only true for the LLM once it has
/// finished composing its reply.
@override
final bool isComplete;

@override
String toString() {
return 'ChatMessage(id: $id, body: $body, origin: $origin, cursorPosition: $cursorPosition, isComplete: $isComplete)';
}

@override
bool operator ==(Object other) {
return identical(this, other) ||
(other.runtimeType == runtimeType &&
other is _$ChatMessageImpl &&
(identical(other.id, id) || other.id == id) &&
(identical(other.body, body) || other.body == body) &&
(identical(other.origin, origin) || other.origin == origin) &&
(identical(other.cursorPosition, cursorPosition) ||
other.cursorPosition == cursorPosition) &&
(identical(other.isComplete, isComplete) ||
other.isComplete == isComplete));
}

@override
int get hashCode =>
Object.hash(runtimeType, id, body, origin, cursorPosition, isComplete);

@JsonKey(ignore: true)
@override
@pragma('vm:prefer-inline')
_$$ChatMessageImplCopyWith<_$ChatMessageImpl> get copyWith =>
__$$ChatMessageImplCopyWithImpl<_$ChatMessageImpl>(this, _$identity);
}

abstract class _ChatMessage extends ChatMessage {
const factory _ChatMessage(
{required final String id,
required final String body,
required final MessageOrigin origin,
required final int cursorPosition,
required final bool isComplete}) = _$ChatMessageImpl;
const _ChatMessage._() : super._();

@override
String get id;
@override
String get body;
@override
MessageOrigin get origin;
@override
int get cursorPosition;
@override

/// Always true for a user's message, but only true for the LLM once it has
/// finished composing its reply.
bool get isComplete;
@override
@JsonKey(ignore: true)
_$$ChatMessageImplCopyWith<_$ChatMessageImpl> get copyWith =>
throw _privateConstructorUsedError;
}
76 changes: 76 additions & 0 deletions packages/mediapipe-task-genai/example/lib/models/llm_model.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:freezed_annotation/freezed_annotation.dart';

part 'llm_model.freezed.dart';

enum Hardware { cpu, gpu }

enum LlmModel {
gemma4bCpu,
gemma4bGpu,
gemma8bCpu,
gemma8bGpu;

Hardware get hardware => switch (this) {
gemma4bCpu => Hardware.cpu,
gemma4bGpu => Hardware.gpu,
gemma8bCpu => Hardware.cpu,
gemma8bGpu => Hardware.gpu,
};

String get dartDefine => switch (this) {
gemma4bCpu => const String.fromEnvironment('GEMMA_4B_CPU_URI'),
gemma4bGpu => const String.fromEnvironment('GEMMA_4B_GPU_URI'),
gemma8bCpu => const String.fromEnvironment('GEMMA_8B_CPU_URI'),
gemma8bGpu => const String.fromEnvironment('GEMMA_8B_GPU_URI'),
};

String get environmentVariableUriName => switch (this) {
gemma4bCpu => 'GEMMA_4B_CPU_URI',
gemma4bGpu => 'GEMMA_4B_GPU_URI',
gemma8bCpu => 'GEMMA_8B_CPU_URI',
gemma8bGpu => 'GEMMA_8B_GPU_URI',
};

String get displayName => switch (this) {
gemma4bCpu => 'Gemma 4b CPU',
gemma4bGpu => 'Gemma 4b GPU',
gemma8bCpu => 'Gemma 8b CPU',
gemma8bGpu => 'Gemma 8b GPU',
};
}

@Freezed()
class ModelInfo with _$ModelInfo {
const ModelInfo._();
const factory ModelInfo({
/// Size of the on-disk location of this model. A null value here either
/// means that the model is currently downloading or completely missing.
int? downloadedBytes,

/// 0-100 if a model is being downloaded. A null value here means no
/// download is in progress for the given model.
int? downloadPercent,

/// Location of the model if it is available on disk.
String? path,

/// Location from which the model can be downloaded if it is not already
/// available.
Uri? remoteLocation,
}) = _ModelInfo;

ModelState get state {
if (downloadedBytes != null) {
return ModelState.downloaded;
} else if (downloadPercent != null) {
return ModelState.downloading;
}
return ModelState.empty;
}
}

enum ModelState { downloaded, downloading, empty }
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
// coverage:ignore-file
// GENERATED CODE - DO NOT MODIFY BY HAND
// ignore_for_file: type=lint
// ignore_for_file: unused_element, deprecated_member_use, deprecated_member_use_from_same_package, use_function_type_syntax_for_parameters, unnecessary_const, avoid_init_to_null, invalid_override_different_default_values_named, prefer_expression_function_bodies, annotate_overrides, invalid_annotation_target, unnecessary_question_mark

part of 'llm_model.dart';

// **************************************************************************
// FreezedGenerator
// **************************************************************************

T _$identity<T>(T value) => value;

final _privateConstructorUsedError = UnsupportedError(
'It seems like you constructed your class using `MyClass._()`. This constructor is only meant to be used by freezed and you are not supposed to need it nor use it.\nPlease check the documentation here for more information: https://github.com/rrousselGit/freezed#adding-getters-and-methods-to-our-models');

/// @nodoc
mixin _$ModelInfo {
/// Size of the on-disk location of this model. A null value here either
/// means that the model is currently downloading or completely missing.
int? get downloadedBytes => throw _privateConstructorUsedError;

/// 0-100 if a model is being downloaded. A null value here means no
/// download is in progress for the given model.
int? get downloadPercent => throw _privateConstructorUsedError;

/// Location of the model if it is available on disk.
String? get path => throw _privateConstructorUsedError;

/// Location from which the model can be downloaded if it is not already
/// available.
Uri? get remoteLocation => throw _privateConstructorUsedError;

@JsonKey(ignore: true)
$ModelInfoCopyWith<ModelInfo> get copyWith =>
throw _privateConstructorUsedError;
}

/// @nodoc
abstract class $ModelInfoCopyWith<$Res> {
factory $ModelInfoCopyWith(ModelInfo value, $Res Function(ModelInfo) then) =
_$ModelInfoCopyWithImpl<$Res, ModelInfo>;
@useResult
$Res call(
{int? downloadedBytes,
int? downloadPercent,
String? path,
Uri? remoteLocation});
}

/// @nodoc
class _$ModelInfoCopyWithImpl<$Res, $Val extends ModelInfo>
implements $ModelInfoCopyWith<$Res> {
_$ModelInfoCopyWithImpl(this._value, this._then);

// ignore: unused_field
final $Val _value;
// ignore: unused_field
final $Res Function($Val) _then;

@pragma('vm:prefer-inline')
@override
$Res call({
Object? downloadedBytes = freezed,
Object? downloadPercent = freezed,
Object? path = freezed,
Object? remoteLocation = freezed,
}) {
return _then(_value.copyWith(
downloadedBytes: freezed == downloadedBytes
? _value.downloadedBytes
: downloadedBytes // ignore: cast_nullable_to_non_nullable
as int?,
downloadPercent: freezed == downloadPercent
? _value.downloadPercent
: downloadPercent // ignore: cast_nullable_to_non_nullable
as int?,
path: freezed == path
? _value.path
: path // ignore: cast_nullable_to_non_nullable
as String?,
remoteLocation: freezed == remoteLocation
? _value.remoteLocation
: remoteLocation // ignore: cast_nullable_to_non_nullable
as Uri?,
) as $Val);
}
}

/// @nodoc
abstract class _$$ModelInfoImplCopyWith<$Res>
implements $ModelInfoCopyWith<$Res> {
factory _$$ModelInfoImplCopyWith(
_$ModelInfoImpl value, $Res Function(_$ModelInfoImpl) then) =
__$$ModelInfoImplCopyWithImpl<$Res>;
@override
@useResult
$Res call(
{int? downloadedBytes,
int? downloadPercent,
String? path,
Uri? remoteLocation});
}

/// @nodoc
class __$$ModelInfoImplCopyWithImpl<$Res>
extends _$ModelInfoCopyWithImpl<$Res, _$ModelInfoImpl>
implements _$$ModelInfoImplCopyWith<$Res> {
__$$ModelInfoImplCopyWithImpl(
_$ModelInfoImpl _value, $Res Function(_$ModelInfoImpl) _then)
: super(_value, _then);

@pragma('vm:prefer-inline')
@override
$Res call({
Object? downloadedBytes = freezed,
Object? downloadPercent = freezed,
Object? path = freezed,
Object? remoteLocation = freezed,
}) {
return _then(_$ModelInfoImpl(
downloadedBytes: freezed == downloadedBytes
? _value.downloadedBytes
: downloadedBytes // ignore: cast_nullable_to_non_nullable
as int?,
downloadPercent: freezed == downloadPercent
? _value.downloadPercent
: downloadPercent // ignore: cast_nullable_to_non_nullable
as int?,
path: freezed == path
? _value.path
: path // ignore: cast_nullable_to_non_nullable
as String?,
remoteLocation: freezed == remoteLocation
? _value.remoteLocation
: remoteLocation // ignore: cast_nullable_to_non_nullable
as Uri?,
));
}
}

/// @nodoc
class _$ModelInfoImpl extends _ModelInfo {
const _$ModelInfoImpl(
{this.downloadedBytes,
this.downloadPercent,
this.path,
this.remoteLocation})
: super._();

/// Size of the on-disk location of this model. A null value here either
/// means that the model is currently downloading or completely missing.
@override
final int? downloadedBytes;

/// 0-100 if a model is being downloaded. A null value here means no
/// download is in progress for the given model.
@override
final int? downloadPercent;

/// Location of the model if it is available on disk.
@override
final String? path;

/// Location from which the model can be downloaded if it is not already
/// available.
@override
final Uri? remoteLocation;

@override
String toString() {
return 'ModelInfo(downloadedBytes: $downloadedBytes, downloadPercent: $downloadPercent, path: $path, remoteLocation: $remoteLocation)';
}

@override
bool operator ==(Object other) {
return identical(this, other) ||
(other.runtimeType == runtimeType &&
other is _$ModelInfoImpl &&
(identical(other.downloadedBytes, downloadedBytes) ||
other.downloadedBytes == downloadedBytes) &&
(identical(other.downloadPercent, downloadPercent) ||
other.downloadPercent == downloadPercent) &&
(identical(other.path, path) || other.path == path) &&
(identical(other.remoteLocation, remoteLocation) ||
other.remoteLocation == remoteLocation));
}

@override
int get hashCode => Object.hash(
runtimeType, downloadedBytes, downloadPercent, path, remoteLocation);

@JsonKey(ignore: true)
@override
@pragma('vm:prefer-inline')
_$$ModelInfoImplCopyWith<_$ModelInfoImpl> get copyWith =>
__$$ModelInfoImplCopyWithImpl<_$ModelInfoImpl>(this, _$identity);
}

abstract class _ModelInfo extends ModelInfo {
const factory _ModelInfo(
{final int? downloadedBytes,
final int? downloadPercent,
final String? path,
final Uri? remoteLocation}) = _$ModelInfoImpl;
const _ModelInfo._() : super._();

@override

/// Size of the on-disk location of this model. A null value here either
/// means that the model is currently downloading or completely missing.
int? get downloadedBytes;
@override

/// 0-100 if a model is being downloaded. A null value here means no
/// download is in progress for the given model.
int? get downloadPercent;
@override

/// Location of the model if it is available on disk.
String? get path;
@override

/// Location from which the model can be downloaded if it is not already
/// available.
Uri? get remoteLocation;
@override
@JsonKey(ignore: true)
_$$ModelInfoImplCopyWith<_$ModelInfoImpl> get copyWith =>
throw _privateConstructorUsedError;
}
2 changes: 2 additions & 0 deletions packages/mediapipe-task-genai/example/lib/models/models.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export 'chat_message.dart';
export 'llm_model.dart';
40 changes: 40 additions & 0 deletions packages/mediapipe-task-genai/example/lib/widgets/chat_input.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import 'package:flutter/material.dart';

class ChatInput extends StatelessWidget {
const ChatInput({
required this.submit,
required this.controller,
required this.isLlmTyping,
super.key,
});

final TextEditingController controller;

final void Function(String) submit;

/// Prevents submitting new messages when true.
final bool isLlmTyping;

@override
Widget build(BuildContext context) {
return Row(
children: [
Expanded(child: TextField(controller: controller)),
ValueListenableBuilder(
valueListenable: controller,
builder: (context, value, child) {
return IconButton(
icon: const Icon(Icons.send),
onPressed: controller.text != '' && !isLlmTyping
? () {
submit(controller.text);
controller.clear();
}
: null,
);
},
),
],
);
}
}
165 changes: 165 additions & 0 deletions packages/mediapipe-task-genai/example/lib/widgets/chat_screen.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:chat_bubbles/chat_bubbles.dart';
import 'package:example/bloc.dart';
import 'package:example/models/models.dart';
import 'package:example/widgets/widgets.dart';
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';

class ChatScreen extends StatefulWidget {
const ChatScreen(this.bloc, {required this.model, super.key});

final TranscriptBloc bloc;
final LlmModel model;

@override
State<ChatScreen> createState() => _ChatScreenState();
}

class _ChatScreenState extends State<ChatScreen> {
final TextEditingController _controller = TextEditingController();

static const initialText = 'Hello, world!';

@override
void initState() {
if (widget.bloc.state.transcript[widget.model]!.isEmpty) {
_controller.text = initialText;
}
super.initState();
}

@override
Widget build(BuildContext context) {
final bloc = widget.bloc;
return BlocBuilder<TranscriptBloc, TranscriptState>(
bloc: bloc,
builder: (context, state) {
return Scaffold(
appBar: AppBar(title: Text(widget.model.displayName)),
endDrawer: Drawer(
child: Padding(
padding: const EdgeInsets.all(12),
child: InferenceConfigurationPanel(
topK: state.topK,
temp: state.temperature,
maxTokens: state.maxTokens,
updateTopK: (val) => bloc.add(UpdateTopK(val)),
updateTemp: (val) => bloc.add(UpdateTemperature(val)),
updateMaxTokens: (val) => bloc.add(UpdateMaxTokens(val)),
),
),
),
body: SafeArea(
child: Padding(
padding: const EdgeInsets.all(8.0),
child: Column(
children: <Widget>[
Expanded(
child: KeyboardHider(
child: ChatTranscript(state.transcript[widget.model]!),
),
),
ChatInput(
controller: _controller,
isLlmTyping: state.isLlmTyping,
submit: (String value) {
bloc.add(
AddMessage(ChatMessage.user(value), widget.model),
);
},
),
],
),
),
),
);
},
);
}
}

class ChatTranscript extends StatefulWidget {
const ChatTranscript(
this.transcript, {
super.key,
});

final List<ChatMessage> transcript;

@override
State<ChatTranscript> createState() => _ChatTranscriptState();
}

class _ChatTranscriptState extends State<ChatTranscript> {
@override
Widget build(BuildContext context) {
return LayoutBuilder(
builder: (context, constraints) {
return ListView.builder(
reverse: true,
itemCount: widget.transcript.length,
itemBuilder: (context, index) {
final messageIndex = widget.transcript.length - index - 1;
final message = widget.transcript[messageIndex];

return Padding(
padding: const EdgeInsets.only(top: 6),
child: message.displayString != ''
? ChatMessageBubble(
message: message,
width: constraints.maxWidth * 0.8,
key: ValueKey('message-${message.id}'),
)
: const TypingIndicator(
showIndicator: true,
),
);
},
);
},
);
}
}

class ChatMessageBubble extends StatelessWidget {
const ChatMessageBubble({
required this.message,
required this.width,
super.key,
});

final ChatMessage message;
final double width;

// Colors.blue[400]
static const llmBgColor = Color(0xFF42A5F5);

// Colors.orange[400]
static const userBgColor = Color(0xFFFFA726);

@override
Widget build(BuildContext context) {
return Row(
children: [
if (message.origin.isUser) Flexible(flex: 2, child: Container()),
Flexible(
flex: 6,
child: BubbleSpecialThree(
text: message.displayString,
color: message.origin.isUser ? userBgColor : llmBgColor,
isSender: message.origin.isUser,
textStyle: const TextStyle(color: Colors.white),
sent: message.isComplete,
delivered: message.isComplete,
seen: message.isComplete,
),
),
if (message.origin.isLlm) Flexible(flex: 2, child: Container()),
],
);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:flutter/material.dart';

/// Contains sliders for each configuration option for option passed to the
/// inference engine.
class InferenceConfigurationPanel extends StatelessWidget {
const InferenceConfigurationPanel({
required this.topK,
required this.temp,
required this.maxTokens,
required this.updateTopK,
required this.updateTemp,
required this.updateMaxTokens,
super.key,
});

/// Top K number of tokens to be sampled from for each decoding step.
final int topK;

/// Handler to update [topK].
final void Function(int) updateTopK;

/// Context size window for the LLM.
final int maxTokens;

/// Handler to update [maxTokens].
final void Function(int) updateMaxTokens;

/// Randomness when decoding the next token.
final double temp;

/// Handler to update [temp].
final void Function(double) updateTemp;

@override
Widget build(BuildContext context) {
return ListView(
children: <Widget>[
Text('Top K', style: Theme.of(context).textTheme.bodyLarge),
Text('Number of tokens to be sampled from for each decoding step.',
style: Theme.of(context).textTheme.bodySmall),
Slider(
value: topK.toDouble(),
min: 1,
max: 100,
divisions: 100,
onChanged: (newTopK) => updateTopK(newTopK.toInt()),
),
Text(
topK.toString(),
style: Theme.of(context)
.textTheme
.bodySmall!
.copyWith(color: Colors.grey),
),
const Divider(),
Text('Temperature', style: Theme.of(context).textTheme.bodyLarge),
Text('Randomness when decoding the next token.',
style: Theme.of(context).textTheme.bodySmall),
Slider(
value: temp,
min: 0,
max: 1,
onChanged: updateTemp,
),
Text(
temp.roundTo(3).toString(),
style: Theme.of(context)
.textTheme
.bodySmall!
.copyWith(color: Colors.grey),
),
const Divider(),
Text('Max Tokens', style: Theme.of(context).textTheme.bodyLarge),
Text(
'Maximum context window for the LLM. Larger windows can tax '
'certain devices.',
style: Theme.of(context).textTheme.bodySmall),
Slider(
value: maxTokens.toDouble(),
min: 512,
max: 8192,
onChanged: (newMaxTokens) => updateMaxTokens(newMaxTokens.toInt()),
),
Text(
maxTokens.toString(),
style: Theme.of(context)
.textTheme
.bodySmall!
.copyWith(color: Colors.grey),
),
const Divider(),
GestureDetector(
onTap: () => Navigator.of(context).pop(),
child: Text(
'Close',
style: Theme.of(context).textTheme.headlineMedium,
),
),
],
);
}
}

extension on double {
double roundTo(int decimalPlaces) =>
double.parse(toStringAsFixed(decimalPlaces));
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// Copyright 2014 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'package:flutter/services.dart';
import 'package:flutter/widgets.dart';

class KeyboardHider extends StatelessWidget {
const KeyboardHider({required this.child, super.key});

final Widget? child;

@override
Widget build(BuildContext context) {
return GestureDetector(
onTap: () {
// Not sure why this one isn't working.
// FocusScope.of(context).unfocus();
SystemChannels.textInput.invokeMethod('TextInput.hide');
},
child: child,
);
}
}
Loading