[video_player] Adds audio track metadata fetching and audio track selection feature #9925

nateshmbhat · 2025-08-29T08:09:37Z

Description

This PR adds comprehensive audio track retrieval and selection support to the video_player package, enabling developers to access detailed information about available audio tracks and switch between them during playback.

Changes Made

Core Features

Added VideoAudioTrack model with comprehensive metadata fields: id, label, language, isSelected, bitrate, sampleRate, channelCount, codec
Added getAudioTracks() method to retrieve all available audio tracks with real metadata
Added selectAudioTrack() method to switch between audio tracks during playback
Updated VideoPlayerController to expose audio track functionality

Platform Implementations

Android:
- Real metadata extraction using ExoPlayer's getCurrentTracks() API
- Robust track selection using TrackSelectionOverride with proper error handling
- Support for multiple audio formats (AAC, AC3, EAC3, MP3, etc.)
iOS:
- Metadata extraction from AVFoundation using AVAssetTrack for regular videos
- HLS stream support using AVMediaSelectionGroup for adaptive streams
- Proper track selection for both asset tracks and media selection options

Technical Infrastructure

Updated Pigeon interfaces for both Android and iOS with new data structures:
- AudioTrackMessage, ExoPlayerAudioTrackData, AssetAudioTrackData, MediaSelectionAudioTrackData, NativeAudioTrackData
Enhanced platform interface with new methods and data classes
Has native unit tests for both Android and iOS platforms
Created demo screen showcasing audio track functionality with interactive UI

Demo Features

Interactive video player with audio track selection
Real-time metadata display (bitrate, sample rate, channels, codec)
Support for multiple video sources including HLS streams
Visual indicators for currently selected tracks

Related Issues

#59437 - Add audio track metadata support to video_player

Testing

Added native unit tests for both Android and iOS
Tested with various video formats and HLS streams
Verified backward compatibility with existing functionality
Demo screen widget for manual testing and validation

Breaking Changes

None - all changes are additive and backward compatible.

Platform PRs here :

Pre-Review Checklist

…roid

…lity

gemini-code-assist

Code Review

This pull request introduces a valuable feature for fetching and selecting audio tracks in the video_player package. The changes are comprehensive, touching the platform interface, Android (ExoPlayer) and iOS (AVFoundation) implementations, and adding a new demo screen.

My review has identified several critical and high-severity issues that should be addressed:

The new native unit tests for both Android and iOS appear to be broken due to type mismatches and incorrect method calls, and will not compile in their current state.
The Dart implementation code for both Android and iOS contains unsafe null assertions on data coming from the native side, which could lead to runtime crashes.
The Android implementation has been updated to use a milestone version of Gradle, which is not recommended for production packages.
The new audio track selection feature has not been implemented for the web platform, which will lead to UnimplementedError on web.
There is a minor issue in the new example app where context is used after an await without checking if the widget is still mounted.

Addressing these points will significantly improve the robustness and completeness of this new feature.

...deo_player_android/android/src/test/java/io/flutter/plugins/videoplayer/AudioTracksTest.java

packages/video_player/video_player_avfoundation/darwin/RunnerTests/AudioTracksTests.m

packages/video_player/video_player_android/android/gradle/wrapper/gradle-wrapper.properties

packages/video_player/video_player_android/lib/src/android_video_player.dart

packages/video_player/video_player_avfoundation/lib/src/avfoundation_video_player.dart

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

…oPlayer async updates

… fix format builders

…o develop

nateshmbhat · 2025-08-30T12:33:22Z

I updated native andorid tests and ran the tests and verified that all native android tests are passing.

…deoPlayerTests.m

…tracks demo

fix(ios): fixed tests

nateshmbhat · 2025-09-01T07:04:20Z

@stuartmorgan-g @ash2moon can you please comment on this PR and lemme know if there are any concerns ?

…eshmbhat/flutter_packages into 4-oct-video-player-interface-updates

…Player

…ethod

packages/video_player/video_player/lib/video_player.dart

…rface to public API

…oss platforms

…nc method

…ce (#10171) ## Description platform interface pr for #9925 #### Core Features - **Added `VideoAudioTrack` model** with comprehensive metadata fields: `id`, `label`, `language`, `isSelected`, `bitrate`, `sampleRate`, `channelCount`, `codec` - **Added [getAudioTracks()]() method** to retrieve all available audio tracks with real metadata - **Added [selectAudioTrack()]() method** to switch between audio tracks during playback ### Breaking Changes None - all changes are additive and backward compatible. ## Pre-Review Checklist

nateshmbhat · 2025-10-27T16:36:20Z

Any update on this PR needed from my side?

stuartmorgan-g · 2025-10-27T16:46:21Z

@nateshmbhat You can split out a new PR with the platform implementations (and with the overrides removed in favor of requiring the newly published interface package version), and then the rest of the platform implementation reviews can happen there.

LongCatIsLooong · 2025-10-24T23:55:07Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+
+  // Sample video URLs with multiple audio tracks
+  final List<String> _sampleVideos = <String>[
+    'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',


Example:

packages/packages/video_player/video_player_avfoundation/example/lib/main.dart

Lines 183 to 185 in 169d555

_controller = MiniController.network(

'https://flutter.github.io/assets-for-api-docs/assets/videos/bee.mp4',

viewType: widget.viewType,

LongCatIsLooong · 2025-10-24T23:56:19Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+
+  // Sample video URLs with multiple audio tracks
+  final List<String> _sampleVideos = <String>[
+    'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',


Or you can upload one with a proper license to https://github.com/flutter/assets-for-api-docs/tree/main/assets/videos

LongCatIsLooong · 2025-10-27T16:59:07Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+      await controller.initialize();
+
+      // Add listener for video player state changes
+      _controller!.addListener(_onVideoPlayerValueChanged);


nit: here and below, use the local controller to avoid !.

LongCatIsLooong · 2025-10-27T17:14:09Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+  }
+
+  void _onVideoPlayerValueChanged() {
+    if (!mounted || _controller == null) {


uber nit: the mounted check is not needed if this method is unsubscribed at unmount.

LongCatIsLooong · 2025-10-27T17:18:32Z

packages/video_player/video_player/example/lib/audio_tracks_demo.dart

+      return const Center(child: CircularProgressIndicator());
+    }
+
+    if (_error != null) {


Suggested change

if (_error != null) {

if (_error case final String error?) {

so you don't need to use !.

LongCatIsLooong · 2025-10-27T17:49:03Z

packages/video_player/video_player/lib/video_player.dart

+    if (!kIsWeb && Platform.isAndroid) {
+      // Add a small delay to allow ExoPlayer to process the track selection change
+      // This is needed because ExoPlayer's track selection update is asynchronous
+      await Future<void>.delayed(const Duration(milliseconds: 100));


This is still scary even with thorough testing. A future version of exoPlayer can break this hardcoded delay timing (in case the selection takes longer to complete in that version).

I'm not familiar with the API of exoPlayer, but not having an API to tell whether the audio track selection is completed just feels a bit strange.

LongCatIsLooong · 2025-10-27T18:28:42Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+      NSString *displayName = option.displayName;
+
+      NSString *languageCode = nil;
+      if (option.locale) {


nit: the if doesn't seem to be necessary?

LongCatIsLooong · 2025-10-27T18:51:12Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+    return [FVPNativeAudioTrackData makeWithAssetTracks:nil mediaSelectionTracks:nil];
+  }
+
+  AVAsset *asset = currentItem.asset;


The documentation says this returns an empty array when tracks haven't been loaded yet. Can this method be called when the tracks are not yet loaded?

LongCatIsLooong · 2025-10-27T18:56:33Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+          // Only process objects that are clearly Core Media format descriptions
+          // This works for both real CMFormatDescription objects and properly configured mock
+          // objects
+          if ([className hasPrefix:@"CMAudioFormatDescription"] ||


I don't think we should include logic that pertains to testing / mocking.

LongCatIsLooong · 2025-10-27T18:59:33Z

...undation/darwin/video_player_avfoundation/Sources/video_player_avfoundation/FVPVideoPlayer.m

+        [asset mediaSelectionGroupForMediaCharacteristic:AVMediaCharacteristicAudible];
+    if (audioGroup && audioGroup.options.count > 0) {
+      // Parse the track ID to get the index
+      NSString *indexString = [trackId substringFromIndex:[@"media_selection_" length]];


This looks very fragile. Could you find an alternative to map trackID to the track?

nateshmbhat · 2025-10-28T15:07:59Z

@nateshmbhat You can split out a new PR with the platform implementations (and with the overrides removed in favor of requiring the newly published interface package version), and then the rest of the platform implementation reviews can happen there.

@stuartmorgan-g Raised #10313 and #10312

- Added new data classes for audio track management (PlaybackState, AudioTrackMessage, ExoPlayerAudioTrackData, NativeAudioTrackData) - Implemented getAudioTracks() and selectAudioTrack() methods in VideoPlayerInstanceApi - Fixed Map comparison in deepEquals to use containsKey instead of contains - Added message codec handling for new audio track data classes - Updated Android video player implementation to support audio track selection

- Updated video_player_platform_interface dependency from ^6.3.0 to ^6.6.0 across all video_player packages - Removed temporary dependency_overrides for video_player_platform_interface that were used for testing - Reformatted dependency override paths for better readability in remaining overrides

- Reduced sample video URLs from 3 to 2 by removing redundant examples - Changed primary video source to flutter.github.io butterfly video for better reliability - Simplified audio track data construction in Android native code by using direct constructor instead of builder pattern - Fixed code formatting and line wrapping in audio tracks demo UI - Removed unnecessary builder pattern usage in NativeAudioTrackData creation

- Removed fully qualified class names (Messages.ExoPlayerAudioTrackData) in favor of direct class names for better readability - Updated getter method name from getIsSelected() to isSelected() to follow Java boolean getter conventions - Maintained same test coverage and assertions while making code more concise - No functional changes to the test behavior or validation logic

- Upgraded video_player_platform_interface dependency from ^6.4.0 to ^6.6.0 in video_player_web package - Maintains compatibility with web platform support while incorporating latest interface updates

…he code formatting changes: style(video_player): improve code formatting and line wrapping - Fixed line wrapping and indentation in audio_tracks_demo.dart for better readability - Simplified import statements and removed fully qualified class names in AudioTracksTest.java - Improved code formatting and line breaks in avfoundation_video_player files - Adjusted indentation and line breaks in example files to follow consistent style

nateshmbhat added 4 commits August 29, 2025 11:32

feat(video_player): add audio track selection support for iOS and And…

c8b0071

…roid

feat(android): implement audio track selection in video player

4e4dc8c

feat(video_player): Format Entire Codebase

25de26c

feat(video_player): add audio track selection and retrieval functiona…

31f9030

…lity

nateshmbhat requested review from LongCatIsLooong, ash2moon, ditman, hellohuanlin and tarrinneal as code owners August 29, 2025 08:09

github-actions bot added platform-android platform-ios platform-macos platform-web p: video_player labels Aug 29, 2025

nateshmbhat mentioned this pull request Aug 29, 2025

[video_player] Add audio track metadata support (bitrate, sample rate, channels, codec) #9782

Closed

11 tasks

gemini-code-assist bot reviewed Aug 29, 2025

View reviewed changes

nateshmbhat added 6 commits August 29, 2025 14:19

test(video_player): add tests for audio track selection and management

8f711b5

fix(video): address PR review comments

e0f6d65

Merge branch 'main' into develop

fc30013

fix(video_player): add delay after audio track selection to handle Ex…

8a68e76

…oPlayer async updates

test(video_player): update audio tracks test to use ImmutableList and…

894f516

… fix format builders

Merge branch 'develop' of github.com:nateshmbhat/flutter_packages int…

644e08e

…o develop

nateshmbhat added 4 commits August 30, 2025 22:03

refactor(tests): move audio track tests from AudioTracksTests.m to Vi…

fdde6f8

…deoPlayerTests.m

fix(ios,android): fixed test failure cases (linting and warnings)

4291609

style(audio_tracks): improve code style and add type safety in audio …

8dfd8e3

…tracks demo

chore(android): bump compileSdk from 34 to 35 for video player plugin

a892a5e

fix(ios): fixed tests

stuartmorgan-g added triage-ios Should be looked at in iOS triage triage-android Should be looked at in Android triage labels Sep 2, 2025

nateshmbhat added 6 commits October 13, 2025 13:34

Merge branch '4-oct-video-player-interface-updates' of github.com:nat…

c06a83d

…eshmbhat/flutter_packages into 4-oct-video-player-interface-updates

Merge branch 'main' into 4-oct-video-player-interface-updates

466946a

Merge branch '4-oct-video-player-interface-updates' into develop

9f391de

refactor(tests): improve formatting of video player test cases

f85c2bf

fix(android): add logging for audio track selection failures in Video…

5aca457

…Player

docs(video_player): improve documentation for audio track selection m…

56191a0

…ethod

nateshmbhat requested review from ash2moon, ezamagni-tg and tarrinneal October 13, 2025 08:45

stuartmorgan-g reviewed Oct 15, 2025

View reviewed changes

packages/video_player/video_player/lib/video_player.dart Outdated Show resolved Hide resolved

nateshmbhat added 5 commits October 16, 2025 21:15

refactor(video_player): move VideoAudioTrack class from platform inte…

27336d4

…rface to public API

refactor(video): normalize null handling for audio track metadata acr…

f202438

…oss platforms

refactor(video): change isAudioTrackSupportAvailable from async to sy…

3341e2d

…nc method

style(video_player): fix code formatting and indentation

bba24d3

Merge branch 'main' into develop

f89e93b

nateshmbhat requested a review from stuartmorgan-g October 16, 2025 16:48

LongCatIsLooong reviewed Oct 27, 2025

View reviewed changes

This was referenced Oct 28, 2025

feat(android): add audio track selection for ExoPlayer #10312

Open

feat(video): add audio track selection support for AVFoundation #10313

Open

nateshmbhat added 7 commits October 28, 2025 20:53

Merge branch 'main' into develop

d69fb63

chore(deps): update video_player_platform_interface to 6.6.0

00b55ed

- Upgraded video_player_platform_interface dependency from ^6.4.0 to ^6.6.0 in video_player_web package - Maintains compatibility with web platform support while incorporating latest interface updates

	_controller = MiniController.network(
	'https://flutter.github.io/assets-for-api-docs/assets/videos/bee.mp4',
	viewType: widget.viewType,

Uh oh!

[video_player] Adds audio track metadata fetching and audio track selection feature #9925

Are you sure you want to change the base?

[video_player] Adds audio track metadata fetching and audio track selection feature #9925

Uh oh!

Conversation

nateshmbhat commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Made

Core Features

Platform Implementations

Technical Infrastructure

Demo Features

Related Issues

Testing

Breaking Changes

Platform PRs here :

Pre-Review Checklist

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nateshmbhat commented Aug 30, 2025

Uh oh!

nateshmbhat commented Sep 1, 2025

Uh oh!

Uh oh!

nateshmbhat commented Oct 27, 2025

Uh oh!

stuartmorgan-g commented Oct 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nateshmbhat commented Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

nateshmbhat commented Aug 29, 2025 •

edited

Loading

nateshmbhat commented Oct 28, 2025 •

edited

Loading