Use ffmpeg library in C/C++/Rust and compile to Wasm32-wasi target.

Rust Examples

Rust can use easily use ffmpeg library using ffmpeg-next crate.

Requirements:

• wasi-sysroot
• libclang_rt.builtins-wasm32.a ( because rust compiler-rt missing some f128 functions)
• ffmpeg static library (wasm32-wasi target)
• rust (with wasm32-wasi target)

1. wasi-sysroot and libclang_rt.builtins-wasm32.a is bundled in wasi-sdk, or download these separately from wasi-sdk release page
2. now stable rustc 1.68.2 (llvm 15.0.6) cannot use the wasi-sdk-20 (llvm 16.0.0), so the static library is built by wasi-sdk-19

Build Examples:

1. Config the wasi-sysroot and ffmpeg path in rust-examples/.cargo/config.toml

[env]
# ffmpeg install path (default is ```../install```)
FFMPEG_DIR = { value = "../install", relative = true }
# config wasi-sysroot path (default is ```/opt/wasi-sdk/share/wasi-sysroot```)
BINDGEN_EXTRA_CLANG_ARGS = "--sysroot=/opt/wasi-sdk/share/wasi-sysroot --target=wasm32-wasi -fvisibility=default"

[build]
# default build target
target = "wasm32-wasi"
# config clang_rt.builtins-wasm32 path (default is ```/opt/wasi-sdk/lib/clang/15.0.7/lib/wasi/```)
# config wasi-sysroot path (default is ```/opt/wasi-sdk/share/wasi-sysroot```)
rustflags = [
    "-L", "/opt/wasi-sdk/lib/clang/15.0.7/lib/wasi/",
    "-l", "clang_rt.builtins-wasm32",
    "-L", "/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi",
    "-l", "wasi-emulated-process-clocks",
]

2. Build using the cargo

cd rust-examples
cargo build --release

3. Run the examples using the WasmEdge/Wasmer/Wasmtime

# use wasmedge
wasmedge --dir .:. ./target/wasm32-wasi/release/codec-info.wasm aac
# use wasmer
wasmer run ./target/wasm32-wasi/release/codec-info.wasm aac
# use wasmtime
wasmtime --dir . ./target/wasm32-wasi/release/codec-info.wasm aac

C Examples

Requirements:

• examples source code in ffmpeg repo
• wasi-sdk
• ffmpeg static library (wasm32-wasi target)

Build script: build_ffmpeg_examples.sh

The ffmpeg C examples which can be built:

• avio_http_serve_files
• avio_list_dir
• avio_read_callback
• decode_audio
• decode_filter_audio
• decode_filter_video
• decode_video
• demux_decode
• encode_audio
• encode_video
• extract_mvs
• hw_decode
• mux
• remux
• resample_audio
• scale_video
• show_metadata
• transcode
• transcode_aac

Details

There are 2 patches:

1. dup: WASI has no dup, so just return an error now.
2. tempnam: WASI has no temp directory support, so just return an error now.

The binary programs:

• ffmpeg (wasm32-wasi-threads target only)
• ffplay (no sdl support now)
• ffprobe

The static libraries:

• libavcodec.a
• libavfilter.a
• libavutil.a (with patch, cannot use temp file)
• libswscale.a
• libavdevice.a
• libavformat.a (with patch, cannot use dup)
• libswresample.a

Build ffmpeg from source

Requirement: wasi-sdk

The script build_ffmpeg.sh can compile ffmpeg source to wasm32-wasi.

Use the ffmpeg binary programs:

Now ffmpeg.wasm can only run in wasmtime:

wasmtime --wasm-features all --wasi-modules wasi-common,experimental-wasi-threads --dir . ./ffmpeg.wasm -- --help

Test Data

download from: https://github.com/ffmpegwasm/testdata.git

Reference

• https://github.com/yamt/FFmpeg-WASI

When I searched how to use the new feature of wasi-sdk-20, I found this repo: https://github.com/yamt/FFmpeg-WASI, which has already compiled the ffmpeg to wasm32-wasi and support thread.