Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for v3 turbo model #314

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions core/src/test.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
wget https://github.com/thewh1teagle/vibe/releases/download/v0.0.1/ggml-tiny.bin
wget https://github.com/thewh1teagle/vibe/releases/download/v0.0.1/ggml-large-v3-turbo.bin
cargo test --features "vulkan" -- --nocapture
cargo test --release --features "vulkan" -- --nocapture
*/
Expand All @@ -13,7 +13,7 @@ use tracing_test::traced_test;
#[serial]
#[traced_test]
fn test_transcribe() {
let ctx = create_context(&PathBuf::from("../ggml-tiny.bin"), None).unwrap();
let ctx = create_context(&PathBuf::from("../ggml-large-v3-turbo.bin"), None).unwrap();
let options = &TranscribeOptions {
init_prompt: None,
lang: Some("en".into()),
Expand Down
52 changes: 52 additions & 0 deletions core/src/transcribe.rs
Original file line number Diff line number Diff line change
Expand Up @@ -320,3 +320,55 @@ pub fn transcribe(

Ok(transcript)
}

fn setup_v3_turbo_params(options: &TranscribeOptions) -> FullParams {
let mut params = FullParams::new(SamplingStrategy::default());
tracing::debug!("set language to {:?}", options.lang);

if let Some(true) = options.word_timestamps {
params.set_token_timestamps(true);
params.set_split_on_word(true);
params.set_max_len(options.max_sentence_len.unwrap_or(1));
}

if let Some(true) = options.translate {
params.set_translate(true);
}
if options.lang.is_some() {
params.set_language(options.lang.as_deref());
}

params.set_print_special(false);
params.set_print_progress(true);
params.set_print_realtime(false);
params.set_print_timestamps(false);
params.set_suppress_blank(true);
params.set_token_timestamps(true);

if let Some(temperature) = options.temperature {
tracing::debug!("setting temperature to {temperature}");
params.set_temperature(temperature);
}

if let Some(max_text_ctx) = options.max_text_ctx {
tracing::debug!("setting n_max_text_ctx to {}", max_text_ctx);
params.set_n_max_text_ctx(max_text_ctx)
}

// handle args
if let Some(init_prompt) = options.init_prompt.to_owned() {
tracing::debug!("setting init prompt to {init_prompt}");
params.set_initial_prompt(&init_prompt);
}

if let Some(n_threads) = options.n_threads {
tracing::debug!("setting n threads to {n_threads}");
params.set_n_threads(n_threads);
}

// Additional configurations for v3 turbo model
params.set_v3_turbo(true);
params.set_v3_turbo_speedup_factor(8.0);

params
}
10 changes: 8 additions & 2 deletions docs/MODELS.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# 🌟 Vibe Models 🌟
# Vibe Models

Welcome to the Vibe Models page! Here you can find a curated list of suggested models to use with Vibe. To install a model, use the "Magic Setup" link to open it in Vibe, or copy and paste the direct download link in Vibe settings.

## Available Models

### 🌱 Tiny Model
### Tiny Model

A compact and efficient version, suitable for quick tasks and limited-resource environments.

Expand Down Expand Up @@ -45,6 +45,12 @@ Specialized for Hebrew (Ivrit) language data, optimized for high accuracy in Heb
[👉 Magic Setup](https://tinyurl.com/2c3bzj2b)
[🔽 Direct Download](https://huggingface.co/ivrit-ai/whisper-v2-d4-ggml/resolve/main/ggml-ivrit-v2-d4.bin?download=true)

### 🚀 Large Model (v3 Turbo)

The latest v3 turbo model offers an 8x speed increase compared to the current medium model, with enhanced accuracy.

[🔽 Direct Download](https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin?download=true)

Enjoy exploring these models and enhancing your Vibe! 🌐✨

### Want More?
Expand Down
Loading