Skip to content

Latest commit

 

History

History
157 lines (133 loc) · 10.9 KB

SDK_advanced_guide_online.md

File metadata and controls

157 lines (133 loc) · 10.9 KB

Real-time Speech Transcription Service Development Guide

(简体中文|English)

FunASR Real-time Speech Recognition Software Package integrates real-time versions of speech endpoint detection model, speech recognition model, punctuation prediction model, and so on. By using multiple models collaboratively, it can perform real-time speech-to-text conversion, as well as high-precision transcription correction at the end of a sentence, with punctuation included in the output text. It supports multiple concurrent requests. Depending on the user's scenarios, it supports three service modes: real-time speech recognition service (online), non-real-time single-sentence transcription (offline), and real-time and non-real-time integrated collaboration (2pass). The software package provides client libraries in various programming languages such as HTML, Python, C++, Java, and C#, allowing users to use and further develop the software.

TIME INFO IMAGE VERSION IMAGE ID
2024.10.29 The 2pass-offline mode supports the SensevoiceSmal model funasr-runtime-sdk-online-cpu-0.1.12 f5febc5cf13a
2024.09.26 Fix memory leak funasr-runtime-sdk-online-cpu-0.1.11 e51a36c42771
2024.05.15 Adapting to FunASR 1.0 model structure funasr-runtime-sdk-online-cpu-0.1.10 1c2adfcff84d
2024.03.05 docker image supports ARM64 platform, update modelscope funasr-runtime-sdk-online-cpu-0.1.9 4a875e08c7a2
2024.01.25 Optimization of the client-side funasr-runtime-sdk-online-cpu-0.1.7 2aa23805572e
2024.01.03 The 2pass-offline mode supports Ngram language model decoding and WFST hotwords, while also addressing known crash issues and memory leak problems funasr-runtime-sdk-online-cpu-0.1.6 f99925110d27
2023.11.09 fix bug: without online results funasr-runtime-sdk-online-cpu-0.1.5 b16584b6d38b
2023.11.08 supporting server-side loading of hotwords, adaptation to runtime structure changes funasr-runtime-sdk-online-cpu-0.1.4 691974017c38
2023.09.19 supporting hotwords, timestamps, and ITN model in 2pass mode funasr-runtime-sdk-online-cpu-0.1.2 7222c5319bcf
2023.08.11 addressing some known bugs (including server crashes) funasr-runtime-sdk-online-cpu-0.1.1 bdbdd0b27dee
2023.08.07 1.0 released funasr-runtime-sdk-online-cpu-0.1.0 bdbdd0b27dee

Quick Start

Docker install

If you have already installed Docker, ignore this step!

curl -O https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/shell/install_docker.sh;
sudo bash install_docker.sh

If you do not have Docker installed, please refer to Docker Installation

Pull Docker Image

Use the following command to pull and start the FunASR software package docker image:

sudo docker pull registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12
mkdir -p ./funasr-runtime-resources/models
sudo docker run -p 10096:10095 -it --privileged=true -v $PWD/funasr-runtime-resources/models:/workspace/models registry.cn-hangzhou.aliyuncs.com/funasr_repo/funasr:funasr-runtime-sdk-online-cpu-0.1.12

Launching the Server

After Docker is launched, start the funasr-wss-server-2pass service program:

cd FunASR/runtime
nohup bash run_server_2pass.sh \
  --download-model-dir /workspace/models \
  --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
  --model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx  \
  --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx  \
  --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \
  --lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
  --itn-dir thuduj12/fst_itn_zh > log.txt 2>&1 &

# If you want to close ssl，please add：--certfile 0

For a more detailed description of server parameters, please refer to Server Introduction

Client Testing and Usage

Download the client testing tool directory samples:

wget https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/sample/funasr_samples.tar.gz

For illustration, we will use the Python language client, which supports audio formats (.wav, .pcm) and a multi-file list wav.scp input.

python3 funasr_wss_client.py --host "127.0.0.1" --port 10096 --mode 2pass

Client Usage Details

After completing the FunASR service deployment on the server, you can test and use the offline file transcription service by following these steps. Currently, the following programming language client versions are supported:

For more detailed usage, please click on the links above. For more client version support, please refer to WebSocket/GRPC Protocol.

Server Introduction

Use the flollowing script to start the server ：

cd /workspace/FunASR/runtime
nohup bash run_server_2pass.sh \
  --model-dir damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx \
  --online-model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online-onnx \
  --vad-dir damo/speech_fsmn_vad_zh-cn-16k-common-onnx \
  --punc-dir damo/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727-onnx \
  --lm-dir damo/speech_ngram_lm_zh-cn-ai-wesp-fst \
  --itn-dir thuduj12/fst_itn_zh \
  --certfile  ../../../ssl_key/server.crt \
  --keyfile ../../../ssl_key/server.key \
  --hotword ../../hotwords.txt > log.txt 2>&1 &

# If you want to close ssl，please add：--certfile 0
# If you want to deploy the SenseVoiceSmall, timestamp or nn hotword model, please set --model-dir to the corresponding model:
#   iic/SenseVoiceSmall-onnx
#   speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-onnx（timestamp）
#   damo/speech_paraformer-large-contextual_asr_nat-zh-cn-16k-common-vocab8404-onnx（hotword）
# The ASR results “<|zh|><|NEUTRAL|><|Speech|> ” in SenseVoiceSmall-onnx means spoken language、speech emotion、audio event

More details about the script run_server_2pass.sh:

--download-model-dir: Model download address, download models from Modelscope by setting the model ID.
--model-dir: modelscope model ID or local model path.
--online-model-dir modelscope model ID
--vad-dir: modelscope model ID or local model path.
--punc-dir: modelscope model ID or local model path.
--lm-dir modelscope model ID or local model path.
--itn-dir modelscope model ID or local model path.
--port: Port number that the server listens on. Default is 10095.
--decoder-thread-num: The number of thread pools on the server side that can handle concurrent requests.
                      The script will automatically configure parameters decoder-thread-num and io-thread-num based on the server's thread count.
--io-thread-num: Number of IO threads that the server starts.
--model-thread-num: The number of internal threads for each recognition route to control the parallelism of the ONNX model. 
        The default value is 1. It is recommended that decoder-thread-num * model-thread-num equals the total number of threads.
--certfile <string>: SSL certificate file. Default is ../../../ssl_key/server.crt. If you want to close ssl，set 0
--keyfile <string>: SSL key file. Default is ../../../ssl_key/server.key. 
--hotword: Hotword file path, one line for each hotword(e.g.:阿里巴巴 20), if the client provides hot words, then combined with the hot words provided by the client.

Shutting Down the FunASR Service

# Check the PID of the funasr-wss-server-2pass process
ps -x | grep funasr-wss-server-2pass
kill -9 PID

Modifying Models and Other Parameters

To replace the currently used model or other parameters, you need to first shut down the FunASR service, make the necessary modifications to the parameters you want to replace, and then restart the FunASR service. The model should be either an ASR/VAD/PUNC model from ModelScope or a fine-tuned model obtained from ModelScope.

# For example, to replace the ASR model with damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx, use the following parameter setting --model-dir
    --model-dir damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-onnx 
# Set the port number using --port
    --port <port number>
# Set the number of inference threads the server will start using --decoder-thread-num
    --decoder-thread-num <decoder thread num>
# Set the number of IO threads the server will start using --io-thread-num
    --io-thread-num <io thread num>
# Disable SSL certificate
    --certfile 0

After executing the above command, the real-time speech transcription service will be started. If the model is specified as a ModelScope model id, the following models will be automatically downloaded from ModelScope: FSMN-VAD model, Paraformer-lagre online, Paraformer-lagre, CT-Transformer, FST-ITN

If you wish to deploy your fine-tuned model (e.g., 10epoch.pb), you need to manually rename the model to model.pb and replace the original model.pb in ModelScope. Then, specify the path as model_dir.