PaddleX/latest/pipeline_deploy/serving #3524
giscus[bot]
bot
Replies: 50 comments 141 replies
-
|
您好，paddlex --serve --pipeline image_classification这样启动服务之后，关闭服务的命令是什么呀？ |
Beta Was this translation helpful? Give feedback.
-
|
ctrl+c |
Beta Was this translation helpful? Give feedback.
-
|
paddlex --serve --pipeline {产线名称或产线配置文件路径} [{其他命令行选项}] 如果在部署多个产线， 是否要启动多个服务吗？ |
Beta Was this translation helpful? Give feedback.
-
|
您好，需要启动多个服务 |
Beta Was this translation helpful? Give feedback.
-
|
服务化部署支持实时数据的高性能推理？ |
Beta Was this translation helpful? Give feedback.
-
|
暂时没有支持流式返回的服务。不过，对于实时数据，可以考虑多实例+高性能推理增加并发、降低延迟～ |
Beta Was this translation helpful? Give feedback.
-
|
我在测试‘高稳定性服务化部署’，使用通用OCR SDK，本地docker启动成功，client.py测试GRPCInferenceService通过，Metrics Service访问成功，但是HTTPService请求失败，一直报错400 Bad Request。想问一下HTTPService的接口调用文档有没有，请求参数是什么，怎么才能正确访问HTTPService |
Beta Was this translation helpful? Give feedback.
-
|
你好，我们暂时没有提供HTTPService的调用文档，目前只支持gRPC～ |
Beta Was this translation helpful? Give feedback.
-
|
SDK的下载链接都失效了，麻烦修复下吧 |
Beta Was this translation helpful? Give feedback.
-
|
你好，目前这块正在升级中，文档先更新了，最新的3.0.1 SDK预计今天之内上传～ |
Beta Was this translation helpful? Give feedback.
-
您好，想问下此次更新主要是更新了什么内容呢？ |
Beta Was this translation helpful? Give feedback.
-
|
这次主要是bug修复，去除了一些无效的参数，另外PP-ChatOCR产线也补充了一个 |
Beta Was this translation helpful? Give feedback.
-
|
您好 |
Beta Was this translation helpful? Give feedback.
-
|
前者是部署镜像，用于高稳定性服务化部署，内置了相关环境；后者是开发镜像，主要用来开发调试paddlex代码，如果对镜像体积不关心，也可以用在部署（高性能推理、基础服务化部署等）。 |
Beta Was this translation helpful? Give feedback.
-
|
docker部署的，为什么λ localhost ~/PaddleX paddlex --serve --pipeline OCR |
Beta Was this translation helpful? Give feedback.
-
|
可能需要先确认一下本地是否能跑通 |
Beta Was this translation helpful? Give feedback.
-
|
我使用阿里云或者腾讯云的linux系统就可以，同样的命令，但是使用麒麟系统的这个就不太行 是不是不太支持这个版本呀？ |
Beta Was this translation helpful? Give feedback.
-
|
建议可以提一个issue，我们会跟进这个问题 |
Beta Was this translation helpful? Give feedback.
-
|
高稳定性服务化部署如何通过http方式调用 |
Beta Was this translation helpful? Give feedback.
-
|
暂不支持，后续我们会考虑添加HTTP调用示例～ |
Beta Was this translation helpful? Give feedback.
-
|
文档中说到 手动构造http请求 的方式，好像不能用呀，端口都没通。 |
Beta Was this translation helpful? Give feedback.
-
|
请问具体报什么错呢？ |
Beta Was this translation helpful? Give feedback.
-
|
已解决，是我机器端口号冲突了。把--network host改为-p 7090:8000 -p 7091:8001 -p 7092:8002就可以了。 |
Beta Was this translation helpful? Give feedback.
-
|
感谢你的反馈，后续我们将考虑增加端口占用检查，并优化错误提示 |
Beta Was this translation helpful? Give feedback.
-
|
您好，这个SDK又不能下载了 |
Beta Was this translation helpful? Give feedback.
-
|
不好意思，这几天在更新中，预计今天之内会上传。 |
Beta Was this translation helpful? Give feedback.
-
|
您好，想请教下CUDA版本为12.4的话 怎么获取ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.1-gpu的版本 这个只支持CUDA版本为11.8的吧？ |
Beta Was this translation helpful? Give feedback.
-
|
你好，我用paddlex3.0.0rc1-gpu版本能跑成功了 但是OCR识别的结果还是和cpu一样 需要10秒左右；显卡是Tesla T4 请问还有其它调优方式吗？config_gpu.pbtxt配置是这样： |
Beta Was this translation helpful? Give feedback.
-
|
请使用paddlex和paddle框架最新版本吧 3.0.0rc1应该是很早以前的版本～ |
Beta Was this translation helpful? Give feedback.
-
|
之前只是paddlex3.1-gpu会报如下错误：Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
这个问题看起来是因为我们底层使用的pypdfium2库无法正确解析这个PDF，建议可以到pypdfium2库提一个issue～ |
Beta Was this translation helpful? Give feedback.
-
|
This could also be a threading issue. pypdfium2 only supports single-threaded use, otherwise you have to guard it with a mutex. |
Beta Was this translation helpful? Give feedback.
-
|
服务化部署如何开启高性能推理？ |
Beta Was this translation helpful? Give feedback.
-
|
基础服务化部署在启动服务时指定参数 |
Beta Was this translation helpful? Give feedback.
-
|
高性能推理插件已经安装，但是加上这个参数启动时报错，不加这个参数是可以正常启动的。 paddlex --serve --use_hpip --pipeline OCRCreating model: ('PP-LCNet_x1_0_doc_ori', None) |
Beta Was this translation helpful? Give feedback.
-
|
请确认你使用的是最新版本的paddlex～ |
Beta Was this translation helpful? Give feedback.
-
|
执行 1.1 paddlex --install serving 一直报错啊，os 版本 AlmaLinux9.6， 在linux 上你们强烈建议使用 docker安装paddlex ，这里怎么没有使用docekr 安装的paddlex 执行服务部署的教程呀， 请大佬指教， 谢谢！ Using cached future-1.0.0-py3-none-any.whl (491 kB) [notice] A new release of pip is available: 25.0.1 -> 25.1.1 During handling of the above exception, another exception occurred: Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
好的，谢谢大佬， 对于 2.4.2 可以使用grpc的方式吗， 监听的有端口， 是不是按http 的方式传入json就可以呢， 如果前台要出图片流， 或者 base64 字符， grpc 如何调用呢，谢谢大佬指导 |
Beta Was this translation helpful? Give feedback.
-
|
不好意思，我刚贴的文档是之前的版本的……可以参考这里： grpc的话，建议用Python客户端来调用；HTTP的话，可以手动构造请求。 |
Beta Was this translation helpful? Give feedback.
-
|
谢谢大佬指导，我们的业务环境比较特殊， 我们前台app 是安卓自动循环调用相机每帧去识别的，所以有大量的请求发送到server端，而且每次识别要尽量快，既然容器监听的有grpc的端口， 那么这种场景grpc 应该比http 性能要好很多吧，可以指导一下grpc的方式吗？ |
Beta Was this translation helpful? Give feedback.
-
|
这个版本 ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.0.3-gpu \ 我运行正常，但是运行 docker run E0716 02:02:04.653384 7 model_repository_manager.cc:1890] Poll failed for model directory 'ocr': instance group ocr_0 of model ocr has kind KIND_GPU but server does not support GPUs I0716 02:02:04.653444 7 server.cc:549] I0716 02:02:04.653449 7 server.cc:592] I0716 02:02:04.653482 7 tritonserver.cc:1920] I0716 02:02:04.653493 7 server.cc:252] Waiting for in-flight requests to complete. |
Beta Was this translation helpful? Give feedback.
-
|
因为你使用的是CPU镜像～请使用GPU镜像吧（以 |
Beta Was this translation helpful? Give feedback.
-
|
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html#23 调整后的 pipeline_config.yaml pipeline_name: OCR text_type: general use_doc_preprocessor: True SubPipelines: SubModules: |
Beta Was this translation helpful? Give feedback.
-
|
明白了，看起来是因为部分模型使用的后端（比如paddle）不支持这个配置，所以出现了上面的报错，在不手动固定后端的情况下，可能还是需要逐模型配置～ |
Beta Was this translation helpful? Give feedback.
-
|
{ 大佬，ocr识别的话，这个file必须必须是个url地址吗？ |
Beta Was this translation helpful? Give feedback.
-
|
不是，请参考产线使用文档吧，其中有详细的API参数说明。 |
Beta Was this translation helpful? Give feedback.
-
|
启用hpi时，启动服务发生以下错误 Creating model: ('PP-DocLayout_plus-L', None) |
Beta Was this translation helpful? Give feedback.
-
|
如果这个错误不影响正常使用（只是程序卡住打印这些错误），可以忽略～ |
Beta Was this translation helpful? Give feedback.
-
|
对于这种启动服务后还需下载的模型，因为网络原因无法下载怎么办，可以手动挂载吗 I0718 07:56:17.887946 7 grpc_server.cc:4117] Started GRPCInferenceService at 0.0.0.0:8001 The above exception was the direct cause of the following exception: Traceback (most recent call last): The above exception was the direct cause of the following exception: Traceback (most recent call last): |
Beta Was this translation helpful? Give feedback.
-
|
再问一下，字体文件又该挂载到哪个位置？ |
Beta Was this translation helpful? Give feedback.
-
|
解决了 |
Beta Was this translation helpful? Give feedback.
-
|
2.4.2 手动构造 HTTP 请求，构造的请求体格式不对，"data"中的json应该是String类型。 |
Beta Was this translation helpful? Give feedback.
-
|
pipeline_name: OCR text_type: general use_doc_preprocessor: True SubPipelines: SubModules: Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]} pipeline_config.yaml配置修改下模型为PP-OCRv5_server_det, PP-OCRv5_server_rec 启动报以上错误 原来使用的mobile模型， 我就修改了这个地方，起订报错了，这两个模型不可用吗？ |
Beta Was this translation helpful? Give feedback.
-
|
NFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine... [INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine... 字后还有两个报错，ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 4: Could not find any implementation for node Conv.140 + BatchNormalization.86 + Relu.59 due to insufficient workspace. 这个是空间不足吗？ |
Beta Was this translation helpful? Give feedback.
-
|
这些错误如果不影响正常直行的话可以忽略～ |
Beta Was this translation helpful? Give feedback.
-
|
我这边调用报错了，没法调用 如果ocr mobile5模型有些内容的话识别不出来，换成ocr server5的话，识别精度会不会好点呢 |
Beta Was this translation helpful? Give feedback.
-
|
请问是在PaddleX官方镜像中安装和使用高性能推理插件的吗？ server模型的识别精度更高，不过可能需要消耗更多的计算资源。 |
Beta Was this translation helpful? Give feedback.
-
|
请问有对应的 api 文档吗？ |
Beta Was this translation helpful? Give feedback.
-
|
如果是http调用，可以参数： “2.4.2 手动构造 HTTP 请求” 这一节说明 |
Beta Was this translation helpful? Give feedback.
-
|
CPU 模式 请求后 返回响应码404 请问是API的请求有改动了吗 |
Beta Was this translation helpful? Give feedback.
-
|
忽略我的评论 |
Beta Was this translation helpful? Give feedback.
-
|
一言难尽的文档，要啥没啥。 |
Beta Was this translation helpful? Give feedback.
-
|
我在使用高稳定性服务化部署（docker）时，高并发请求访问调用grpc接口，约会有10%左右的请求返回空结果，但是实际上这些图片中有文字内容。当我将这些图片再次调用grpc接口进行计算时，其中又有一部分可以正常输出结果。我检查了docker容器日志，发现会偶发性产生如下错误： [ ERROR] [2025-11-03 06:21:12,130] [60098771ab20491380763bb02c35a93b] [b70b0af0-16c0-4e41-a52c-53d81ae7b4ea] - Unhandled exception
Traceback (most recent call last):
File "/paddlex/py310/lib/python3.10/site-packages/paddlex_hps_server/base_model.py", line 88, in execute
result_or_output = self.run(input_, log_id)
File "/paddlex/var/paddlex_model_repo/ocr/1/model.py", line 80, in run
images, data_info = utils.file_to_images(
File "/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/serving/infra/utils.py", line 252, in file_to_images
data_info = get_image_info(images[0])
File "/paddlex/py310/lib/python3.10/site-packages/paddlex/inference/serving/infra/utils.py", line 261, in get_image_info
return ImageInfo(width=image.shape[1], height=image.shape[0])
AttributeError: 'NoneType' object has no attribute 'shape'想请问这种情况是什么原因产生，以及如何解决？谢谢！ |
Beta Was this translation helpful? Give feedback.
-
|
请问后续有跟进如何处理应对这种情况吗？ @tatocode (也是偶现这种情况，里面的 paddlex_ps_OCR_sdk/server/server.sh 启动的 tritonserver 服务出现问题） |
Beta Was this translation helpful? Give feedback.
-
|
能不能提供详细的http请求文档，想要用本机图片测试路径怎么设置，以及batch推理是否支持，该怎么传递图片 |
Beta Was this translation helpful? Give feedback.
-
|
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/paddlex:paddlex3.3.4-paddlepaddle3.2.0-gpu-cuda12.9-cudnn9.9这个镜像太大了，接近60G，有没有小一点的，适合部署的，我只需要OCR相关产线 |
Beta Was this translation helpful? Give feedback.
-
|
大佬，请问一下，你是在哪里可以查看paddlex有哪些镜像及版本？ |
Beta Was this translation helpful? Give feedback.
-
|
您好，我今天测试了paddlex_hps_PaddleOCR-VL_sdk，其中使用client.py测试一个pdf文件，该pdf文件多达几十页，运行client.py后，报错，报错信息为 请问如何解决 |
Beta Was this translation helpful? Give feedback.
-
|
高稳定性服务化部署在triton，但是OCRVL模型不支持加速好像，那么用什么最快 用原始的VLLm方式最快吗 |
Beta Was this translation helpful? Give feedback.
-
|
高稳定性部署会自动转化为onnx的吗 |
Beta Was this translation helpful? Give feedback.
-
|
我部署PP-chatocrV4，里面的server中的pipeline_config.yaml的chat-botllm怎么修改成其他的api呢，还是只能用百度智能云中的api |
Beta Was this translation helpful? Give feedback.
-
|
已经部署完毕了 curl -s -X POST http://localhost:8000/v2/models/ocr/infer -H 'Content-Type: application/json' -d "{ |
Beta Was this translation helpful? Give feedback.
-
|
提示是这个 {"error":"failed to parse the request JSON buffer: Missing a name for object member. at 4"} |
Beta Was this translation helpful? Give feedback.
-
|
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.3-cpu 这个镜像进入到容器里 咋啥也没有呢？paddlex, paddle都没有安装么？只有一个python？ |
Beta Was this translation helpful? Give feedback.
-
|
有没有把高稳定性服务化部署完成了的，分享一下咋搞呢？ 然后就报错了，Failed to initialize Python stub: ModuleNotFoundError: No module named 'numpy' |
Beta Was this translation helpful? Give feedback.
-
|
请问你确定c4a02c2d3ac3是ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlex/hps:paddlex3.3-cpu的image ID嘛？我刚刚试了下，这个镜像里是有Python、PaddleX和Paddle的 
|
Beta Was this translation helpful? Give feedback.
-
|
我搞定了，应该是启动命令写的有问题。应该是-v的目录有问题 |
Beta Was this translation helpful? Give feedback.
-
|
您好，我在910B上使用基础服务部署的，命令paddlex --serve --pipeline ocr 为什么发送请求每张新的图片显存就会增长，但是跑完显存不会释放。有没有解决办法？ |
Beta Was this translation helpful? Give feedback.
-
|
我想问下，可以在paddleOCR-VL产线的ymal配置文件中加入印章识别模块吗？我尝试了将ppstructure的中配置印章模块复制过去，模型路径也做了修改。但是返回结果一直是图片格式，没有返回印章文本。 |
Beta Was this translation helpful? Give feedback.
-
|
在启动容器后，转TensorRT模型，出现这个错误,Error Code 3: API Usage Error,容器会一直循环下载转换模型这一过程。这是什么错误啊。显存还有20多G呢 Creating model: ('PP-LCNet_x1_0_doc_ori', None)
Using official model (PP-LCNet_x1_0_doc_ori), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Downloading [README.md]: 100%|██████████| 6.67k/6.67k [00:00<00:00, 15.0kB/s]
Downloading [inference.json]: 100%|██████████| 102k/102k [00:00<00:00, 226kB/s]
Downloading [config.json]: 100%|██████████| 2.50k/2.50k [00:00<00:00, 3.08kB/s]
Downloading [inference.yml]: 100%|██████████| 766/766 [00:00<00:00, 1.04kB/s]
Downloading [inference.pdiparams]: 100%|██████████| 6.44M/6.44M [00:02<00:00, 2.34MB/s]
Processing 5 items: 100%|██████████| 5.00/5.00 [00:03<00:00, 1.34it/s]<00:00, 1.94MB/s]
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvcaffe_parser.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvcaffe_parser.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer_plugin.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer_plugin.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvinfer.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvonnxparser.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvonnxparser.so.
INFO:root:Create a symbolic link pointing to /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvparsers.so.8 named /paddlex/py310/lib/python3.10/site-packages/ultra_infer/libs/third_libs/tensorrt/lib/libnvparsers.so.
Automatically converting PaddlePaddle model to ONNX format
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(567)::BuildTrtEngine [TrtBackend] Use FP16 to inference.
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine...
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(659)::BuildTrtEngine TensorRT Engine is built successfully.
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(661)::BuildTrtEngine Serialize TensorRTEngine to local file /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt.
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(672)::BuildTrtEngine TensorRTEngine is serialized to local file /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt, we can load this model from the serialized engine directly next time.
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU. |
Beta Was this translation helpful? Give feedback.
-
|
如果不影响最终结果的话，这些ERROR信息可以忽略 |
Beta Was this translation helpful? Give feedback.
-
|
有影响，日志显示已经有服务端口了，使用 服务端会去重复下载转换模型 E0128 00:48:28.966493 19 python.cc:1944] Stub process is unhealthy and it will be restarted.
[ INFO] [2026-01-28 00:48:35,433] [*] [*] - Triton model config: {'name': 'ocr', 'platform': '', 'backend': 'python', 'version_policy': {'latest': {'num_versions': 1}}, 'max_batch_size': 1, 'input': [{'name': 'input', 'data_type': 'TYPE_STRING', 'format': 'FORMAT_NONE', 'dims': [1], 'is_shape_tensor': False, 'allow_ragged_batch': False}], 'output': [{'name': 'output', 'data_type': 'TYPE_STRING', 'dims': [1], 'label_filename': '', 'is_shape_tensor': False}], 'batch_input': [], 'batch_output': [], 'optimization': {'priority': 'PRIORITY_DEFAULT', 'input_pinned_memory': {'enable': True}, 'output_pinned_memory': {'enable': True}, 'gather_kernel_buffer_threshold': 0, 'eager_batching': False}, 'instance_group': [{'name': 'ocr_0', 'kind': 'KIND_GPU', 'count': 3, 'gpus': [2], 'secondary_devices': [], 'profile': [], 'passive': False, 'host_policy': ''}], 'default_model_filename': '', 'cc_model_filenames': {}, 'metric_tags': {}, 'parameters': {}, 'model_warmup': []}
[ INFO] [2026-01-28 00:48:35,434] [*] [*] - Input names: ['input']
[ INFO] [2026-01-28 00:48:35,434] [*] [*] - Output names: ['output']
Creating model: ('PP-LCNet_x1_0_doc_ori', None)
Using official model (PP-LCNet_x1_0_doc_ori), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 8.46it/s]
TensorRT dynamic shapes will be loaded from the file.
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 224, 224], [1, 3, 224, 224], [8, 3, 224, 224]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(719)::CreateTrtEngineFromOnnx Detect serialized TensorRT Engine file in /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt, will load it directly.
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [8, 3, 224, 224], The shape range before: min_shape=[-1, 3, 224, 224], max_shape=[-1, 3, 224, 224].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[8, 3, 224, 224], max_shape=[8, 3, 224, 224].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [1, 3, 224, 224], The shape range before: min_shape=[8, 3, 224, 224], max_shape=[8, 3, 224, 224].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[1, 3, 224, 224], max_shape=[8, 3, 224, 224].
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(108)::LoadTrtCache Build TensorRT Engine from cache file: /root/.paddlex/official_models/PP-LCNet_x1_0_doc_ori/.cache/tensorrt/trt_serialized.trt with shape range information as below,
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(111)::LoadTrtCache Input name: x, shape=[-1, 3, 224, 224], min=[1, 3, 224, 224], max=[8, 3, 224, 224]
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
Creating model: ('UVDoc', None)
Using official model (UVDoc), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 8.91it/s]
The Paddle Inference backend is selected with the default configuration. This may not provide optimal performance.
Using Paddle Inference backend
Paddle predictor option: device_type: gpu, device_id: 2, run_mode: paddle, trt_dynamic_shapes: {'img': [[1, 3, 128, 64], [1, 3, 256, 128], [8, 3, 512, 256]]}, cpu_threads: 10, delete_pass: [], enable_new_ir: True, enable_cinn: False, trt_cfg_setting: {}, trt_use_dynamic_shapes: True, trt_collect_shape_range_info: True, trt_discard_cached_shape_range_info: False, trt_dynamic_shape_input_data: None, trt_shape_range_info_path: None, trt_allow_rebuild_at_runtime: True, mkldnn_cache_capacity: 10
Creating model: ('PP-LCNet_x1_0_textline_ori', None)
Using official model (PP-LCNet_x1_0_textline_ori), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 6.42it/s]
TensorRT dynamic shapes will be loaded from the file.
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 80, 160], [1, 3, 80, 160], [8, 3, 80, 160]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(719)::CreateTrtEngineFromOnnx Detect serialized TensorRT Engine file in /root/.paddlex/official_models/PP-LCNet_x1_0_textline_ori/.cache/tensorrt/trt_serialized.trt, will load it directly.
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [8, 3, 80, 160], The shape range before: min_shape=[-1, 3, 80, 160], max_shape=[-1, 3, 80, 160].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[8, 3, 80, 160], max_shape=[8, 3, 80, 160].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [1, 3, 80, 160], The shape range before: min_shape=[8, 3, 80, 160], max_shape=[8, 3, 80, 160].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[1, 3, 80, 160], max_shape=[8, 3, 80, 160].
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(108)::LoadTrtCache Build TensorRT Engine from cache file: /root/.paddlex/official_models/PP-LCNet_x1_0_textline_ori/.cache/tensorrt/trt_serialized.trt with shape range information as below,
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(111)::LoadTrtCache Input name: x, shape=[-1, 3, 80, 160], min=[1, 3, 80, 160], max=[8, 3, 80, 160]
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
Creating model: ('PP-OCRv5_server_det', None)
Using official model (PP-OCRv5_server_det), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 8.15it/s]
Inference backend: tensorrt
Inference backend config: precision='fp32' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 32, 32], [1, 3, 736, 736], [1, 3, 4000, 4000]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(572)::BuildTrtEngine Start to building TensorRT Engine...
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 1: No Myelin Error exists
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 1: [runnerBuilderBase.cpp::buildAndSerializeMyelinGraph::326] Error Code 1: Myelin (No Myelin Error exists)
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(636)::BuildTrtEngine Failed to call buildSerializedNetwork().
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(752)::CreateTrtEngineFromOnnx Failed to build tensorrt engine.
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
Creating model: ('PP-OCRv5_server_rec', None)
Using official model (PP-OCRv5_server_rec), the model files will be automatically downloaded and saved in /root/.paddlex/official_models.
Processing 5 items: 100%|██████████| 5.00/5.00 [00:00<00:00, 6.34it/s]
TensorRT dynamic shapes will be loaded from the file.
Inference backend: tensorrt
Inference backend config: precision='fp16' use_dynamic_shapes=True dynamic_shapes={'x': [[1, 3, 48, 160], [1, 3, 48, 320], [8, 3, 48, 3200]]}
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(719)::CreateTrtEngineFromOnnx Detect serialized TensorRT Engine file in /root/.paddlex/official_models/PP-OCRv5_server_rec/.cache/tensorrt/trt_serialized.trt, will load it directly.
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [8, 3, 48, 3200], The shape range before: min_shape=[-1, 3, 48, -1], max_shape=[-1, 3, 48, -1].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[8, 3, 48, 3200], max_shape=[8, 3, 48, 3200].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(40)::Update [New Shape Out of Range] input name: x, shape: [1, 3, 48, 160], The shape range before: min_shape=[8, 3, 48, 3200], max_shape=[8, 3, 48, 3200].
[WARNING] ultra_infer/runtime/backends/tensorrt/utils.cc(52)::Update [New Shape Out of Range] The updated shape range now: min_shape=[1, 3, 48, 160], max_shape=[8, 3, 48, 3200].
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(108)::LoadTrtCache Build TensorRT Engine from cache file: /root/.paddlex/official_models/PP-OCRv5_server_rec/.cache/tensorrt/trt_serialized.trt with shape range information as below,
[INFO] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(111)::LoadTrtCache Input name: x, shape=[-1, 3, 48, -1], min=[1, 3, 48, 160], max=[8, 3, 48, 3200]
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(239)::log 3: [runtime.cpp::~Runtime::346] Error Code 3: API Usage Error (Parameter check failed at: runtime/rt/runtime.cpp::~Runtime::346, condition: mEngineCounter.use_count() == 1. Destroying a runtime before destroying deserialized engines created by the runtime leads to undefined behavior.
)
[INFO] ultra_infer/runtime/runtime.cc(320)::CreateTrtBackend Runtime initialized with Backend::TRT in Device::GPU.
[ INFO] [2026-01-28 00:52:34,475] [*] [*] - 79c490ea18004a21ba751b9a3ad1ff21 initialized successfully
[ERROR] ultra_infer/runtime/backends/tensorrt/trt_backend.cc(445)::SetInputs TRTBackend SetInputs not find name:x
--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
No stack trace in paddle, may be caused by external reasons.
----------------------
Error Message Summary:
----------------------
FatalError: `Process abort signal` is detected by the operating system.
[TimeInfo: *** Aborted at 1769563483 (unix time) try "date -d @1769563483" if you are using GNU date ***]
[SignalInfo: *** SIGABRT (@0x165c) received by PID 5724 (TID 0x7fcf78777000) from PID 5724 ***] |
Beta Was this translation helpful? Give feedback.
-
|
有可能是trt引擎构建失败导致的，可以试试给OCRv5_server_rec更换一个后端 |
Beta Was this translation helpful? Give feedback.
and others
Uh oh!
There was an error while loading. Please reload this page.
-
PaddleX/latest/pipeline_deploy/serving
https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/serving.html
Beta Was this translation helpful? Give feedback.
All reactions