Search results

如何处理标准大模型数据集 {"messges": [{"role": "assistant", "content": "xxx"}]}

dj:post-tuningissues/PRs about post-tuning scenariosissues/PRs about post-tuning scenarios

questionFurther information is requestedFurther information is requested

Status: Open.

#693 In modelscope/data-juicer;

· Xingyu-Romantic opened on Jun 9, 2025

在执行自己的算子的时候出现RuntimeError: One of the subprocesses has abruptly died during map operation.To debug the error, disable multiprocessing.

questionFurther information is requestedFurther information is requested

Status: Open.

#685 In modelscope/data-juicer;

· stalwart0465 opened on May 28, 2025

运行image_text_similarity_filter以及image_text_matching_filter算子时遇到疑似没有执行compute_stats_single的问题

questionFurther information is requestedFurther information is requested

Status: Open.

#682 In modelscope/data-juicer;

· butterbutterflies opened on May 23, 2025

运行image_deduplication的时候，consider_text的部分如何传参

questionFurther information is requestedFurther information is requested

Status: Open.

#672 In modelscope/data-juicer;

· DietDietDiet opened on May 14, 2025

flagged_words_filter算子的process_batched()输入为何没有做Fields.stats值为list的判断？

questionFurther information is requestedFurther information is requested

Status: Open.

#671 In modelscope/data-juicer;

· spacegrass opened on May 13, 2025

如何实现两个算子的逻辑运算关系

questionFurther information is requestedFurther information is requested

Status: Open.

#667 In modelscope/data-juicer;

· DietDietDiet opened on May 9, 2025

希望data-juicer可以适配python3.11的版本。

questionFurther information is requestedFurther information is requested

Status: Open.

#661 In modelscope/data-juicer;

· HunterLG opened on May 6, 2025

sandbox 的trainModelHook是否支持训练本地模型和自定义模型

dj:coreissues/PRs about the core functions of Data-Juicerissues/PRs about the core functions of Data-Juicer

questionFurther information is requestedFurther information is requested

Status: Open.

#649 In modelscope/data-juicer;

· fengzx99 opened on Apr 22, 2025

后微调工具

dj:post-tuningissues/PRs about post-tuning scenariosissues/PRs about post-tuning scenarios

dj:toolsissues/PRs about specific toolsissues/PRs about specific tools

questionFurther information is requestedFurther information is requested

Status: Open.

#629 In modelscope/data-juicer;

· heningsu opened on Mar 31, 2025

配好环境之后尝试运行python tools/process_data.py --config demos/process_on_ray/configs/demo.yaml，遇到疑似卡住无任何日志输出的情况

questionFurther information is requestedFurther information is requested

Status: Open.

#624 In modelscope/data-juicer;

· butterbutterflies opened on Mar 18, 2025

参考demo, Ray 模式下只是通过多个CPU 读了数据，但是mapper 处理数据的时候只用了一个CPU

dj:distissues/PRs about distributed data processingissues/PRs about distributed data processing

questionFurther information is requestedFurther information is requested

Status: Open.

#623 In modelscope/data-juicer;

· Cheendfdf opened on Mar 17, 2025

算子的使用方法

questionFurther information is requestedFurther information is requested

Status: Open.

#621 In modelscope/data-juicer;

· tiandidatongJLR opened on Mar 15, 2025