生成的视频越来越差，越来越失真

[tianxiajianghu]

大佬，生成的视频随着时间的延申，越来越差，越来越失真，是每个chunk视频都会采取上一段视频的最后一帧作为参考图吗，但是发现这个生成的参考图越来越差，导致视频内容变形，原理是什么，应该怎么调整

[Vicky0522]

Errors tend to accumulate as the length of the edited video increases. Given that you are working with over 20 clips, it is likely that the output quality will degrade over such an extended duration.

[tianxiajianghu]

item11_train.txt
这是item11的训练脚本，你看看有什么问题吗？

生成的视频转身衣服开始变化，和正面的衣服不一样了，转身之后就完全变化了，还有伪影

[Vicky0522]

Please exchange these parameters for the two editing cases. You can refer to the parameters used in the inference script on the hugging face page.

[tianxiajianghu]

改好了之后，运动伪影没有了，但是中间的衣服变了，变成短袖了，颜色一致性也不行，不知道为什么结尾又好了

blue_dress.mp4

[Vicky0522]

Did you obtain the results by training the model from scratch, or did you use pre-existing data on the Hugging Face page?

[tianxiajianghu]

Did you obtain the results by training the model from scratch, or did you use pre-existing data on the Hugging Face page?

train the model from scratch.

[Vicky0522]

I am not entirely certain about the exact cause, there may be some degree of randomness influencing the training outcomes. Since the motion "turn back" is particularly challenging, one possible explanation could be overfitting in the trained Motion LoRA for the clips featuring the woman turning back. To investigate further, you can review the validation results to identify any issues in the outputs. If problems are observed, you might consider retraining the Motion LoRA specifically for these clips and selecting a more suitable checkpoint for these cases.

[goo2go]

I am not entirely certain about the exact cause, there may be some degree of randomness influencing the training outcomes. Since the motion "turn back" is particularly challenging, one possible explanation could be overfitting in the trained Motion LoRA for the clips featuring the woman turning back. To investigate further, you can review the validation results to identify any issues in the outputs. If problems are observed, you might consider retraining the Motion LoRA specifically for these clips and selecting a more suitable checkpoint for these cases.

by the way, why the clip data are so large in the pre-existing data on the Hugging Face page, it's too large

[2hiTee]

改好了之后，运动伪影没有了，但是中间的衣服变了，变成短袖了，颜色一致性也不行，不知道为什么结尾又好了

blue_dress.mp4

我猜测可能是给的编辑好的那一帧和背后视角相差太大，SVD模型可能无法把编辑效果spread到背后视角，所以无法编辑

[yoyoNTNU]

我想請問一下，我在嘗試使用use_consistency_attention_control=true並從頭開始訓練的時候，為什麼會出現IndexError: list index out of range，不論是在我的資料或是作者提供的範例都有這個問題

[Vicky0522]

我想請問一下，我在嘗試使用use_consistency_attention_control=true並從頭開始訓練的時候，為什麼會出現IndexError: list index out of range，不論是在我的資料或是作者提供的範例都有這個問題

Which item?

[yoyoNTNU]

我想請問一下，我在嘗試使用use_consistency_attention_control=true並從頭開始訓練的時候，為什麼會出現IndexError: list index out of range，不論是在我的資料或是作者提供的範例都有這個問題

Which item?

item1
this is my log on item1 & my data

06/20/2025 11:55:40 - INFO - i2vedit.train - ***** Running training for motion lora*****
06/20/2025 11:55:40 - INFO - i2vedit.train -   Num examples = 1
06/20/2025 11:55:40 - INFO - i2vedit.train -   Num Epochs = 1
06/20/2025 11:55:40 - INFO - i2vedit.train -   Instantaneous batch size per device = 1
06/20/2025 11:55:40 - INFO - i2vedit.train -   Total train batch size (w. parallel, distributed & accumulation) = 1
06/20/2025 11:55:40 - INFO - i2vedit.train -   Gradient Accumulation steps = 1
06/20/2025 11:55:40 - INFO - i2vedit.train -   Total optimization steps = 1
Steps:   0%|                                                                                                                                                                              | 0/1 [00:00<?, ?it/s]common_modules {}
find_modules {}
consistency_find_modules {'TemporalBasicTransformerBlock': ['attn1'], 'BasicTransformerBlock': []}
controller None consistency_controller <i2vedit.prompt_attention.attention_util.ConsistencyAttentionControl object at 0x0000025318BF9BB0>
Number of attention layer registered for controller: 0
Number of attention layer registered for consistency_controller: 16
Traceback (most recent call last):
  File "D:\project\I2VEdit\main.py", line 552, in <module>
    main(**OmegaConf.load(args.config))
  File "D:\project\I2VEdit\main.py", line 330, in main
    train_motion_lora(
  File "D:\project\I2VEdit\i2vedit\train.py", line 1266, in train_motion_lora
    loss_spatial, loss_temporal, latents, init_noise = finetune_unet(batch, step, mask_spatial_lora=mask_spatial_lora, mask_temporal_lora=mask_temporal_lora)
  File "D:\project\I2VEdit\i2vedit\train.py", line 1203, in finetune_unet
    model_pred = unet(
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\accelerate\utils\operations.py", line 822, in forward
    return model_forward(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\accelerate\utils\operations.py", line 810, in __call__
    return convert_to_fp32(self.model_forward(*args, **kwargs))
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\amp\autocast_mode.py", line 16, in decorate_autocast
    return func(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\diffusers\models\unet_spatio_temporal_condition.py", line 434, in forward
    sample, res_samples = downsample_block(
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\diffusers\models\unet_3d_blocks.py", line 2161, in forward
    hidden_states = attn(
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\diffusers\models\transformer_temporal.py", line 359, in forward
    hidden_states_mix = temporal_block(
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\diffusers\models\attention.py", line 509, in forward
    attn_output = self.attn1(norm_hidden_states, encoder_hidden_states=None)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\cgal\anaconda3\envs\i2vedit\lib\site-packages\diffusers\models\attention_processor.py", line 527, in forward
    return self.processor(
  File "D:\project\I2VEdit\i2vedit\prompt_attention\attention_register.py", line 87, in __call__
    key = self.consistency_controller(
  File "D:\project\I2VEdit\i2vedit\prompt_attention\attention_util.py", line 503, in __call__
    attn = self.forward(attn, is_cross, place_in_unet)
  File "D:\project\I2VEdit\i2vedit\prompt_attention\attention_util.py", line 538, in forward
    attn_base = step_in_store_atten_dict[key][current_pos].to(attn.device, attn.dtype)
IndexError: list index out of range
Steps:   0%|                                                                                                                                                                              | 0/1 [00:00<?, ?it/s]

Maybe the config settings need to be changed?

[yoyoNTNU]

@Vicky0522 When i use the resolution 384x384, I will encounter this problem. Can you explain the relationship about resolution and consistency attention?

[Vicky0522]

I've attempted to reproduce the issues by training from scratch using the item1 config file available in the Hugging Face repository. My training completed successfully, and I haven't observed the reported issues. Can you provide your config file so I can investigate further?

@Vicky0522 When i use the resolution 384x384, I will encounter this problem. Can you explain the relationship about resolution and consistency attention?

[yoyoNTNU]

Sure. This is my config file. My data resolution is square by the way, I'm not sure if that affects the whole process.
If possible, could you please try to edit a square video? Thanks for your time.
In my test, it can success edit square video for single chunk(w/o consistency attention), but multi-chunks will fail.
config_multi_chunks.txt