Closed
Description
Describe the bug
Log files generated doesn't include all lines shown on the console
To Reproduce
I testing this on vista3d and whole body bundle. Run a training job see output on the console showing lines as
2024-09-14 03:19:12,752 - INFO - Epoch: 881/900, Iter: 24/25 -- train_loss: 0.4078
2024-09-14 03:19:13,500 - INFO - Epoch: 881/900, Iter: 25/25 -- train_loss: 0.4699
2024-09-14 03:19:13,500 - ignite.engine.engine.SupervisedTrainer - INFO - Epoch[881] Complete. Time taken: 00:00:19.345
2024-09-14 03:19:14,636 - INFO - Epoch: 882/900, Iter: 1/25 -- train_loss: 0.3655
2024-09-14 03:19:15,386 - INFO - Epoch: 882/900, Iter: 2/25 -- train_loss: 0.3317
2024-09-14 03:19:16,135 - INFO - Epoch: 882/900, Iter: 3/25 -- train_loss: 0.3545
2024-09-14 03:19:16,884 - INFO - Epoch: 882/900, Iter: 4/25 -- train_loss: 0.2083
2024-09-14 03:19:17,633 - INFO - Epoch: 882/900, Iter: 5/25 -- train_loss: 0.2773
....
2024-09-14 03:19:31,124 - INFO - Epoch: 882/900, Iter: 23/25 -- train_loss: 0.2539
2024-09-14 03:19:31,874 - INFO - Epoch: 882/900, Iter: 24/25 -- train_loss: 0.1814
2024-09-14 03:19:32,622 - INFO - Epoch: 882/900, Iter: 25/25 -- train_loss: 0.2402
2024-09-14 03:19:32,622 - ignite.engine.engine.SupervisedTrainer - INFO - Epoch[882] Complete. Time taken: 00:00:19.122
2024-09-14 03:19:33,923 - INFO - Epoch: 883/900, Iter: 1/25 -- train_loss: 0.2866
however the log file will show only
2024-09-14 03:19:13,500 - ignite.engine.engine.SupervisedTrainer - INFO - Epoch[881] Complete. Time taken: 00:00:19.345
2024-09-14 03:19:32,622 - ignite.engine.engine.SupervisedTrainer - INFO - Epoch[882] Complete. Time taken: 00:00:19.122
Expected behavior
The file log should show exactly the same as what is on the console.
Additional context
I have tried changing the loggin.config file but it seems the monai bundle is loading logging config from else where.
Metadata
Metadata
Assignees
Labels
No labels
Activity
Nic-Ma commentedon Oct 11, 2024
Hi @AHarouni ,
May I know how do you collect the
log
in file? Do you have a logging config, maybe the logging level is not correct?Have you seen this log message in your side:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/bundle/workflows.py#L319
Thanks.
KumoLiu commentedon Oct 11, 2024
Hi @AHarouni, @Nic-Ma, try to fix the issue in Project-MONAI/MONAI#8142
After the pr merge, users can simply add a
FileHandler
in thelogging.conf
to log all the things in the console.Enable redirection of all loggers by configuring a FileHandler within…
KumoLiu commentedon Oct 29, 2024
To enable logging output from MONAI, users can modify the logging.conf file as follows:
Add a new logger named monai and attach the filehandler to it, set propagate=0 to prevent duplicate logging..
cc @AHarouni