Questions about replicating the results of colight's experiment

[Clown1206]

I conducted experiments using colight methods on a hangzhou_4x4 network，and the final results were as follows:
2023-12-25 02:09:05 (INFO): Final Travel Time is 343.5820, mean rewards: -21.5031, queue: 1.2889, delay: 0.0717, throughput: 2732.
The environment of my experimental system is: win10, sumo Version 1.16.0.
There are many differences between my experimental results and those in the paper. Do cityflow and sumo perform experiments with similar results with the same data and the same hyperparameter settings?

[derekmei233]

Hi, thanks for the report. SUMO has more fluctuated results compared to cityflow. Did you change the default setting of the implementation of Colight? These methods are sensitive to hyperparameter changing and to achieve the same result, the minimum epochs should be at least 100 and 200 for safe. Could you help me report the hyperparameters in your colight.yaml file so I can try to reproduce it?

[Clown1206]

colight.yml:includes:

• configs/tsc/base.yml

model:
name: colight
train_model: True
graphic: True
vehicle_max: 1
epsilon: 0.8
epsilon_decay: 0.9995
epsilon_min: 0.01
NEIGHBOR_NUM: 4
NEIGHBOR_EDGE_NUM: 4
N_LAYERS: 1
INPUT_DIM: [128, 128]
OUTPUT_DIM: [128, 128]
NODE_EMB_DIM: [128, 128]
NUM_HEADS: [5, 5]
NODE_LAYER_DIMS_EACH_HEAD: [16, 16]
OUTPUT_LAYERS: []

one_hot: True
phase: False

trainer:
learning_start: 1000

sumo4x4.cfg:
{
"network": "grid4x4",
"interval": 1.0,
"seed": 0,
"dir": "data/",
"roadnetFile": "raw_data/grid4x4/grid4x4.net.xml",
"flowFile": "raw_data/grid4x4/grid4x4.rou.xml",
"convertroadnetFile": "raw_data/grid4x4/grid4x4_roadnet_red.json",
"convertflowFile": "raw_data/grid4x4/grid4x4_flow.json",
"no_warning": true,
"name": "debug",
"yellow_length": 3,
"gui": false
}
I didn't change the hyperparameter in the original code. Thanks!