我使用AADB数据集训练模型，训练时报错：
[i 0616 08:08:25.320000 52 cuda_flags.cc:32] CUDA enabled.
@ [Training Model] Arch = [resnet50_HLAGCN]; Dataset = [aadb]
@ LR = [0.01]; Total epoch = [20]; Batch size = [8]
@ weight_decay = [0.0001]; momentum = [0.9]; workers = [8]
@ model save dir: results/resnet50_HLAGCN_v0_aadb
@ model save period: 4
Preprocessing dataset…
AADB dataset info preloaded in ./preprocess/: #8435 train #498 val #996 test
Preloading file saved!
[w 0616 08:10:07.622000 52 init.py:1118] load parameter fc.weight failed …
[w 0616 08:10:07.625000 52 init.py:1118] load parameter fc.bias failed …
[w 0616 08:10:07.627000 52 init.py:1136] load total 267 params, 2 failed
=> Start training #Ep 1 /20
08:10:07->Ep:[1][ 0/8435] - Net:59.5 - Load:56.1 - loss_avg:nan
Compiling Operators(52/52) used: 2.02s eta: 0s
08:10:07->Ep:[1][ 500/8435] - Net:0.5 - Load:0.1 - loss_avg:nan
08:10:07->Ep:[1][1000/8435] - Net:0.5 - Load:0.1 - loss_avg:nan
—> Train: 8.26 min/epoch, train loss: nan - lr: 0.01000
Traceback (most recent call last):
File “D:\Python\SetUpPath\lib\runpy.py”, line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File “D:\Python\SetUpPath\lib\runpy.py”, line 86, in run_code
exec(code, run_globals)
File “D:\Python\Python_Demo\图像评分模型\hlagcn-jittor-main\utils_jittor\train_jittor.py”, line 237, in
main()
File “D:\Python\Python_Demo\图像评分模型\hlagcn-jittor-main\utils_jittor\train_jittor.py”, line 40, in main
main_worker(args)
File “D:\Python\Python_Demo\图像评分模型\hlagcn-jittor-main\utils_jittor\train_jittor.py”, line 150, in main_worker
val_loss, val_acc_aes = val_test_process(val_loader, model, criterions, args)
File "D:\Python\SetUpPath\lib\site-packages\jittor_init.py", line 291, in inner
ret = func(*args, **kw)
File “D:\Python\Python_Demo\图像评分模型\hlagcn-jittor-main\utils_jittor\train_jittor.py”, line 224, in val_test_process
metrics = cal_metrics(scores_hist, labels_hist, args.bins)
File “D:\Python\Python_Demo\图像评分模型\hlagcn-jittor-main\utils_jittor\util.py”, line 143, in cal_metrics
plcc, _ = pearsonr(scores_mean, labels_mean)
File “D:\Python\SetUpPath\lib\site-packages\scipy\stats_stats_py.py”, line 4090, in pearsonr
normxm = linalg.norm(xm)
File “D:\Python\SetUpPath\lib\site-packages\scipy\linalg_misc.py”, line 145, in norm
a = np.asarray_chkfinite(a)
File “D:\Python\SetUpPath\lib\site-packages\numpy\lib\function_base.py”, line 603, in asarray_chkfinite
raise ValueError(
ValueError: array must not contain infs or NaNs
不知道哪个部分出了问题。
