code-bench-CodeGemma-7B-cg-nv9n_zs

This model is a fine-tuned version of google/codegemma-7b on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0352

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 3
• seed: 42
• distributed_type: multi-GPU
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.03
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.6222	0.0530	50	0.5965
0.4468	0.1061	100	0.4544
0.3745	0.1591	150	0.3239
0.2795	0.2121	200	0.2315
0.1976	0.2652	250	0.1633
0.1415	0.3182	300	0.1274
0.1071	0.3713	350	0.0852
0.0845	0.4243	400	0.0698
0.0724	0.4773	450	0.0610
0.0716	0.5304	500	0.0574
0.0745	0.5834	550	0.0501
0.0618	0.6364	600	0.0507
0.0602	0.6895	650	0.0494
0.0646	0.7425	700	0.0480
0.0554	0.7955	750	0.0471
0.0506	0.8486	800	0.0450
0.0591	0.9016	850	0.0445
0.0498	0.9547	900	0.0440
0.0457	1.0077	950	0.0430
0.0484	1.0607	1000	0.0425
0.0434	1.1138	1050	0.0416
0.0473	1.1668	1100	0.0416
0.0512	1.2198	1150	0.0424
0.0521	1.2729	1200	0.0410
0.0526	1.3259	1250	0.0407
0.0393	1.3789	1300	0.0409
0.041	1.4320	1350	0.0399
0.0505	1.4850	1400	0.0405
0.0495	1.5381	1450	0.0399
0.0454	1.5911	1500	0.0392
0.0428	1.6441	1550	0.0393
0.045	1.6972	1600	0.0386
0.046	1.7502	1650	0.0383
0.0436	1.8032	1700	0.0383
0.0445	1.8563	1750	0.0386
0.0414	1.9093	1800	0.0382
0.05	1.9623	1850	0.0385
0.0444	2.0154	1900	0.0382
0.0422	2.0684	1950	0.0382
0.0494	2.1215	2000	0.0381
0.0408	2.1745	2050	0.0378
0.0394	2.2275	2100	0.0379
0.0392	2.2806	2150	0.0372
0.0356	2.3336	2200	0.0375
0.0448	2.3866	2250	0.0371
0.0477	2.4397	2300	0.0369
0.0404	2.4927	2350	0.0369
0.0378	2.5457	2400	0.0367
0.0448	2.5988	2450	0.0367
0.0408	2.6518	2500	0.0364
0.0419	2.7049	2550	0.0363
0.0431	2.7579	2600	0.0365
0.042	2.8109	2650	0.0363
0.0398	2.8640	2700	0.0362
0.033	2.9170	2750	0.0360
0.0447	2.9700	2800	0.0359
0.0402	3.0231	2850	0.0360
0.0433	3.0761	2900	0.0359
0.0342	3.1291	2950	0.0363
0.0404	3.1822	3000	0.0360
0.0365	3.2352	3050	0.0357
0.0374	3.2883	3100	0.0357
0.0363	3.3413	3150	0.0358
0.0378	3.3943	3200	0.0357
0.0376	3.4474	3250	0.0356
0.0347	3.5004	3300	0.0355
0.0391	3.5534	3350	0.0353
0.0306	3.6065	3400	0.0353
0.0351	3.6595	3450	0.0353
0.0349	3.7125	3500	0.0354
0.0399	3.7656	3550	0.0354
0.033	3.8186	3600	0.0352
0.0304	3.8717	3650	0.0352
0.0322	3.9247	3700	0.0354
0.0362	3.9777	3750	0.0350
0.0346	4.0308	3800	0.0352
0.0292	4.0838	3850	0.0353
0.0294	4.1368	3900	0.0354
0.0318	4.1899	3950	0.0354
0.0357	4.2429	4000	0.0351
0.0325	4.2959	4050	0.0352
0.0337	4.3490	4100	0.0352
0.0346	4.4020	4150	0.0353
0.0341	4.4551	4200	0.0352
0.0347	4.5081	4250	0.0353
0.0343	4.5611	4300	0.0352
0.0339	4.6142	4350	0.0352
0.0358	4.6672	4400	0.0352
0.0312	4.7234	4450	0.0352
0.0355	4.7765	4500	0.0351
0.0292	4.8295	4550	0.0351
0.0365	4.8825	4600	0.0352
0.036	4.9356	4650	0.0352
0.0319	4.9886	4700	0.0352

Framework versions

• PEFT 0.12.0
• Transformers 4.44.2
• Pytorch 2.5.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1