Regression Benchmarks

Dataset Overview

Dataset	Instances	Features	Numeric	Categorical	Target
Geographical origin of music	1059	116	116	0	latitude
Student performance por	649	30	13	17	G3

Quick Check

Dataset	Type	Family / Variant	Test RMSE	Test R²
Geographical origin of music	Best classical	Trees / CatBoost	16.09875 ± 0.27347	0.28649 ± 0.02426
Geographical origin of music	Best transformed	CNN / REFINED	15.02703 ± 0.62945	0.37760 ± 0.05210
Student performance por	Best classical	Trees / LightGBM	2.07327 ± 0.01229	0.24692 ± 0.00894
Student performance por	Best transformed	ViT+MLP / REFINED	2.00620 ± 0.02164	0.29482 ± 0.01526

Geographical origin of music

Source: OpenML · Original

Leaderboard

Family	Best variant	Test RMSE	Test R²	Train time (s)	#Params	FLOPs
Trees	CatBoost	16.09875 ± 0.27347	0.28649 ± 0.02426	5.40574 ± 0.01840	—	—
MLP	MLP	16.82875 ± 0.58908	0.21974 ± 0.05508	6.15275 ± 0.54803	224,257	447,616
ViT	BarGraph	16.91933 ± 0.91566	0.21024 ± 0.08402	59.57013 ± 7.93472	6,900,631	223,482,786
ViT+MLP	Combination	16.10768 ± 0.12356	0.28583 ± 0.01098	52.62258 ± 1.54688	9,753,271	6,869,758,986
CNN	REFINED	15.02703 ± 0.62945	0.37760 ± 0.05210	17.44302 ± 0.11142	3,738,049	120,195,072
CNN+MLP	REFINED	15.29549 ± 0.55394	0.35539 ± 0.04664	18.31227 ± 0.12764	3,843,265	120,404,960

Architecture Results

Tree Baselines

Method	Train RMSE	Val RMSE	Test RMSE	Train R²	Val R²	Test R²	Train time (s)	#Params	FLOPs
CatBoost	0.05557 ± 0.00772	16.06202 ± 0.18918	16.09875 ± 0.27347	0.99999 ± 0.00000	0.34184 ± 0.01551	0.28649 ± 0.02426	5.40574 ± 0.01840	—	—
XGBoost	0.33621 ± 0.00319	16.37622 ± 0.32143	16.30701 ± 0.36834	0.99965 ± 0.00001	0.31570 ± 0.02669	0.26779 ± 0.03284	1.55595 ± 0.01979	—	—
LightGBM	0.16624 ± 0.03577	16.21642 ± 0.25397	16.98768 ± 0.27991	0.99991 ± 0.00004	0.32907 ± 0.02109	0.20553 ± 0.02621	0.28268 ± 0.01466	—	—

MLP

Method	Train RMSE	Val RMSE	Test RMSE	Train R²	Val R²	Test R²	#Params	FLOPs	Train time (s)
MLP	10.69368 ± 1.91871	15.51013 ± 0.46948	16.82875 ± 0.58908	0.63755 ± 0.13473	0.38591 ± 0.03751	0.21974 ± 0.05508	224,257	447,616	6.15275 ± 0.54803

ViT

Method	Train RMSE	Val RMSE	Test RMSE	Train R²	Val R²	Test R²	#Params	FLOPs	Train time (s)
TINTO	13.70347 ± 2.51237	17.03049 ± 0.48423	18.52677 ± 0.28237	0.40415 ± 0.21213	0.25968 ± 0.04228	0.05509 ± 0.02877	5,811,351	190,865,164	46.01703 ± 4.92198
IGTD	4.38130 ± 2.75908	15.54277 ± 0.44909	17.99486 ± 0.78529	0.92187 ± 0.10129	0.38336 ± 0.03551	0.10737 ± 0.07837	18,975,257	187,320,724	98.01576 ± 3.22877
REFINED	12.73110 ± 3.20448	17.47121 ± 0.58240	17.68148 ± 0.54004	0.47380 ± 0.24188	0.22068 ± 0.05152	0.13886 ± 0.05282	4,072,967	1,237,334,920	50.97426 ± 1.32109
DistanceMatrix	11.53372 ± 2.91193	16.06591 ± 0.18580	17.57294 ± 0.84132	0.56799 ± 0.18100	0.34152 ± 0.01526	0.14848 ± 0.08357	24,509,305	228,074,320	51.79272 ± 2.42832
BarGraph	11.66409 ± 1.05832	17.42123 ± 0.35203	16.91933 ± 0.91566	0.57684 ± 0.07260	0.22557 ± 0.03103	0.21024 ± 0.08402	6,900,631	223,482,786	59.57013 ± 7.93472
Combination	11.55673 ± 2.67506	16.69437 ± 0.38998	17.58062 ± 1.12846	0.56962 ± 0.17848	0.28876 ± 0.03341	0.14649 ± 0.11207	9,560,503	314,769,189	43.79353 ± 0.88633
SuperTML	15.00412 ± 1.59699	19.24220 ± 0.30550	19.04030 ± 0.12776	0.29808 ± 0.14780	0.05533 ± 0.03011	0.00213 ± 0.01335	5,039,297	168,272,312	71.57072 ± 0.39260
FeatureWrap	15.36428 ± 0.55970	18.35840 ± 0.28669	18.77633 ± 0.23099	0.26981 ± 0.05265	0.14012 ± 0.02685	0.02952 ± 0.02381	2,223,271	22,210,476	32.03794 ± 0.18428
BIE	17.83038 ± 0.36524	19.74467 ± 0.09284	19.09284 ± 0.02706	0.01731 ± 0.04006	0.01506 ± 0.06072	-0.00336 ± 0.00284	20,250,457	190,315,952	50.08791 ± 1.41411

ViT + MLP

Method	Train RMSE	Val RMSE	Test RMSE	Train R²	Val R²	Test R²	#Params	FLOPs	Train time (s)
TINTO	11.27870 ± 0.31786	15.67644 ± 0.16878	16.16954 ± 0.32113	0.60668 ± 0.02228	0.37307 ± 0.01343	0.28015 ± 0.02850	5,800,087	1,888,211,296	45.43299 ± 1.59490
IGTD	11.49398 ± 1.82290	15.90038 ± 0.40134	17.39535 ± 0.77172	0.58357 ± 0.13235	0.35476 ± 0.03226	0.16582 ± 0.07515	19,047,193	2,692,426,912	54.17769 ± 0.81412
REFINED	10.88294 ± 0.70574	15.46777 ± 0.30883	16.18275 ± 0.41792	0.63280 ± 0.04735	0.38951 ± 0.02433	0.27881 ± 0.03704	4,107,015	22,952,799,504	41.30654 ± 0.10477
DistanceMatrix	10.69971 ± 1.13039	15.59085 ± 0.23527	16.26941 ± 0.43020	0.64309 ± 0.07573	0.37984 ± 0.01857	0.27105 ± 0.03890	23,906,425	3,014,266,752	55.16052 ± 0.52730
BarGraph	8.79558 ± 1.46960	15.62614 ± 0.41414	16.46694 ± 0.35351	0.75562 ± 0.08199	0.37679 ± 0.03273	0.25338 ± 0.03206	6,862,999	7,575,609,720	69.28047 ± 1.21371
Combination	12.21999 ± 1.18632	15.81405 ± 0.25097	16.10768 ± 0.12356	0.53511 ± 0.08945	0.36195 ± 0.02026	0.28583 ± 0.01098	9,753,271	6,869,758,986	52.62258 ± 1.54688
SuperTML	7.93617 ± 1.83935	15.33343 ± 0.54976	16.71844 ± 0.29170	0.79702 ± 0.09977	0.39964 ± 0.04318	0.23050 ± 0.02715	5,360,865	1,577,520,064	81.53160 ± 1.05610
FeatureWrap	9.21742 ± 0.92978	15.51625 ± 0.51796	16.67730 ± 0.72491	0.73534 ± 0.05056	0.38533 ± 0.04116	0.23331 ± 0.06575	2,517,607	731,278,872	41.72618 ± 1.19304
BIE	11.16109 ± 1.23325	15.92937 ± 0.75222	16.55521 ± 0.17354	0.61133 ± 0.08502	0.35158 ± 0.06225	0.24557 ± 0.01578	19,967,065	8,862,326,592	67.69650 ± 3.22348

CNN

Method	Train RMSE	Val RMSE	Test RMSE	Train R²	Val R²	Test R²	#Params	FLOPs	Train time (s)
TINTO	11.42706 ± 5.63796	17.50269 ± 0.46048	18.69147 ± 1.01999	0.51795 ± 0.33917	0.21813 ± 0.04067	0.03610 ± 0.10680	1,064,865	112,489,856	22.07763 ± 0.29004
IGTD	3.40363 ± 1.57712	15.76505 ± 0.74902	17.51502 ± 0.49999	0.95806 ± 0.03678	0.36488 ± 0.05951	0.15508 ± 0.04785	283,521	12,525,632	23.86034 ± 0.15064
REFINED	1.45952 ± 0.31864	15.26121 ± 0.31388	15.02703 ± 0.62945	0.99317 ± 0.00279	0.40569 ± 0.02440	0.37760 ± 0.05210	3,738,049	120,195,072	17.44302 ± 0.11142
DistanceMatrix	5.26771 ± 4.56349	15.72246 ± 0.50765	16.66955 ± 0.89513	0.86278 ± 0.18246	0.36892 ± 0.04026	0.23341 ± 0.08207	1,203,209	137,155,464	29.85507 ± 1.14905
BarGraph	5.41248 ± 4.20403	17.10003 ± 0.67535	17.98593 ± 0.71919	0.86579 ± 0.16321	0.25317 ± 0.05967	0.10848 ± 0.07187	86,545	90,185,728	28.58858 ± 0.19662
Combination	2.21433 ± 1.04818	15.95586 ± 0.17249	16.76053 ± 0.27255	0.98213 ± 0.01559	0.35052 ± 0.01405	0.22664 ± 0.02509	2,976,641	991,275,840	27.91533 ± 0.61219
SuperTML	14.58648 ± 1.90400	18.78454 ± 0.51314	19.16329 ± 0.47282	0.33361 ± 0.17012	0.09937 ± 0.04895	-0.01126 ± 0.04976	277,457	279,559,680	34.35937 ± 0.34717
FeatureWrap	15.37580 ± 1.24064	18.89610 ± 0.46891	19.02199 ± 0.44515	0.26569 ± 0.11599	0.08874 ± 0.04542	0.00364 ± 0.04706	36,769	3,266,144	13.14731 ± 0.89654
BIE	4.88207 ± 4.02859	19.64237 ± 0.61272	19.34129 ± 0.51306	0.88623 ± 0.13971	0.01506 ± 0.06072	-0.03022 ± 0.05498	19,046,097	1,580,752,896	38.22191 ± 0.87412

CNN + MLP

Method	Train RMSE	Val RMSE	Test RMSE	Train R²	Val R²	Test R²	#Params	FLOPs	Train time (s)
TINTO	10.11680 ± 2.57885	15.85264 ± 0.61112	17.19357 ± 0.86346	0.66731 ± 0.15259	0.35819 ± 0.04981	0.18469 ± 0.08119	1,354,913	113,068,800	24.04323 ± 0.43570
IGTD	3.37414 ± 3.68402	16.04367 ± 0.78891	17.85496 ± 0.89318	0.93127 ± 0.13106	0.34214 ± 0.06378	0.12078 ± 0.08993	505,473	12,968,736	23.35832 ± 0.13444
REFINED	3.90924 ± 2.74580	15.06717 ± 0.42406	15.29549 ± 0.55394	0.93414 ± 0.07172	0.42054 ± 0.03240	0.35539 ± 0.04664	3,843,265	120,404,960	18.31227 ± 0.12764
DistanceMatrix	2.46382 ± 1.43092	16.01207 ± 0.36239	16.19995 ± 0.35934	0.97618 ± 0.02643	0.34573 ± 0.02997	0.27738 ± 0.03198	1,472,777	137,693,672	30.08562 ± 0.68356
BarGraph	13.46694 ± 0.92622	17.15620 ± 0.49037	17.43264 ± 0.60262	0.43749 ± 0.07690	0.24870 ± 0.04300	0.16275 ± 0.05797	318,865	90,649,472	31.54327 ± 0.74361
Combination	2.69676 ± 0.48891	15.55850 ± 0.28827	16.38854 ± 0.14145	0.97694 ± 0.00809	0.38235 ± 0.02296	0.26070 ± 0.01276	3,013,761	991,349,664	28.90632 ± 1.47219
SuperTML	12.44674 ± 1.87392	16.39167 ± 0.44488	17.21217 ± 0.44550	0.51262 ± 0.14688	0.31422 ± 0.03778	0.18414 ± 0.04263	542,865	280,089,472	35.44523 ± 0.53667
FeatureWrap	11.59757 ± 3.70233	17.73740 ± 0.23284	18.04943 ± 0.44827	0.55051 ± 0.21409	0.19735 ± 0.02110	0.10287 ± 0.04485	257,745	3,707,280	13.81799 ± 1.11630
BIE	3.94150 ± 0.84846	18.73255 ± 0.55031	18.62305 ± 0.62363	0.95022 ± 0.01925	0.10427 ± 0.05202	0.04456 ± 0.06396	18,945,553	1,580,551,520	40.39863 ± 0.22127

Student performance por

Source: OpenML · Original

Leaderboard

Family	Best variant	Test RMSE (↓)	Test R² (↑)	Train time (s)	#Params	FLOPs
Trees	LightGBM	2.07327 ± 0.01229	0.24692 ± 0.00894	0.06854 ± 0.00324	—	—
MLP	MLP	2.09287 ± 0.08028	0.23174 ± 0.05981	3.05389 ± 0.51833	292,353	583,680
ViT (only)	REFINED	2.13202 ± 0.06906	0.20300 ± 0.05223	19.78263 ± 0.04077	8,970,759	1,111,201,232
ViT+MLP	REFINED	2.00620 ± 0.02164	0.29482 ± 0.01526	24.76651 ± 0.31276	8,878,343	33,044,561,088
CNN	IGTD	2.08120 ± 0.15865	0.23765 ± 0.12046	8.71615 ± 0.05789	824,241	12,292,800
CNN+MLP	REFINED	2.03476 ± 0.06501	0.27407 ± 0.04684	20.00194 ± 0.48703	6,591,281	730,020,896

Architecture Results

Tree Baselines

Method	Test RMSE	Test R²	Train time (s)	#Params	FLOPs
LightGBM	2.07327 ± 0.01229	0.24692 ± 0.00894	0.06854 ± 0.00324	—	—
XGBoost	2.07417 ± 0.01860	0.24625 ± 0.01354	0.15527 ± 0.00400	—	—
CatBoost	2.08151 ± 0.00951	0.24094 ± 0.00693	12.64899 ± 0.14024	—	—

MLP

Train loss	Val loss	Test loss	Train RMSE	Val RMSE	Test RMSE	Test R²	Total params	FLOPs
5.52015 ± 3.62724	5.85536 ± 0.08859	4.64651 ± 0.62714	2.20826 ± 0.92555	2.75390 ± 0.02521	2.09287 ± 0.08028	0.23174 ± 0.05981	292,353	583,680

ViT

Method	Test RMSE	Test R²	Train time (s)	#Params	FLOPs
TINTO	2.23586 ± 0.02876	0.12409 ± 0.02259	18.44586 ± 0.19326	903,703	29,203,002
IGTD	2.28951 ± 0.07894	0.08080 ± 0.06307	11.72688 ± 0.10983	2,774,881	27,840,536
REFINED	2.13202 ± 0.06906	0.20300 ± 0.05223	19.78263 ± 0.04077	8,970,759	1,111,201,232
DistanceMatrix	2.15614 ± 0.07647	0.18473 ± 0.05856	32.18724 ± 0.15245	17,054,305	1,703,980,720
BarGraph	2.25989 ± 0.12131	0.10322 ± 0.09584	12.96351 ± 0.18219	1,216,353	459,910,052
Combination	2.21138 ± 0.12874	0.14096 ± 0.09914	14.19763 ± 0.12567	4,855,105	456,591,504
SuperTML	2.22813 ± 0.13748	0.12761 ± 0.11154	18.68712 ± 0.78474	1,753,089	55,688,968
FeatureWrap	2.26426 ± 0.03637	0.10163 ± 0.02890	40.14557 ± 4.76981	22,093,623	442,033,759
BIE	2.32541 ± 0.10244	0.05117 ± 0.08415	50.89112 ± 0.34698	22,192,513	2,904,925,784

ViT + MLP

Method	Test RMSE	Test R²	Train time (s)	#Params	FLOPs
TINTO	2.22408 ± 0.13201	0.13097 ± 0.10137	22.41009 ± 0.33038	1,360,151	293,072,208
IGTD	2.04088 ± 0.02291	0.27022 ± 0.01646	16.02527 ± 0.26102	3,116,769	914,998,320
REFINED	2.00620 ± 0.02164	0.29482 ± 0.01526	24.76651 ± 0.31276	8,878,343	33,044,561,088
DistanceMatrix	2.06605 ± 0.03636	0.25200 ± 0.02656	38.64666 ± 0.90085	17,145,249	14,740,056,064
BarGraph	2.04117 ± 0.01500	0.27005 ± 0.01072	18.48115 ± 0.51367	1,436,097	17,873,390,744
Combination	2.15356 ± 0.06181	0.18696 ± 0.04625	18.95864 ± 0.27732	4,862,817	8,873,642,016
SuperTML	2.06424 ± 0.05476	0.25307 ± 0.03928	21.70629 ± 1.15053	2,078,081	1,683,872,736
FeatureWrap	2.18022 ± 0.09262	0.16605 ± 0.06917	53.58027 ± 1.19315	22,633,079	23,547,780,378
BIE	2.10730 ± 0.07137	0.22131 ± 0.05285	57.14752 ± 1.44125	22,625,345	24,682,275,008

CNN

Method	Test RMSE	Test R²	Train time (s)	#Params	FLOPs
TINTO	2.20393 ± 0.05426	0.14863 ± 0.04235	9.40622 ± 0.28257	44,137	9,609,200
IGTD	2.08120 ± 0.15865	0.23765 ± 0.12046	8.71615 ± 0.05789	824,241	12,292,800
REFINED	2.13354 ± 0.11906	0.20055 ± 0.09003	14.35458 ± 0.06314	6,492,657	66,168,768
DistanceMatrix	2.12103 ± 0.06746	0.21121 ± 0.05013	24.99695 ± 0.21563	1,405,713	487,848,704
BarGraph	2.32736 ± 0.10341	0.04956 ± 0.08450	20.30143 ± 0.14342	668,033	184,843,344
Combination	2.21374 ± 0.12857	0.13913 ± 0.09999	16.30842 ± 0.07185	1,710,385	1,457,369,280
SuperTML	2.25473 ± 0.07110	0.10865 ± 0.05721	19.47898 ± 0.36295	319,761	310,244,736
FeatureWrap	2.58094 ± 0.12282	-0.16911 ± 0.11451	23.04521 ± 0.18939	3,067,097	63,097,992
BIE	2.30823 ± 0.17893	0.06211 ± 0.14743	21.22984 ± 1.06166	2,692,721	256,454,400

CNN + MLP

Method	Test RMSE	Test R²	Train time (s)	#Params	FLOPs
TINTO	2.10339 ± 0.05933	0.22442 ± 0.04448	12.84134 ± 0.07940	537,033	116,529,336
IGTD	2.16746 ± 0.10360	0.17547 ± 0.07799	12.30086 ± 0.17006	1,066,417	140,541,632
REFINED	2.03476 ± 0.06501	0.27407 ± 0.04684	20.00194 ± 0.48703	6,591,281	730,020,896
DistanceMatrix	2.20535 ± 0.11973	0.14593 ± 0.09290	27.69409 ± 1.14331	1,836,817	488,709,824
BarGraph	2.09846 ± 0.08224	0.22759 ± 0.06083	30.53304 ± 1.71425	1,040,449	2,041,457,440
Combination	2.21351 ± 0.13429	0.13910 ± 0.10258	18.47078 ± 0.45905	1,969,073	1,457,885,888
SuperTML	2.05941 ± 0.01953	0.25693 ± 0.01407	21.61322 ± 0.57559	656,721	310,917,504
FeatureWrap	2.08629 ± 0.08131	0.23653 ± 0.05844	24.74906 ± 0.23222	3,502,169	63,967,080
BIE	2.30793 ± 0.04552	0.06655 ± 0.03657	21.66396 ± 0.61913	3,000,625	257,069,440