Multiclass Benchmarks

Dataset Overview

Dataset	Instances	Features	Classes	Numeric	Categorical	Target
Cmc	1473	9	3	2	7	Contraceptive_method_used
Cnae-9	1080	856	9	856	0	Class
Connect-4	67557	42	3	0	42	class
Covtype	581012	54	7	10	44	54
Dna	3186	180	3	0	180	class
Gas	13910	128	6	128	0	Class
Isolet	7797	617	26	617	0	class
Mfeat-fourier	2000	76	10	76	0	class

Quick Check

Dataset	Type	Family / Variant	Test accuracy	Test F1
Cmc	Best classical	Trees / CatBoost	0.55928 ± 0.00248	0.53078 ± 0.00346
Cmc	Best transformed	ViT / BarGraph	0.56290 ± 0.01452	0.56045 ± 0.01604
Cnae-9	Best classical	MLP / MLP	0.97037 ± 0.00805	0.96992 ± 0.00846
Cnae-9	Best transformed	ViT+MLP / SuperTML	0.97037 ± 0.00517	0.97035 ± 0.00532
Connect-4	Best classical	Trees / XGBoost	0.86860 ± 0.00059	0.73852 ± 0.00118
Connect-4	Best transformed	ViT+MLP / IGTD	0.86819 ± 0.00221	0.86365 ± 0.00250
Covtype	Best classical	MLP / MLP	0.96310 ± 0.00000	0.96312 ± 0.00000
Covtype	Best transformed	ViT+MLP / REFINED	0.97050 ± 0.00000	0.97051 ± 0.00000
Dna	Best classical	Trees / LightGBM	0.96904 ± 0.00229	0.96586 ± 0.00223
Dna	Best transformed	ViT+MLP / IGTD	0.96569 ± 0.00482	0.96579 ± 0.00484
Gas	Best classical	Trees / CatBoost	0.99482 ± 0.00021	0.99480 ± 0.00026
Gas	Best transformed	CNN / REFINED	0.99559 ± 0.00092	0.99559 ± 0.00092
Isolet	Best classical	Trees / XGBoost	0.96222 ± 0.00316	0.96221 ± 0.00314
Isolet	Best transformed	ViT / REFINED	0.96376 ± 0.00268	0.96371 ± 0.00267
Mfeat-fourier	Best classical	Trees / CatBoost	0.858667 ± 0.009603	N/A
Mfeat-fourier	Best transformed	ViT / TINTO_blur	0.870000 ± 0.016997	N/A

Cmc

Source: OpenML · Original

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
Trees	CatBoost	0.55928 ± 0.00248	0.53078 ± 0.00346	15.49520 ± 0.52367	—	—
MLP	MLP	0.52670 ± 0.01711	0.52548 ± 0.01566	20.21081 ± 0.15326	899.00000 ± 0.00000	—
ViT	BarGraph	0.56290 ± 0.01452	0.56045 ± 0.01604	70.66814 ± 2.09838	3,222,435 ± 0.00000	31,831,688 ± 0.00000
ViT+MLP	Combination	0.55837 ± 0.01855	0.55904 ± 0.01354	54.89044 ± 0.72794	1,167,587 ± 0.00000	257,758,240 ± 50,943,945.25477
CNN	DistanceMatrix	0.55475 ± 0.02707	0.55731 ± 0.02529	41.15781 ± 0.98381	764,595 ± 0.00000	124,157,184 ± 0.00000
CNN+MLP	SuperTML	0.54751 ± 0.01534	0.54463 ± 0.01651	136.04745 ± 2.61454	353,731 ± 0.00000	6,508,601,440 ± 0.00000

Architecture Results

Tree Baselines

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Train time (s)	#Params	FLOPs
CatBoost	0.59088 ± 0.00467	0.60815 ± 0.00686	0.55928 ± 0.00248	0.53078 ± 0.00346	0.54658 ± 0.00803	0.52793 ± 0.00371	15.49520 ± 0.52367	—	—
LightGBM	0.62231 ± 0.00556	0.61086 ± 0.00715	0.55928 ± 0.01380	0.53068 ± 0.01289	0.54304 ± 0.01377	0.52804 ± 0.01305	0.49725 ± 0.09115	—	—
XGBoost	0.67352 ± 0.00421	0.60091 ± 0.00743	0.54932 ± 0.00248	0.52019 ± 0.00285	0.53467 ± 0.00314	0.51779 ± 0.00278	0.36403 ± 0.00770	—	—

MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	Train time (s)
MLP	0.64597 ± 0.00873	0.56199 ± 0.00981	0.52670 ± 0.01711	0.52548 ± 0.01566	0.53118 ± 0.01556	0.52670 ± 0.01711	0.70562 ± 0.00858	0.97326 ± 0.01014	0.26985 ± 0.02506	899.00000 ± 0.00000	20.21081 ± 0.15326

ViT

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.58060 ± 0.02010	0.54480 ± 0.02184	0.51403 ± 0.01304	0.50566 ± 0.01671	0.51244 ± 0.01460	0.51403 ± 0.01304	0.68201 ± 0.00601	0.99348 ± 0.00872	0.23816 ± 0.02143	18,557,531 ± 0.00000	185,223,012 ± 0.00000	116.61239 ± 2.29721
IGTD	0.42677 ± 0.00000	0.42534 ± 0.00000	0.42986 ± 0.00000	0.25846 ± 0.00000	0.18478 ± 0.00000	0.42986 ± 0.00000	0.50000 ± 0.00000	1.06630 ± 0.00002	0.00000 ± 0.00000	18,756,425 ± 0.00000	961,520,359 ± 0.00000	148.18030 ± 1.54484
REFINED	0.61028 ± 0.02942	0.60543 ± 0.02008	0.55747 ± 0.01675	0.55971 ± 0.01710	0.57576 ± 0.01606	0.55747 ± 0.01675	0.73846 ± 0.01551	0.91166 ± 0.01975	0.32415 ± 0.02569	8,034,345 ± 0.00000	405,902,263 ± 0.00000	131.15941 ± 5.83637
DistanceMatrix	0.62405 ± 0.04610	0.57014 ± 0.03247	0.52941 ± 0.01534	0.52638 ± 0.01697	0.53164 ± 0.01732	0.52941 ± 0.01534	0.70803 ± 0.00756	0.99914 ± 0.03235	0.27248 ± 0.02357	4,389,851 ± 0.00000	154,472,600 ± 0.00000	128.09730 ± 3.34499
BarGraph	0.62464 ± 0.01715	0.59186 ± 0.01849	0.56290 ± 0.01452	0.56045 ± 0.01604	0.56920 ± 0.01562	0.56290 ± 0.01452	0.72685 ± 0.00615	0.94493 ± 0.01276	0.32581 ± 0.02336	3,222,435 ± 0.00000	31,831,688 ± 0.00000	70.66814 ± 2.09838
Combination	0.59476 ± 0.00835	0.60091 ± 0.01735	0.54480 ± 0.02650	0.53989 ± 0.02733	0.55690 ± 0.01973	0.54480 ± 0.02650	0.73836 ± 0.00875	0.92103 ± 0.01228	0.29605 ± 0.03647	1,152,003 ± 0.00000	23,116,164 ± 0.00000	50.64266 ± 3.07093
SuperTML	0.62250 ± 0.01884	0.59457 ± 0.01180	0.52851 ± 0.00671	0.52090 ± 0.01065	0.52975 ± 0.02017	0.52851 ± 0.00671	0.71625 ± 0.00255	0.95196 ± 0.02299	0.26546 ± 0.01895	14,585,091 ± 0.00000	480,755,835 ± 0.00000	123.16419 ± 3.17526
FeatureWrap	0.52434 ± 0.01430	0.56652 ± 0.00870	0.51131 ± 0.01154	0.49867 ± 0.02234	0.50469 ± 0.01292	0.51131 ± 0.01154	0.66903 ± 0.00290	1.00951 ± 0.00536	0.22990 ± 0.01796	9,119,321 ± 0.00000	89,289,244 ± 0.00000	134.36640 ± 1.40692
BIE	0.59573 ± 0.00771	0.58914 ± 0.02741	0.55294 ± 0.01821	0.55140 ± 0.01781	0.56052 ± 0.01137	0.55294 ± 0.01821	0.71498 ± 0.00598	0.94962 ± 0.00646	0.30942 ± 0.02454	21,438,339 ± 0.00000	730,066,308 ± 0.00000	124.84714 ± 6.21710

ViT + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.57595 ± 0.01134	0.57285 ± 0.00686	0.49683 ± 0.01409	0.48781 ± 0.01639	0.48894 ± 0.01589	0.49683 ± 0.01409	0.68981 ± 0.00179	0.99148 ± 0.00483	0.20697 ± 0.02791	18,560,475 ± 0.00000	3,658,451,148 ± 525,865,378.91535	117.50083 ± 2.54110
IGTD	0.60815 ± 0.04145	0.56471 ± 0.04793	0.52489 ± 0.02950	0.51519 ± 0.03628	0.52757 ± 0.02946	0.52489 ± 0.02950	0.69828 ± 0.01713	0.97191 ± 0.01441	0.25929 ± 0.05176	18,356,809 ± 0.00000	23,306,968,083 ± 1,754,835,821.31258	149.76682 ± 7.78152
REFINED	0.61125 ± 0.01059	0.60272 ± 0.00981	0.55113 ± 0.01480	0.55077 ± 0.01840	0.57143 ± 0.02202	0.55113 ± 0.01480	0.73435 ± 0.00985	0.92641 ± 0.01140	0.31665 ± 0.02847	7,760,777 ± 0.00000	3,748,450,744 ± 740,852,627.99955	125.47250 ± 13.94367
DistanceMatrix	0.64753 ± 0.03200	0.56561 ± 0.02217	0.54389 ± 0.01930	0.54059 ± 0.02547	0.55826 ± 0.01901	0.54389 ± 0.01930	0.71359 ± 0.01572	0.98598 ± 0.01211	0.30462 ± 0.03669	4,368,251 ± 0.00000	5,390,830,368 ± 304,416,115.04435	150.77362 ± 4.50453
BarGraph	0.62987 ± 0.01155	0.58552 ± 0.02161	0.54027 ± 0.03543	0.54345 ± 0.03433	0.55883 ± 0.03626	0.54027 ± 0.03543	0.72027 ± 0.02308	0.94456 ± 0.02947	0.30260 ± 0.05185	3,262,851 ± 0.00000	1,024,690,320 ± 90,009,869.65352	66.23054 ± 0.78443
Combination	0.59302 ± 0.01491	0.60905 ± 0.01136	0.55837 ± 0.01855	0.55904 ± 0.01354	0.57543 ± 0.01271	0.55837 ± 0.01855	0.74016 ± 0.00807	0.91011 ± 0.02151	0.32445 ± 0.02101	1,167,587 ± 0.00000	257,758,240 ± 50,943,945.25477	54.89044 ± 0.72794
SuperTML	0.60737 ± 0.01098	0.59728 ± 0.01319	0.53122 ± 0.01711	0.52844 ± 0.01931	0.53484 ± 0.01654	0.53122 ± 0.01711	0.71292 ± 0.00740	0.94419 ± 0.01945	0.27085 ± 0.02917	14,344,355 ± 0.00000	10,194,143,094 ± 895,464,193.631	143.16534 ± 2.41321
FeatureWrap	0.52609 ± 0.00816	0.58371 ± 0.00905	0.53122 ± 0.00757	0.52487 ± 0.00862	0.52775 ± 0.00806	0.53122 ± 0.00757	0.67457 ± 0.00162	1.01469 ± 0.00180	0.26195 ± 0.01374	8,971,289 ± 0.00000	4,481,517,712 ± 253,067,916.86262	139.72991 ± 11.27500
BIE	0.59282 ± 0.02763	0.56833 ± 0.02428	0.54299 ± 0.02619	0.54168 ± 0.02300	0.54634 ± 0.02360	0.54299 ± 0.02619	0.70445 ± 0.01312	0.95276 ± 0.01201	0.29270 ± 0.03527	21,571,715 ± 0.00000	7,198,581,280 ± 1,422,744,547.91564	139.94504 ± 2.70236

CNN

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.54394 ± 0.01165	0.52851 ± 0.01580	0.47602 ± 0.02904	0.46387 ± 0.02599	0.47440 ± 0.02399	0.47602 ± 0.02904	0.65797 ± 0.02125	1.03012 ± 0.02895	0.17912 ± 0.03591	13,356,803 ± 0.00000	477,365,760 ± 0.00000	99.38911 ± 0.98989
IGTD	0.64093 ± 0.03060	0.50860 ± 0.04452	0.51131 ± 0.01319	0.50027 ± 0.01458	0.50829 ± 0.01456	0.51131 ± 0.01319	0.68163 ± 0.00613	1.01187 ± 0.04742	0.23748 ± 0.02476	11,252,291 ± 0.00000	201,483,328 ± 0.00000	59.34190 ± 1.03583
REFINED	0.60330 ± 0.02913	0.56923 ± 0.03481	0.52398 ± 0.01480	0.52086 ± 0.01808	0.53631 ± 0.02372	0.52398 ± 0.01480	0.69901 ± 0.01071	1.01439 ± 0.05581	0.27037 ± 0.01940	1,171,875 ± 0.00000	21,680,800 ± 0.00000	86.79629 ± 6.35222
DistanceMatrix	0.59302 ± 0.02564	0.59276 ± 0.02170	0.55475 ± 0.02707	0.55731 ± 0.02529	0.56809 ± 0.02266	0.55475 ± 0.02707	0.72989 ± 0.01112	0.92750 ± 0.01318	0.32104 ± 0.03662	764,595 ± 0.00000	124,157,184 ± 0.00000	41.15781 ± 0.98381
BarGraph	0.64171 ± 0.00896	0.59728 ± 0.01061	0.53394 ± 0.02437	0.53428 ± 0.02706	0.54430 ± 0.02776	0.53394 ± 0.02437	0.71769 ± 0.00987	0.95512 ± 0.02738	0.28367 ± 0.04156	1,114,931 ± 0.00000	191,019,072 ± 0.00000	67.33545 ± 4.62728
Combination	0.60892 ± 0.00926	0.59095 ± 0.01304	0.54751 ± 0.01781	0.54866 ± 0.01771	0.56023 ± 0.01100	0.54751 ± 0.01781	0.72817 ± 0.00822	0.93757 ± 0.02026	0.30937 ± 0.02199	26,275 ± 0.00000	8,797,568 ± 0.00000	37.43239 ± 2.38739
SuperTML	0.59224 ± 0.01423	0.57828 ± 0.01793	0.53937 ± 0.02424	0.52629 ± 0.01800	0.55776 ± 0.02714	0.53937 ± 0.02424	0.71672 ± 0.00634	0.95154 ± 0.01303	0.28218 ± 0.02980	301,923 ± 0.00000	591,587,968 ± 0.00000	129.37107 ± 0.30095
FeatureWrap	0.56683 ± 0.03371	0.56018 ± 0.02294	0.50769 ± 0.01613	0.49073 ± 0.02660	0.50451 ± 0.01334	0.50769 ± 0.01613	0.68416 ± 0.01095	0.98980 ± 0.02288	0.21870 ± 0.03122	47,035 ± 0.00000	1,170,576 ± 0.00000	37.97507 ± 3.29608
BIE	0.62619 ± 0.03554	0.60000 ± 0.00991	0.55294 ± 0.01256	0.54402 ± 0.01736	0.55550 ± 0.00999	0.55294 ± 0.01256	0.73294 ± 0.01282	0.93518 ± 0.01830	0.30180 ± 0.01825	1,531,619 ± 0.00000	1,648,116,864 ± 0.00000	75.70332 ± 0.15330

CNN + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.58138 ± 0.02148	0.54570 ± 0.02137	0.48688 ± 0.01909	0.47571 ± 0.02089	0.48561 ± 0.02730	0.48688 ± 0.01909	0.66685 ± 0.01691	1.06727 ± 0.05824	0.19771 ± 0.03222	11,121,187 ± 0.00000	5,201,861,280 ± 0.00000	58.12462 ± 1.90735
IGTD	0.60446 ± 0.05032	0.54118 ± 0.02908	0.50045 ± 0.01342	0.47949 ± 0.03770	0.48193 ± 0.06481	0.50045 ± 0.01342	0.67919 ± 0.00750	1.03052 ± 0.03764	0.21786 ± 0.04125	10,765,475 ± 0.00000	2,205,616,160 ± 0.00000	67.84435 ± 0.69047
REFINED	0.61455 ± 0.02869	0.55566 ± 0.02507	0.52127 ± 0.01127	0.52341 ± 0.00904	0.54837 ± 0.01201	0.52127 ± 0.01127	0.70467 ± 0.01246	0.99529 ± 0.03033	0.27771 ± 0.01275	1,184,771 ± 0.00000	238,771,104 ± 0.00000	104.82458 ± 3.45941
DistanceMatrix	0.63220 ± 0.01282	0.61810 ± 0.02977	0.54208 ± 0.02179	0.54219 ± 0.02130	0.56618 ± 0.01260	0.54208 ± 0.02179	0.71733 ± 0.01292	0.94284 ± 0.01894	0.30570 ± 0.02134	699,059 ± 0.00000	1,364,290,400 ± 0.00000	33.44982 ± 2.31255
BarGraph	0.66149 ± 0.03365	0.60633 ± 0.04147	0.53575 ± 0.00248	0.53573 ± 0.00406	0.55262 ± 0.02107	0.53575 ± 0.00248	0.71719 ± 0.00791	0.95926 ± 0.03626	0.28710 ± 0.01162	1,105,427 ± 0.00000	2,101,001,056 ± 0.00000	56.81205 ± 2.08185
Combination	0.63628 ± 0.02356	0.60091 ± 0.02226	0.54480 ± 0.02650	0.54351 ± 0.02811	0.54956 ± 0.03032	0.54480 ± 0.02650	0.71769 ± 0.01576	0.94702 ± 0.02667	0.29524 ± 0.04483	33,571 ± 0.00000	96,932,704 ± 0.00000	38.03557 ± 2.37867
SuperTML	0.60175 ± 0.01094	0.57738 ± 0.01380	0.54751 ± 0.01534	0.54463 ± 0.01651	0.55968 ± 0.01551	0.54751 ± 0.01534	0.72676 ± 0.00391	0.92772 ± 0.00556	0.30052 ± 0.02404	353,731 ± 0.00000	6,508,601,440 ± 0.00000	136.04745 ± 2.61454
FeatureWrap	0.58332 ± 0.02737	0.52941 ± 0.02373	0.51765 ± 0.01304	0.51622 ± 0.01403	0.53846 ± 0.00605	0.51765 ± 0.01304	0.68590 ± 0.00847	1.00077 ± 0.02192	0.26501 ± 0.01341	109,771 ± 0.00000	14,251,248 ± 0.00000	57.12546 ± 1.01150
BIE	0.65936 ± 0.02150	0.56742 ± 0.01042	0.54480 ± 0.00405	0.54588 ± 0.00203	0.55830 ± 0.01191	0.54480 ± 0.00405	0.72639 ± 0.01346	0.95990 ± 0.02545	0.30242 ± 0.00778	1,648,579 ± 0.00000	1,648,350,240 ± 0.00000	77.10235 ± 0.14990

Cnae-9

Source: OpenML · Original

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
Trees	XGBoost	0.95432 ± 0.00338	0.95461 ± 0.00345	8.44052 ± 0.11371	—	—
MLP	MLP	0.97037 ± 0.00805	0.96992 ± 0.00846	10.34689 ± 0.44999	706,057 ± 0.00000	1,411,072 ± 0.00000
ViT	IGTD	0.93827 ± 0.01310	0.93838 ± 0.01313	26.78283 ± 1.99133	476,097 ± 0.00000	9,725,388 ± 0.00000
ViT+MLP	SuperTML	0.97037 ± 0.00517	0.97035 ± 0.00532	42.45165 ± 1.58963	3,207,145 ± 0.00000	10,130,914,304 ± 421,536,368.16605
CNN	SuperTML	0.94691 ± 0.00552	0.94697 ± 0.00573	155.72271 ± 0.08343	2,081,961 ± 0.00000	2,760,667,584 ± 0.00000
CNN+MLP	SuperTML	0.96543 ± 0.01033	0.96571 ± 0.00999	157.23238 ± 0.21957	2,804,841 ± 0.00000	30,383,234,816 ± 0.00000

Architecture Results

Tree Baselines

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Train time (s)	#Params	FLOPs
CatBoost	1.00000 ± 0.00000	0.93210 ± 0.00756	0.93951 ± 0.00805	0.94003 ± 0.00780	0.94732 ± 0.00626	0.93951 ± 0.00805	6.54664 ± 0.10359	—	—
LightGBM	0.98809 ± 0.00162	0.94444 ± 0.00000	0.94938 ± 0.00805	0.95004 ± 0.00766	0.95597 ± 0.00687	0.94938 ± 0.00805	8.07665 ± 1.28849	—	—
XGBoost	0.99868 ± 0.00000	0.94444 ± 0.00000	0.95432 ± 0.00338	0.95461 ± 0.00345	0.95848 ± 0.00428	0.95432 ± 0.00338	8.44052 ± 0.11371	—	—

MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
MLP	0.99788 ± 0.00201	0.96914 ± 0.00976	0.97037 ± 0.00805	0.96992 ± 0.00846	0.97239 ± 0.00756	0.97037 ± 0.00805	0.99961 ± 0.00018	0.10320 ± 0.01376	0.96701 ± 0.00891	706,057 ± 0.00000	1,411,072 ± 0.00000	10.34689 ± 0.44999

ViT

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.70635 ± 0.04166	0.67407 ± 0.02019	0.68642 ± 0.02243	0.68280 ± 0.01561	0.70297 ± 0.01509	0.68642 ± 0.02243	0.93421 ± 0.00295	0.93329 ± 0.01252	0.65048 ± 0.02538	3,142,689 ± 0.00000	31,518,296 ± 0.00000	49.77751 ± 2.40462
IGTD	0.99868 ± 0.00296	0.92593 ± 0.01852	0.93827 ± 0.01310	0.93838 ± 0.01313	0.94212 ± 0.01160	0.93827 ± 0.01310	0.99494 ± 0.00151	0.23574 ± 0.03603	0.93106 ± 0.01450	476,097 ± 0.00000	9,725,388 ± 0.00000	26.78283 ± 1.99133
REFINED	0.99550 ± 0.00865	0.93210 ± 0.01574	0.92346 ± 0.02849	0.92387 ± 0.02848	0.93110 ± 0.02637	0.92346 ± 0.02849	0.99342 ± 0.00305	0.32760 ± 0.05950	0.91483 ± 0.03174	1,269,455 ± 0.00000	12,377,946 ± 0.00000	51.73347 ± 8.33281
BarGraph	0.99709 ± 0.00367	0.92716 ± 0.00916	0.93333 ± 0.01821	0.93401 ± 0.01758	0.94037 ± 0.01197	0.93333 ± 0.01821	0.99698 ± 0.00196	0.22455 ± 0.07629	0.92588 ± 0.01960	214,697,577 ± 0.00000	1,724,899,336 ± 0.00000	378.47980 ± 12.39942
SuperTML	0.39365 ± 0.08137	0.39506 ± 0.06984	0.39876 ± 0.06319	0.34760 ± 0.07799	0.47892 ± 0.09961	0.39876 ± 0.06319	0.82871 ± 0.02798	1.51644 ± 0.14370	0.37221 ± 0.02346	2,499,945 ± 0.00000	251,634,112 ± 0.00000	32.09513 ± 0.48480
FeatureWrap	0.96746 ± 0.01174	0.81605 ± 0.01187	0.75926 ± 0.01746	0.76072 ± 0.01852	0.79167 ± 0.01411	0.75926 ± 0.01746	0.97051 ± 0.00461	0.77868 ± 0.08332	0.73310 ± 0.01861	17,470,447 ± 0.00000	174,454,188 ± 0.00000	78.42049 ± 11.15532

ViT + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99815 ± 0.00221	0.95679 ± 0.00756	0.96049 ± 0.01883	0.96042 ± 0.01906	0.96218 ± 0.01829	0.96049 ± 0.01883	0.99859 ± 0.00162	0.13929 ± 0.07387	0.95578 ± 0.02106	3,893,025 ± 0.00000	464,977,088 ± 91,899,166.11703	51.20903 ± 5.63295
IGTD	0.99815 ± 0.00274	0.95803 ± 0.00916	0.95556 ± 0.01408	0.95568 ± 0.01408	0.95874 ± 0.01229	0.95556 ± 0.01408	0.99784 ± 0.00181	0.22177 ± 0.09474	0.95040 ± 0.01557	1,218,401 ± 0.00000	269,517,528 ± 23,674,701.60606	36.43950 ± 1.83848
REFINED	0.99974 ± 0.00059	0.95803 ± 0.00517	0.96173 ± 0.00517	0.96194 ± 0.00487	0.96512 ± 0.00333	0.96173 ± 0.00517	0.99862 ± 0.00102	0.14492 ± 0.04696	0.95734 ± 0.00561	1,989,455 ± 0.00000	648,339,608 ± 36,611,247.47466	75.39355 ± 2.91066
SuperTML	0.99947 ± 0.00072	0.97037 ± 0.00916	0.97037 ± 0.00517	0.97035 ± 0.00532	0.97305 ± 0.00492	0.97037 ± 0.00517	0.99973 ± 0.00005	0.09663 ± 0.01046	0.96701 ± 0.00575	3,207,145 ± 0.00000	10,130,914,304 ± 421,536,368.16605	42.45165 ± 1.58963
FeatureWrap	0.98783 ± 0.01648	0.94074 ± 0.03255	0.94568 ± 0.01598	0.94587 ± 0.01549	0.95165 ± 0.01262	0.94568 ± 0.01598	0.99831 ± 0.00069	0.21044 ± 0.05732	0.93962 ± 0.01761	18,746,639 ± 0.00000	2,513,403,744 ± 496,755,031.91467	94.19557 ± 3.95154

CNN

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.79100 ± 0.01282	0.67654 ± 0.01203	0.68148 ± 0.01549	0.68115 ± 0.01536	0.69369 ± 0.01499	0.68148 ± 0.01549	0.93206 ± 0.00367	0.96300 ± 0.03732	0.64317 ± 0.01736	77,937 ± 0.00000	3,614,528 ± 0.00000	18.50549 ± 0.88259
IGTD	1.00000 ± 0.00000	0.86914 ± 0.02908	0.93704 ± 0.01537	0.93723 ± 0.01538	0.94180 ± 0.01521	0.93704 ± 0.01537	0.99752 ± 0.00138	0.22672 ± 0.07117	0.92973 ± 0.01727	190,897 ± 0.00000	22,969,392 ± 0.00000	23.99849 ± 3.09947
REFINED	0.99180 ± 0.00616	0.92099 ± 0.02484	0.91358 ± 0.00756	0.91345 ± 0.00719	0.92283 ± 0.00767	0.91358 ± 0.00756	0.99358 ± 0.00182	0.37951 ± 0.03106	0.90395 ± 0.00863	733,137 ± 0.00000	63,914,592 ± 0.00000	39.24781 ± 1.83563
SuperTML	1.00000 ± 0.00000	0.92469 ± 0.00805	0.94691 ± 0.00552	0.94697 ± 0.00573	0.95051 ± 0.00624	0.94691 ± 0.00552	0.99771 ± 0.00068	0.23146 ± 0.01665	0.94072 ± 0.00625	2,081,961 ± 0.00000	2,760,667,584 ± 0.00000	155.72271 ± 0.08343
FeatureWrap	0.99418 ± 0.00424	0.91975 ± 0.01512	0.88148 ± 0.01104	0.88317 ± 0.01134	0.89928 ± 0.01478	0.88148 ± 0.01104	0.99107 ± 0.00173	0.53426 ± 0.04917	0.86863 ± 0.01292	11,985,833 ± 0.00000	1,669,530,816 ± 0.00000	77.13566 ± 0.42878

CNN + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99762 ± 0.00217	0.94938 ± 0.01598	0.96420 ± 0.00276	0.96424 ± 0.00263	0.96609 ± 0.00225	0.96420 ± 0.00276	0.99911 ± 0.00047	0.10987 ± 0.02834	0.95995 ± 0.00305	820,465 ± 0.00000	56,082,752 ± 0.00000	29.90100 ± 2.27126
IGTD	0.99894 ± 0.00237	0.94815 ± 0.01831	0.95556 ± 0.00805	0.95514 ± 0.00828	0.95896 ± 0.00707	0.95556 ± 0.00805	0.99932 ± 0.00017	0.16153 ± 0.04349	0.95052 ± 0.00886	1,067,889 ± 0.00000	271,943,232 ± 0.00000	30.57734 ± 1.64419
REFINED	0.99868 ± 0.00229	0.94815 ± 0.01487	0.95185 ± 0.01187	0.95199 ± 0.01212	0.95589 ± 0.01141	0.95185 ± 0.01187	0.99915 ± 0.00040	0.15621 ± 0.02966	0.94634 ± 0.01320	1,422,257 ± 0.00000	718,212,000 ± 0.00000	42.14406 ± 2.49778
SuperTML	0.99868 ± 0.00187	0.95309 ± 0.00936	0.96543 ± 0.01033	0.96571 ± 0.00999	0.96891 ± 0.00881	0.96543 ± 0.01033	0.99940 ± 0.00038	0.13416 ± 0.03825	0.96151 ± 0.01145	2,804,841 ± 0.00000	30,383,234,816 ± 0.00000	157.23238 ± 0.21957
FeatureWrap	0.99815 ± 0.00221	0.95803 ± 0.01014	0.93580 ± 0.01421	0.93599 ± 0.01373	0.94377 ± 0.01086	0.93580 ± 0.01421	0.99883 ± 0.00059	0.22689 ± 0.08034	0.92877 ± 0.01565	12,981,545 ± 0.00000	18,386,727,744 ± 0.00000	97.10381 ± 4.24365

Connect-4

Source: OpenML · Original

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
Trees	XGBoost	0.86860 ± 0.00059	0.73852 ± 0.00118	7.46423 ± 0.41959	—	—
MLP	MLP	0.86300 ± 0.00099	0.85441 ± 0.00174	137.17053 ± 1.67854	66,563 ± 0.00000	132,608 ± 0.00000
ViT	Combination	0.85852 ± 0.00537	0.85227 ± 0.00557	6,459.70419 ± 737.80948	14,165,371 ± 0.00000	4,403,208,502,428.7998 ± 9,719,427,985,675.61133
ViT+MLP	IGTD	0.86819 ± 0.00221	0.86365 ± 0.00250	3,294.58531 ± 943.89277	19,592,315 ± 0.00000	1,226,724,971,052.80005 ± 2,725,978,272,247.06592
CNN	BIE	0.85437 ± 0.00345	0.84537 ± 0.00593	4,724.28435 ± 433.84260	1,606,067 ± 0.00000	1,189,191,183,379.19995 ± 2,597,463,716,358.88281
CNN+MLP	BarGraph	0.85382 ± 0.00278	0.84488 ± 0.00467	4,804.959 ± 449.91699	1,163,859 ± 0.00000	519,475,749,942.40002 ± 1,139,923,846,571.62622

Architecture Results

Tree Baselines

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Train time (s)	#Params	FLOPs
CatBoost	0.93083 ± 0.00226	0.86303 ± 0.00121	0.85506 ± 0.00133	0.70725 ± 0.00577	0.74560 ± 0.00578	0.68951 ± 0.00424	181.61038 ± 1.83578	—	—
LightGBM	0.94523 ± 0.00000	0.86452 ± 0.00000	0.85554 ± 0.00000	0.71701 ± 0.00000	0.74147 ± 0.00000	0.70200 ± 0.00000	3.30468 ± 0.40202	—	—
XGBoost	0.98862 ± 0.00038	0.87279 ± 0.00098	0.86860 ± 0.00059	0.73852 ± 0.00118	0.76687 ± 0.00292	0.72174 ± 0.00139	7.46423 ± 0.41959	—	—

MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
MLP	0.93258 ± 0.00333	0.87249 ± 0.00218	0.86300 ± 0.00099	0.85441 ± 0.00174	0.85070 ± 0.00185	0.86300 ± 0.00099	0.95992 ± 0.00038	0.35456 ± 0.00105	0.71707 ± 0.00233	66,563 ± 0.00000	132,608 ± 0.00000	137.17053 ± 1.67854

ViT

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.71345 ± 0.00240	0.69485 ± 0.00150	0.69743 ± 0.00233	0.65065 ± 0.00331	0.65303 ± 0.00402	0.69743 ± 0.00233	0.77163 ± 0.00114	0.70665 ± 0.00229	0.29669 ± 0.00618	8,472,451 ± 0.00000	899,883,414,152.80005 ± 2,001,077,131,734.39478	3,734.34874 ± 970.17133
IGTD	0.90747 ± 0.01085	0.86424 ± 0.00429	0.85410 ± 0.00323	0.84736 ± 0.00454	0.84377 ± 0.00487	0.85410 ± 0.00323	0.95671 ± 0.00134	0.38029 ± 0.00345	0.70014 ± 0.00722	19,939,899 ± 0.00000	498,074,710,800 ± 1,106,963,680,989.76514	2,967.25645 ± 864.50463
REFINED	0.90624 ± 0.01178	0.86136 ± 0.00258	0.85717 ± 0.00204	0.85050 ± 0.00308	0.84701 ± 0.00320	0.85717 ± 0.00204	0.95768 ± 0.00105	0.35784 ± 0.00402	0.70651 ± 0.00349	6,339,465 ± 0.00000	7,611,645,317,385.59961 ± 16,910,428,256,738.60742	4,837.46139 ± 113.28264
DistanceMatrix	0.91426 ± 0.01254	0.86442 ± 0.00132	0.85777 ± 0.00343	0.85384 ± 0.00386	0.85109 ± 0.00423	0.85777 ± 0.00343	0.95841 ± 0.00125	0.37113 ± 0.00475	0.71012 ± 0.00675	15,615,579 ± 0.00000	353,206,830,993.59998 ± 767,325,631,115.39844	5,650.87453 ± 828.55672
BarGraph	0.90582 ± 0.01062	0.85976 ± 0.00390	0.85313 ± 0.00217	0.84707 ± 0.00316	0.84343 ± 0.00380	0.85313 ± 0.00217	0.95580 ± 0.00126	0.37646 ± 0.00563	0.69867 ± 0.00499	7,677,019 ± 0.00000	260,700,949,174.39999 ± 573,462,935,121.85742	4,836.73781 ± 350.10447
Combination	0.91602 ± 0.01066	0.86643 ± 0.00471	0.85852 ± 0.00537	0.85227 ± 0.00557	0.84892 ± 0.00580	0.85852 ± 0.00537	0.95984 ± 0.00207	0.36214 ± 0.00596	0.70891 ± 0.00975	14,165,371 ± 0.00000	4,403,208,502,428.7998 ± 9,719,427,985,675.61133	6,459.70419 ± 737.80948
SuperTML	0.90546 ± 0.00765	0.85660 ± 0.00373	0.84626 ± 0.00304	0.84026 ± 0.00287	0.83728 ± 0.00336	0.84626 ± 0.00304	0.95164 ± 0.00133	0.38937 ± 0.00557	0.68451 ± 0.00828	17,791,747 ± 0.00000	433,472,959,452 ± 953,674,396,877.40051	7,495.51417 ± 870.92907
FeatureWrap	0.91074 ± 0.00977	0.86006 ± 0.00228	0.85054 ± 0.00233	0.84247 ± 0.00421	0.83902 ± 0.00431	0.85054 ± 0.00233	0.95429 ± 0.00065	0.38311 ± 0.00555	0.69225 ± 0.00492	2,540,315 ± 0.00000	178,853,395,445.60001 ± 395,507,779,949.31891	2,406.81428 ± 636.98173
BIE	0.90568 ± 0.01360	0.86290 ± 0.00319	0.85528 ± 0.00200	0.84804 ± 0.00316	0.84415 ± 0.00328	0.85528 ± 0.00200	0.95760 ± 0.00158	0.36426 ± 0.00243	0.70120 ± 0.00352	13,433,115 ± 0.00000	1,045,166,154,542.40002 ± 2,307,449,081,091.5	5,775.03304 ± 305.90916

ViT + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.93435 ± 0.00783	0.87204 ± 0.00182	0.86282 ± 0.00318	0.85837 ± 0.00369	0.85545 ± 0.00413	0.86282 ± 0.00318	0.96241 ± 0.00079	0.34073 ± 0.00374	0.71943 ± 0.00699	8,640,483 ± 0.00000	912,817,909,791.19995 ± 2,020,171,485,487.94141	4,188.90154 ± 1,125.38759
IGTD	0.94931 ± 0.00681	0.87547 ± 0.00193	0.86819 ± 0.00221	0.86365 ± 0.00250	0.86086 ± 0.00265	0.86819 ± 0.00221	0.96399 ± 0.00093	0.33538 ± 0.00437	0.73041 ± 0.00378	19,592,315 ± 0.00000	1,226,724,971,052.80005 ± 2,725,978,272,247.06592	3,294.58531 ± 943.89277
REFINED	0.92572 ± 0.00830	0.87069 ± 0.00116	0.86227 ± 0.00112	0.85416 ± 0.00120	0.85076 ± 0.00105	0.86227 ± 0.00112	0.95921 ± 0.00082	0.35667 ± 0.00236	0.71667 ± 0.00202	6,258,761 ± 0.00000	14,030,313,714,328 ± 29,438,466,150,706.24609	5,091.08685 ± 192.75738
DistanceMatrix	0.93294 ± 0.00696	0.87158 ± 0.00231	0.86499 ± 0.00201	0.86004 ± 0.00229	0.85688 ± 0.00247	0.86499 ± 0.00201	0.96225 ± 0.00082	0.33958 ± 0.00148	0.72275 ± 0.00374	15,258,651 ± 0.00000	1,364,578,640,235.19995 ± 3,011,336,350,380.71533	6,505.18565 ± 663.34706
BarGraph	0.94892 ± 0.00686	0.87454 ± 0.00135	0.86663 ± 0.00173	0.86171 ± 0.00198	0.85867 ± 0.00222	0.86663 ± 0.00173	0.96328 ± 0.00055	0.33884 ± 0.00413	0.72670 ± 0.00392	7,939,931 ± 0.00000	440,659,712,818.40002 ± 968,471,046,722.47913	5,125.7472 ± 535.15380
Combination	0.93725 ± 0.00482	0.87318 ± 0.00152	0.86489 ± 0.00139	0.85968 ± 0.00188	0.85646 ± 0.00213	0.86489 ± 0.00139	0.96266 ± 0.00101	0.33884 ± 0.00309	0.72284 ± 0.00315	14,093,339 ± 0.00000	7,161,710,259,286.40039 ± 15,826,396,161,415.5957	6,977.73327 ± 910.96092
SuperTML	0.92000 ± 0.00518	0.86990 ± 0.00179	0.86140 ± 0.00154	0.85271 ± 0.00345	0.84871 ± 0.00379	0.86140 ± 0.00154	0.96065 ± 0.00079	0.34802 ± 0.00327	0.71263 ± 0.00454	17,883,843 ± 0.00000	963,737,054,338.40002 ± 2,126,847,466,896.84741	10,616.86089 ± 2,455.20884
FeatureWrap	0.93592 ± 0.00888	0.87239 ± 0.00134	0.86357 ± 0.00130	0.85749 ± 0.00301	0.85407 ± 0.00360	0.86357 ± 0.00130	0.96204 ± 0.00118	0.34445 ± 0.00416	0.71896 ± 0.00308	2,646,683 ± 0.00000	1,246,668,889,986.3999 ± 2,737,963,693,406.41064	5,269.05582 ± 1,853.4156

CNN

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.71220 ± 0.00259	0.69657 ± 0.00198	0.69503 ± 0.00269	0.64016 ± 0.00317	0.65104 ± 0.00801	0.69503 ± 0.00269	0.76540 ± 0.00230	0.71344 ± 0.00330	0.27740 ± 0.00693	133,307 ± 0.00000	13,548,962,240 ± 30,106,367,691.06504	1,148.86206 ± 355.98040
IGTD	0.90703 ± 0.02432	0.85159 ± 0.00558	0.84454 ± 0.00520	0.83759 ± 0.00997	0.83480 ± 0.01118	0.84454 ± 0.00520	0.95032 ± 0.00418	0.39911 ± 0.00688	0.67885 ± 0.01424	3,572,451 ± 0.00000	210,226,552,876.79999 ± 382,063,806,970.60919	1,462.19425 ± 461.35553
REFINED	0.89683 ± 0.01074	0.85354 ± 0.00279	0.84342 ± 0.00310	0.83448 ± 0.00458	0.83022 ± 0.00499	0.84342 ± 0.00310	0.94878 ± 0.00256	0.39481 ± 0.00996	0.67599 ± 0.00773	3,063,971 ± 0.00000	159,951,752,448 ± 283,078,091,946.64594	1,464.93873 ± 470.53684
DistanceMatrix	0.90649 ± 0.02135	0.85256 ± 0.00489	0.84616 ± 0.00208	0.83696 ± 0.00625	0.83460 ± 0.00646	0.84616 ± 0.00208	0.95029 ± 0.00294	0.39819 ± 0.01003	0.68102 ± 0.00752	1,708,867 ± 0.00000	3,491,576,535,427.2002 ± 7,707,901,491,666.37207	4,964.70239 ± 123.36804
BarGraph	0.90792 ± 0.01282	0.85946 ± 0.00354	0.85413 ± 0.00312	0.84797 ± 0.00422	0.84495 ± 0.00413	0.85413 ± 0.00312	0.95584 ± 0.00151	0.37566 ± 0.00614	0.70135 ± 0.00551	1,110,579 ± 0.00000	311,607,712,003.20001 ± 679,867,503,774.60254	4,512.3646 ± 349.22010
Combination	0.90712 ± 0.01527	0.85490 ± 0.00365	0.84616 ± 0.00195	0.84059 ± 0.00398	0.83862 ± 0.00432	0.84616 ± 0.00195	0.95210 ± 0.00160	0.39418 ± 0.00743	0.68624 ± 0.00321	3,359,267 ± 0.00000	3,728,836,729,651.2002 ± 7,992,908,531,108.56445	7,634.51986 ± 404.33011
SuperTML	0.90867 ± 0.00832	0.86392 ± 0.00210	0.85228 ± 0.00144	0.84541 ± 0.00262	0.84210 ± 0.00250	0.85228 ± 0.00144	0.95521 ± 0.00071	0.37773 ± 0.00645	0.69723 ± 0.00308	5,266,899 ± 0.00000	2,141,033,393,254.3999 ± 4,678,190,253,192.8623	7,420.98223 ± 739.91857
FeatureWrap	0.90654 ± 0.01187	0.85410 ± 0.00371	0.85060 ± 0.00232	0.84471 ± 0.00535	0.84197 ± 0.00687	0.85060 ± 0.00232	0.95319 ± 0.00077	0.38091 ± 0.00653	0.69344 ± 0.00718	2,757,747 ± 0.00000	375,817,308,748.79999 ± 829,679,119,080.63367	1,686.67107 ± 572.40545
BIE	0.90377 ± 0.01335	0.85891 ± 0.00386	0.85437 ± 0.00345	0.84537 ± 0.00593	0.84220 ± 0.00583	0.85437 ± 0.00345	0.95548 ± 0.00273	0.37063 ± 0.00619	0.69920 ± 0.00932	1,606,067 ± 0.00000	1,189,191,183,379.19995 ± 2,597,463,716,358.88281	4,724.28435 ± 433.84260

CNN + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.89611 ± 0.00795	0.85686 ± 0.00275	0.84997 ± 0.00207	0.83822 ± 0.00473	0.83459 ± 0.00466	0.84997 ± 0.00207	0.95321 ± 0.00126	0.37265 ± 0.00327	0.68654 ± 0.00690	378,283 ± 0.00000	11,118,993,590.4 ± 24,616,967,503.58926	1,352.04697 ± 421.20323
IGTD	0.89933 ± 0.01484	0.84849 ± 0.00475	0.84190 ± 0.00583	0.83342 ± 0.00745	0.82935 ± 0.00783	0.84190 ± 0.00583	0.94754 ± 0.00211	0.40304 ± 0.00608	0.67222 ± 0.01395	3,675,971 ± 0.00000	208,189,100,467.20001 ± 461,916,824,304.87598	1,643.26583 ± 530.95584
REFINED	0.91310 ± 0.01035	0.84983 ± 0.00300	0.84440 ± 0.00274	0.83628 ± 0.00393	0.83282 ± 0.00401	0.84440 ± 0.00274	0.94989 ± 0.00070	0.39970 ± 0.01150	0.67935 ± 0.00697	3,128,355 ± 0.00000	175,697,881,356.79999 ± 389,773,617,625.3717	1,751.01563 ± 586.83492
DistanceMatrix	0.88316 ± 0.00763	0.85340 ± 0.00248	0.84664 ± 0.00431	0.83343 ± 0.00459	0.83062 ± 0.00448	0.84664 ± 0.00431	0.94989 ± 0.00090	0.39532 ± 0.01196	0.68039 ± 0.01146	1,723,971 ± 0.00000	5,095,256,195,558.40039 ± 11,211,795,084,795.75391	5,155.71169 ± 232.70114
BarGraph	0.89839 ± 0.01132	0.86067 ± 0.00470	0.85382 ± 0.00278	0.84488 ± 0.00467	0.84167 ± 0.00373	0.85382 ± 0.00278	0.95566 ± 0.00225	0.37123 ± 0.00794	0.69766 ± 0.00579	1,163,859 ± 0.00000	519,475,749,942.40002 ± 1,139,923,846,571.62622	4,804.959 ± 449.91699
Combination	0.89388 ± 0.00351	0.85254 ± 0.00228	0.84843 ± 0.00255	0.83873 ± 0.00320	0.83518 ± 0.00400	0.84843 ± 0.00255	0.95196 ± 0.00089	0.38681 ± 0.00683	0.68724 ± 0.00706	2,971,939 ± 0.00000	15,843,091,900,857.59961 ± 34,910,752,920,593.26172	7,442.73934 ± 63.86921
SuperTML	0.89551 ± 0.00672	0.86323 ± 0.00234	0.85281 ± 0.00075	0.84373 ± 0.00471	0.84058 ± 0.00417	0.85281 ± 0.00075	0.95540 ± 0.00103	0.36847 ± 0.00559	0.69559 ± 0.00266	5,463,507 ± 0.00000	5,610,715,454,259.2002 ± 12,342,820,772,949.64648	7,634.20695 ± 750.53481
FeatureWrap	0.90022 ± 0.01044	0.85828 ± 0.00255	0.85222 ± 0.00345	0.84230 ± 0.00557	0.83890 ± 0.00589	0.85222 ± 0.00345	0.95344 ± 0.00089	0.38222 ± 0.00544	0.69375 ± 0.00994	2,881,203 ± 0.00000	661,479,649,996.80005 ± 1,253,726,830,906.76416	2,375.2474 ± 866.19194
BIE	0.89989 ± 0.00965	0.86069 ± 0.00232	0.85295 ± 0.00430	0.84680 ± 0.00407	0.84559 ± 0.00428	0.85295 ± 0.00430	0.95687 ± 0.00223	0.37188 ± 0.00807	0.70102 ± 0.00779	1,730,419 ± 0.00000	170,941,288,227.20001 ± 374,187,299,306.86566	4,251.9051 ± 227.82844

Covtype

Source: UCI Covertype / OpenML

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
ViT+MLP	REFINED	0.97050 ± 0.00000	0.97051 ± 0.00000	4491.06401 ± 0.00000	—	—
CNN	Combination	0.96485 ± 0.00000	0.96487 ± 0.00000	4482.55339 ± 0.00000	—	—
MLP	MLP	0.96310 ± 0.00000	0.96312 ± 0.00000	2870.39325 ± 0.00000	—	—
CNN+MLP	REFINED	0.96289 ± 0.00000	0.96284 ± 0.00000	1832.13226 ± 0.00000	—	—
ViT	DistanceMatrix	0.96078 ± 0.00000	0.96075 ± 0.00000	7455.48702 ± 0.00000	—	—

Architecture Results

MLP

Method	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
MLP	0.96310 ± 0.00000	0.96312 ± 0.00000	2870.39325 ± 0.00000	—	—

ViT

Method	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
TINTO	0.95210 ± 0.00000	0.95211 ± 0.00000	3717.71217 ± 0.00000	—	—
IGTD	0.93557 ± 0.00000	0.93560 ± 0.00000	4028.44666 ± 0.00000	—	—
REFINED	0.94441 ± 0.00000	0.94441 ± 0.00000	3636.44167 ± 0.00000	—	—
BarGraph	0.95900 ± 0.00000	0.95899 ± 0.00000	3740.85926 ± 0.00000	—	—
FeatureWrap	0.94988 ± 0.00000	0.94983 ± 0.00000	3783.60048 ± 0.00000	—	—
Combination	0.94756 ± 0.00000	0.94756 ± 0.00000	3769.21102 ± 0.00000	—	—
DistanceMatrix	0.96078 ± 0.00000	0.96075 ± 0.00000	7455.48702 ± 0.00000	—	—

ViT + MLP

Method	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
TINTO	0.95972 ± 0.00000	0.95972 ± 0.00000	4179.14185 ± 0.00000	—	—
IGTD	0.96671 ± 0.00000	0.96671 ± 0.00000	4298.49003 ± 0.00000	—	—
REFINED	0.97050 ± 0.00000	0.97051 ± 0.00000	4491.06401 ± 0.00000	—	—
BarGraph	0.95683 ± 0.00000	0.95682 ± 0.00000	4305.19298 ± 0.00000	—	—
FeatureWrap	0.95517 ± 0.00000	0.95518 ± 0.00000	4388.84096 ± 0.00000	—	—
Combination	0.95903 ± 0.00000	0.95903 ± 0.00000	4331.18001 ± 0.00000	—	—
DistanceMatrix	0.96499 ± 0.00000	0.96499 ± 0.00000	4372.84549 ± 0.00000	—	—

CNN

Method	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
TINTO	0.95707 ± 0.00000	0.95707 ± 0.00000	4064.05686 ± 0.00000	—	—
IGTD	0.95949 ± 0.00000	0.95949 ± 0.00000	4029.40301 ± 0.00000	—	—
REFINED	0.96206 ± 0.00000	0.96207 ± 0.00000	4004.55836 ± 0.00000	—	—
BarGraph	0.96467 ± 0.00000	0.96467 ± 0.00000	3929.04679 ± 0.00000	—	—
FeatureWrap	0.94784 ± 0.00000	0.94780 ± 0.00000	3922.66296 ± 0.00000	—	—
Combination	0.96485 ± 0.00000	0.96487 ± 0.00000	4482.55339 ± 0.00000	—	—
DistanceMatrix	0.95455 ± 0.00000	0.95455 ± 0.00000	4027.21556 ± 0.00000	—	—
BIE	0.95304 ± 0.00000	0.95301 ± 0.00000	3952.53097 ± 0.00000	—	—

CNN + MLP

Method	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
TINTO	0.95034 ± 0.00000	0.95032 ± 0.00000	1772.66925 ± 0.00000	—	—
IGTD	0.94519 ± 0.00000	0.94519 ± 0.00000	1722.16181 ± 0.00000	—	—
REFINED	0.96289 ± 0.00000	0.96284 ± 0.00000	1832.13226 ± 0.00000	—	—
BarGraph	0.93826 ± 0.00000	0.93826 ± 0.00000	1745.82248 ± 0.00000	—	—
FeatureWrap	0.94837 ± 0.00000	0.94836 ± 0.00000	1655.45252 ± 0.00000	—	—
Combination	0.94017 ± 0.00000	0.94018 ± 0.00000	1712.62713 ± 0.00000	—	—
DistanceMatrix	0.96089 ± 0.00000	0.96089 ± 0.00000	1734.12690 ± 0.00000	—	—
BIE	0.90212 ± 0.00000	0.90207 ± 0.00000	1684.29330 ± 0.00000	—	—

Dna

Source: OpenML · Original

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
Trees	LightGBM	0.96904 ± 0.00229	0.96586 ± 0.00223	0.83394 ± 0.08025	—	—
MLP	MLP	0.95900 ± 0.00919	0.95908 ± 0.00924	17.34056 ± 0.04865	125,699 ± 0.00000	251,008 ± 0.00000
ViT	IGTD	0.96276 ± 0.00716	0.96270 ± 0.00716	288.60882 ± 6.51746	20,028,443 ± 0.00000	3,916,903,412 ± 0.00000
ViT+MLP	IGTD	0.96569 ± 0.00482	0.96579 ± 0.00484	296.84681 ± 1.41752	19,147,355 ± 0.00000	73,330,566,120 ± 6,441,433,640.24378
CNN	REFINED	0.94686 ± 0.01423	0.94686 ± 0.01444	79.57766 ± 4.85588	1,364,507 ± 0.00000	155,817,264 ± 0.00000
CNN+MLP	FeatureWrap	0.94812 ± 0.00581	0.94823 ± 0.00591	51.14995 ± 2.83025	2,817,195 ± 0.00000	403,004,000 ± 0.00000

Architecture Results

Tree Baselines

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Train time (s)	#Params	FLOPs
CatBoost	0.99955 ± 0.00000	0.97155 ± 0.00238	0.96653 ± 0.00148	0.96349 ± 0.00178	0.95981 ± 0.00158	0.96761 ± 0.00205	4.20438 ± 0.08675	—	—
LightGBM	0.99910 ± 0.00000	0.97280 ± 0.00209	0.96904 ± 0.00229	0.96586 ± 0.00223	0.96290 ± 0.00285	0.96923 ± 0.00147	0.83394 ± 0.08025	—	—
XGBoost	0.99955 ± 0.00000	0.96946 ± 0.00350	0.96778 ± 0.00317	0.96469 ± 0.00376	0.96042 ± 0.00381	0.96966 ± 0.00389	2.38268 ± 0.05833	—	—

MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
MLP	0.98655 ± 0.00678	0.96234 ± 0.00936	0.95900 ± 0.00919	0.95908 ± 0.00924	0.95966 ± 0.00902	0.95900 ± 0.00919	0.99520 ± 0.00088	0.12762 ± 0.01930	0.93382 ± 0.01495	125,699 ± 0.00000	251,008 ± 0.00000	17.34056 ± 0.04865

ViT

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.94386 ± 0.01146	0.92008 ± 0.00880	0.91716 ± 0.00688	0.91752 ± 0.00719	0.91880 ± 0.00762	0.91716 ± 0.00688	0.98487 ± 0.00164	0.22577 ± 0.01743	0.86639 ± 0.01171	6,476,985 ± 0.00000	217,466,360 ± 0.00000	87.84516 ± 0.66980
IGTD	0.98834 ± 0.00603	0.96402 ± 0.00760	0.96276 ± 0.00716	0.96270 ± 0.00716	0.96286 ± 0.00708	0.96276 ± 0.00716	0.99547 ± 0.00114	0.13422 ± 0.02999	0.93947 ± 0.01162	20,028,443 ± 0.00000	3,916,903,412 ± 0.00000	288.60882 ± 6.51746
REFINED	0.98395 ± 0.00394	0.96695 ± 0.00477	0.95481 ± 0.00545	0.95494 ± 0.00543	0.95543 ± 0.00526	0.95481 ± 0.00545	0.99211 ± 0.00159	0.14318 ± 0.01237	0.92712 ± 0.00863	7,690,243 ± 0.00000	403,679,040 ± 0.00000	201.60364 ± 25.39346
FeatureWrap	0.97300 ± 0.00985	0.95481 ± 0.00845	0.95607 ± 0.00769	0.95602 ± 0.00779	0.95627 ± 0.00769	0.95607 ± 0.00769	0.99324 ± 0.00119	0.14132 ± 0.01830	0.92891 ± 0.01242	3,159,771 ± 0.00000	1,057,233,840 ± 0.00000	121.53474 ± 1.87864

ViT + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.98475 ± 0.01014	0.96736 ± 0.00482	0.95941 ± 0.00317	0.95947 ± 0.00319	0.95972 ± 0.00328	0.95941 ± 0.00317	0.99300 ± 0.00060	0.13922 ± 0.00394	0.93437 ± 0.00525	6,584,217 ± 0.00000	2,153,332,160 ± 425,589,636.53063	118.35021 ± 1.94925
IGTD	0.99004 ± 0.00588	0.96611 ± 0.00273	0.96569 ± 0.00482	0.96579 ± 0.00484	0.96613 ± 0.00494	0.96569 ± 0.00482	0.99354 ± 0.00115	0.14215 ± 0.02601	0.94442 ± 0.00788	19,147,355 ± 0.00000	73,330,566,120 ± 6,441,433,640.24378	296.84681 ± 1.41752
REFINED	0.98170 ± 0.00484	0.96234 ± 0.00512	0.95481 ± 0.00281	0.95489 ± 0.00282	0.95544 ± 0.00277	0.95481 ± 0.00281	0.99164 ± 0.00130	0.16187 ± 0.01319	0.92705 ± 0.00455	7,810,659 ± 0.00000	3,738,010,240 ± 738,789,142.21454	222.49335 ± 4.77872
FeatureWrap	0.98735 ± 0.00753	0.96444 ± 0.00331	0.96276 ± 0.00541	0.96284 ± 0.00540	0.96315 ± 0.00529	0.96276 ± 0.00541	0.99460 ± 0.00078	0.12526 ± 0.01582	0.93986 ± 0.00862	3,255,611 ± 0.00000	19,644,640,128 ± 1,725,605,739.41171	136.39478 ± 4.12009

CNN

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.98924 ± 0.00609	0.92929 ± 0.00855	0.91590 ± 0.00541	0.91597 ± 0.00567	0.91649 ± 0.00601	0.91590 ± 0.00541	0.98324 ± 0.00229	0.24036 ± 0.00774	0.86376 ± 0.00940	5,670,235 ± 0.00000	532,993,344 ± 0.00000	69.24992 ± 2.75125
IGTD	0.99534 ± 0.00345	0.94561 ± 0.00709	0.93264 ± 0.00829	0.93242 ± 0.00847	0.93302 ± 0.00852	0.93264 ± 0.00829	0.98925 ± 0.00470	0.23842 ± 0.06370	0.89060 ± 0.01387	3,382,019 ± 0.00000	312,868,128 ± 0.00000	52.91997 ± 1.50567
REFINED	0.99552 ± 0.00597	0.94979 ± 0.01097	0.94686 ± 0.01423	0.94686 ± 0.01444	0.94714 ± 0.01436	0.94686 ± 0.01423	0.99071 ± 0.00272	0.20894 ± 0.01598	0.91375 ± 0.02348	1,364,507 ± 0.00000	155,817,264 ± 0.00000	79.57766 ± 4.85588
FeatureWrap	0.98413 ± 0.02596	0.94519 ± 0.02964	0.93347 ± 0.02908	0.93390 ± 0.02835	0.93578 ± 0.02526	0.93347 ± 0.02908	0.98649 ± 0.00910	0.26083 ± 0.05225	0.89384 ± 0.04392	2,676,523 ± 0.00000	402,723,072 ± 0.00000	41.64876 ± 0.55029

CNN + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.98206 ± 0.00827	0.95816 ± 0.00491	0.94603 ± 0.01121	0.94591 ± 0.01130	0.94680 ± 0.01035	0.94603 ± 0.01121	0.99321 ± 0.00149	0.16841 ± 0.02508	0.91261 ± 0.01786	5,569,563 ± 0.00000	532,792,224 ± 0.00000	72.98984 ± 1.24142
IGTD	0.99937 ± 0.00040	0.95398 ± 0.00754	0.94812 ± 0.00229	0.94815 ± 0.00223	0.94840 ± 0.00199	0.94812 ± 0.00229	0.99275 ± 0.00147	0.18653 ± 0.01943	0.91590 ± 0.00341	3,507,715 ± 0.00000	313,119,136 ± 0.00000	58.24372 ± 1.41547
REFINED	0.99713 ± 0.00108	0.95105 ± 0.01031	0.93807 ± 0.00832	0.93811 ± 0.00845	0.93929 ± 0.00898	0.93807 ± 0.00832	0.98941 ± 0.00117	0.21958 ± 0.02508	0.90001 ± 0.01415	1,490,203 ± 0.00000	156,068,272 ± 0.00000	75.13162 ± 3.41050
FeatureWrap	0.99596 ± 0.00394	0.94895 ± 0.00671	0.94812 ± 0.00581	0.94823 ± 0.00591	0.94874 ± 0.00614	0.94812 ± 0.00581	0.99045 ± 0.00118	0.22568 ± 0.03002	0.91610 ± 0.00977	2,817,195 ± 0.00000	403,004,000 ± 0.00000	51.14995 ± 2.83025

Gas

Source: UCI · OpenML

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
Trees	CatBoost	0.99482 ± 0.00021	0.99480 ± 0.00026	36.80809 ± 0.25908	—	—
MLP	MLP	0.99023 ± 0.00055	0.99023 ± 0.00055	46.54599 ± 0.11946	198,918 ± 0.00000	397,056 ± 0.00000
ViT	REFINED	0.99425 ± 0.00176	0.99425 ± 0.00176	138.15323 ± 3.43411	3,467,982 ± 0.00000	252,901,890 ± 0.00000
ViT+MLP	DistanceMatrix	0.99291 ± 0.00109	0.99291 ± 0.00110	552.69914 ± 30.75840	11,757,862 ± 0.00000	3,821,865,312 ± 755,362,456.01938
CNN	REFINED	0.99559 ± 0.00092	0.99559 ± 0.00092	106.43015 ± 0.47909	263,238 ± 0.00000	8,991,024 ± 0.00000

Architecture Results

Tree Baselines

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Train time (s)	#Params	FLOPs
CatBoost	1.00000 ± 0.00000	0.99703 ± 0.00021	0.99482 ± 0.00021	0.99480 ± 0.00026	0.99544 ± 0.00025	0.99419 ± 0.00027	36.80809 ± 0.25908	—	—
LightGBM	1.00000 ± 0.00000	0.99684 ± 0.00026	0.99339 ± 0.00079	0.99345 ± 0.00076	0.99405 ± 0.00082	0.99287 ± 0.00071	4.49590 ± 0.15569	—	—
XGBoost	1.00000 ± 0.00000	0.99664 ± 0.00000	0.99435 ± 0.00021	0.99444 ± 0.00022	0.99501 ± 0.00023	0.99388 ± 0.00025	5.76104 ± 0.04317	—	—

MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
MLP	0.99649 ± 0.00086	0.99492 ± 0.00064	0.99023 ± 0.00055	0.99023 ± 0.00055	0.99027 ± 0.00056	0.99023 ± 0.00055	0.99748 ± 0.00034	0.06793 ± 0.00848	0.98815 ± 0.00066	198,918 ± 0.00000	397,056 ± 0.00000	46.54599 ± 0.11946

ViT

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99741 ± 0.00118	0.99434 ± 0.00133	0.99061 ± 0.00199	0.99061 ± 0.00199	0.99065 ± 0.00200	0.99061 ± 0.00199	0.99956 ± 0.00014	0.04212 ± 0.00671	0.98861 ± 0.00242	5,737,950 ± 0.00000	54,820,530 ± 0.00000	182.82055 ± 41.48864
IGTD	0.99914 ± 0.00072	0.99616 ± 0.00034	0.65424 ± 0.01442	0.64211 ± 0.01677	0.68921 ± 0.02778	0.65424 ± 0.01442	0.91613 ± 0.01030	1.82867 ± 0.25459	0.59116 ± 0.02123	11,447,886 ± 0.00000	862,030,152 ± 0.00000	363.89538 ± 0.51325
REFINED	0.99756 ± 0.00119	0.99722 ± 0.00086	0.99425 ± 0.00176	0.99425 ± 0.00176	0.99426 ± 0.00176	0.99425 ± 0.00176	0.99983 ± 0.00010	0.02459 ± 0.00371	0.99303 ± 0.00214	3,467,982 ± 0.00000	252,901,890 ± 0.00000	138.15323 ± 3.43411
DistanceMatrix	0.99813 ± 0.00083	0.99722 ± 0.00092	0.99252 ± 0.00129	0.99252 ± 0.00130	0.99255 ± 0.00130	0.99252 ± 0.00129	0.99980 ± 0.00008	0.03687 ± 0.00308	0.99094 ± 0.00157	11,474,022 ± 0.00000	391,095,820 ± 0.00000	421.74331 ± 10.55267
BarGraph	0.99852 ± 0.00025	0.99616 ± 0.00034	0.99224 ± 0.00137	0.99224 ± 0.00137	0.99228 ± 0.00138	0.99224 ± 0.00137	0.99951 ± 0.00017	0.03886 ± 0.00613	0.99059 ± 0.00167	1,735,910 ± 0.00000	59,010,255 ± 0.00000	195.19845 ± 4.07837
Combination	0.99834 ± 0.00065	0.99626 ± 0.00040	0.99272 ± 0.00052	0.99272 ± 0.00052	0.99276 ± 0.00052	0.99272 ± 0.00052	0.99953 ± 0.00027	0.03742 ± 0.00465	0.99117 ± 0.00064	14,435,142 ± 0.00000	482,492,376 ± 0.00000	462.68637 ± 10.41090
SuperTML	0.99943 ± 0.00112	0.99214 ± 0.00110	0.98562 ± 0.00232	0.98564 ± 0.00232	0.98572 ± 0.00232	0.98562 ± 0.00232	0.99860 ± 0.00046	0.09438 ± 0.00864	0.98257 ± 0.00282	21,510,150 ± 0.00000	9,067,504,704 ± 0.00000	2,897.46299 ± 27.53749
FeatureWrap	0.81035 ± 0.07579	0.59712 ± 0.05523	0.59655 ± 0.06369	0.58378 ± 0.07979	0.63992 ± 0.09459	0.59655 ± 0.06369	0.87598 ± 0.03935	1.24476 ± 0.19879	0.52546 ± 0.08155	3,983,148 ± 0.00000	39,824,176 ± 0.00000	327.32415 ± 2.58176
BIE	0.99998 ± 0.00005	0.99453 ± 0.00055	0.98630 ± 0.00218	0.98629 ± 0.00219	0.98634 ± 0.00219	0.98630 ± 0.00218	0.99859 ± 0.00018	0.08844 ± 0.01330	0.98338 ± 0.00265	34,724,502 ± 0.00000	4,566,851,280 ± 0.00000	1,448.67827 ± 2.37018

ViT + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99714 ± 0.00136	0.99482 ± 0.00171	0.99090 ± 0.00096	0.99090 ± 0.00096	0.99093 ± 0.00096	0.99090 ± 0.00096	0.99929 ± 0.00023	0.04582 ± 0.00691	0.98896 ± 0.00116	5,870,302 ± 0.00000	779,343,120 ± 154,031,208.62387	180.41575 ± 7.10909
IGTD	0.99869 ± 0.00057	0.99597 ± 0.00087	0.70321 ± 0.00904	0.69452 ± 0.01129	0.71298 ± 0.01420	0.70321 ± 0.00904	0.93515 ± 0.00801	1.53880 ± 0.19012	0.64465 ± 0.00909	11,555,966 ± 0.00000	7,666,723,008 ± 1,515,269,180.93108	430.32818 ± 3.91669
REFINED	0.99708 ± 0.00118	0.99645 ± 0.00105	0.99281 ± 0.00131	0.99281 ± 0.00131	0.99284 ± 0.00131	0.99281 ± 0.00131	0.99982 ± 0.00010	0.03157 ± 0.00640	0.99128 ± 0.00159	3,822,910 ± 0.00000	5,059,519,524 ± 444,433,487.83141	192.69938 ± 12.95578
DistanceMatrix	0.99803 ± 0.00059	0.99741 ± 0.00055	0.99291 ± 0.00109	0.99291 ± 0.00110	0.99293 ± 0.00109	0.99291 ± 0.00109	0.99928 ± 0.00026	0.04195 ± 0.00326	0.99140 ± 0.00133	11,757,862 ± 0.00000	3,821,865,312 ± 755,362,456.01938	552.69914 ± 30.75840
BarGraph	0.99817 ± 0.00046	0.99626 ± 0.00098	0.99128 ± 0.00114	0.99128 ± 0.00114	0.99133 ± 0.00114	0.99128 ± 0.00114	0.99919 ± 0.00044	0.04655 ± 0.01206	0.98943 ± 0.00139	1,961,926 ± 0.00000	558,790,072 ± 110,440,585.08809	240.12472 ± 12.82240
Combination	0.99764 ± 0.00051	0.99664 ± 0.00034	0.99281 ± 0.00096	0.99281 ± 0.00096	0.99285 ± 0.00097	0.99281 ± 0.00096	0.99971 ± 0.00014	0.03550 ± 0.00279	0.99129 ± 0.00117	14,742,406 ± 0.00000	4,699,479,744 ± 928,816,238.05406	572.58901 ± 23.88611
SuperTML	0.99688 ± 0.00027	0.99482 ± 0.00063	0.99080 ± 0.00052	0.99080 ± 0.00053	0.99084 ± 0.00053	0.99080 ± 0.00052	0.99759 ± 0.00010	0.06655 ± 0.00271	0.98884 ± 0.00064	21,711,110 ± 0.00000	74,096,585,216 ± 14,644,623,507.70746	2,891.24381 ± 57.72719

CNN

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99873 ± 0.00104	0.99607 ± 0.00092	0.99214 ± 0.00115	0.99214 ± 0.00115	0.99216 ± 0.00114	0.99214 ± 0.00115	0.99992 ± 0.00003	0.02851 ± 0.00273	0.99047 ± 0.00140	1,619,470 ± 0.00000	171,610,944 ± 0.00000	136.59140 ± 2.88332
IGTD	0.99788 ± 0.00101	0.98888 ± 0.00380	0.17010 ± 0.04243	0.05139 ± 0.02348	0.06725 ± 0.07757	0.17010 ± 0.04243	0.57662 ± 0.04208	3.08575 ± 1.31520	0.00262 ± 0.00768	3,636,422 ± 0.00000	150,166,224 ± 0.00000	122.98220 ± 1.10191
REFINED	0.99940 ± 0.00034	0.99731 ± 0.00026	0.99559 ± 0.00092	0.99559 ± 0.00092	0.99560 ± 0.00092	0.99559 ± 0.00092	0.99999 ± 0.00001	0.01392 ± 0.00281	0.99465 ± 0.00112	263,238 ± 0.00000	8,991,024 ± 0.00000	106.43015 ± 0.47909
DistanceMatrix	0.99912 ± 0.00030	0.99712 ± 0.00059	0.99444 ± 0.00093	0.99444 ± 0.00093	0.99447 ± 0.00093	0.99444 ± 0.00093	0.99981 ± 0.00019	0.02764 ± 0.00898	0.99326 ± 0.00113	288,438 ± 0.00000	36,890,464 ± 0.00000	225.64282 ± 6.66273
BarGraph	0.99908 ± 0.00057	0.99626 ± 0.00141	0.99224 ± 0.00164	0.99224 ± 0.00164	0.99227 ± 0.00163	0.99224 ± 0.00164	0.99941 ± 0.00021	0.04300 ± 0.00658	0.99059 ± 0.00199	203,198 ± 0.00000	109,102,288 ± 0.00000	206.07859 ± 2.82049
Combination	0.99895 ± 0.00071	0.99703 ± 0.00052	0.99377 ± 0.00203	0.99377 ± 0.00203	0.99381 ± 0.00201	0.99377 ± 0.00203	0.99979 ± 0.00012	0.02849 ± 0.00648	0.99245 ± 0.00246	135,822 ± 0.00000	101,826,752 ± 0.00000	202.90694 ± 4.58884
SuperTML	0.99867 ± 0.00145	0.99444 ± 0.00110	0.98860 ± 0.00210	0.98860 ± 0.00210	0.98869 ± 0.00208	0.98860 ± 0.00210	0.99915 ± 0.00014	0.06455 ± 0.01081	0.98618 ± 0.00254	1,360,406 ± 0.00000	505,178,176 ± 0.00000	526.23909 ± 24.16208
FeatureWrap	0.86423 ± 0.06309	0.53960 ± 0.04474	0.62741 ± 0.03281	0.63051 ± 0.02636	0.71772 ± 0.03312	0.62741 ± 0.03281	0.89263 ± 0.01529	1.20806 ± 0.18070	0.57339 ± 0.03365	2,443,174 ± 0.00000	135,307,296 ± 0.00000	161.80681 ± 3.03091

Isolet

Source: OpenML · Original

Leaderboard

Family	Best variant	Test Accuracy (↑)	Test F1 (↑)	Train time (s)	#Params	FLOPs
Trees	XGBoost	0.96222 ± 0.00316	0.96221 ± 0.00314	168.39582 ± 4.28198	—	—
MLP	MLP	0.95829 ± 0.00412	0.95833 ± 0.00407	26.72631 ± 0.56690	454,426 ± 0.00000	908,032 ± 0.00000
ViT	REFINED	0.96376 ± 0.00268	0.96371 ± 0.00267	279.52203 ± 34.80752	10,017,680 ± 0.00000	529,073,367 ± 0.00000
ViT+MLP	REFINED	0.96051 ± 0.00706	0.96053 ± 0.00700	302.87938 ± 8.28478	10,532,176 ± 0.00000	4,902,711,480 ± 968,983,436.71594
CNN	IGTD	0.95897 ± 0.00423	0.95912 ± 0.00421	227.41112 ± 0.34707	11,012,890 ± 0.00000	824,791,872 ± 0.00000
CNN+MLP	IGTD	0.96188 ± 0.00536	0.96185 ± 0.00537	237.21687 ± 0.68981	11,507,930 ± 0.00000	825,781,120 ± 0.00000

Architecture Results

Tree Baselines

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Train time (s)	#Params	FLOPs
XGBoost	1.00000 ± 0.00000	0.96752 ± 0.00283	0.96222 ± 0.00316	0.96221 ± 0.00314	0.96323 ± 0.00310	0.96222 ± 0.00316	168.39582 ± 4.28198	—	—

MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
MLP	0.99399 ± 0.00389	0.95915 ± 0.00525	0.95829 ± 0.00412	0.95833 ± 0.00407	0.95976 ± 0.00380	0.95829 ± 0.00412	0.99924 ± 0.00012	0.15922 ± 0.00899	0.95668 ± 0.00427	454,426 ± 0.00000	908,032 ± 0.00000	26.72631 ± 0.56690

ViT

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.98142 ± 0.00843	0.92325 ± 0.00615	0.92684 ± 0.00732	0.92656 ± 0.00751	0.92888 ± 0.00680	0.92684 ± 0.00732	0.99872 ± 0.00009	0.23731 ± 0.01226	0.92402 ± 0.00757	1,802,354 ± 0.00000	18,282,472 ± 0.00000	216.46732 ± 29.10459
IGTD	0.99436 ± 0.01086	0.95983 ± 0.01320	0.94479 ± 0.01697	0.94449 ± 0.01731	0.94693 ± 0.01466	0.94479 ± 0.01697	0.99914 ± 0.00035	0.23000 ± 0.03803	0.94269 ± 0.01753	9,077,296 ± 0.00000	456,306,955 ± 0.00000	243.64260 ± 26.39967
REFINED	0.99993 ± 0.00016	0.96615 ± 0.00470	0.96376 ± 0.00268	0.96371 ± 0.00267	0.96497 ± 0.00247	0.96376 ± 0.00268	0.99923 ± 0.00023	0.15980 ± 0.01716	0.96236 ± 0.00278	10,017,680 ± 0.00000	529,073,367 ± 0.00000	279.52203 ± 34.80752
SuperTML	1.00000 ± 0.00000	0.85795 ± 0.01258	0.84427 ± 0.01152	0.84332 ± 0.01124	0.84561 ± 0.01052	0.84427 ± 0.01152	0.99221 ± 0.00109	0.50840 ± 0.03374	0.83818 ± 0.01196	10,036,794 ± 0.00000	83,538,992 ± 0.00000	262.63460 ± 2.58628
FeatureWrap	0.99780 ± 0.00206	0.91846 ± 0.00511	0.90821 ± 0.01090	0.90736 ± 0.01170	0.90953 ± 0.01088	0.90821 ± 0.01090	0.99672 ± 0.00049	0.31558 ± 0.01919	0.90465 ± 0.01128	14,200,242 ± 0.00000	142,015,880 ± 0.00000	351.11808 ± 15.76725

ViT + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.98889 ± 0.00369	0.94974 ± 0.00746	0.94479 ± 0.00609	0.94468 ± 0.00606	0.94574 ± 0.00563	0.94479 ± 0.00609	0.99864 ± 0.00042	0.21929 ± 0.03245	0.94263 ± 0.00632	2,303,090 ± 0.00000	272,973,120 ± 53,951,049.95015	278.14390 ± 7.19155
IGTD	0.99645 ± 0.00229	0.95607 ± 0.00215	0.94359 ± 0.00218	0.94356 ± 0.00198	0.94632 ± 0.00289	0.94359 ± 0.00218	0.99915 ± 0.00014	0.20866 ± 0.02095	0.94145 ± 0.00230	9,540,848 ± 0.00000	9,491,839,110 ± 833,773,076.98515	279.28474 ± 16.77670
REFINED	0.99802 ± 0.00226	0.96051 ± 0.00571	0.96051 ± 0.00706	0.96053 ± 0.00700	0.96178 ± 0.00674	0.96051 ± 0.00706	0.99942 ± 0.00002	0.16082 ± 0.01207	0.95898 ± 0.00733	10,532,176 ± 0.00000	4,902,711,480 ± 968,983,436.71594	302.87938 ± 8.28478
SuperTML	0.98384 ± 0.01251	0.94838 ± 0.01263	0.94376 ± 0.02125	0.94367 ± 0.02136	0.94594 ± 0.01920	0.94376 ± 0.02125	0.99904 ± 0.00038	0.19208 ± 0.04362	0.94161 ± 0.02200	10,609,402 ± 0.00000	1,743,066,720 ± 153,112,804.13442	289.08523 ± 3.50653
FeatureWrap	0.98252 ± 0.00567	0.94701 ± 0.00973	0.94701 ± 0.00880	0.94707 ± 0.00874	0.94868 ± 0.00794	0.94701 ± 0.00880	0.99886 ± 0.00033	0.19788 ± 0.02461	0.94495 ± 0.00912	14,660,242 ± 0.00000	2,048,710,208 ± 404,911,907.68233	372.53602 ± 17.60092

CNN

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99978 ± 0.00049	0.93231 ± 0.00140	0.93504 ± 0.00831	0.93506 ± 0.00835	0.93696 ± 0.00798	0.93504 ± 0.00831	0.99878 ± 0.00026	0.20936 ± 0.01879	0.93252 ± 0.00863	1,552,738 ± 0.00000	64,476,192 ± 0.00000	91.40022 ± 4.36537
IGTD	1.00000 ± 0.00000	0.94051 ± 0.00532	0.95897 ± 0.00423	0.95912 ± 0.00421	0.96171 ± 0.00367	0.95897 ± 0.00423	0.99937 ± 0.00016	0.14785 ± 0.00747	0.95744 ± 0.00437	11,012,890 ± 0.00000	824,791,872 ± 0.00000	227.41112 ± 0.34707
REFINED	0.99985 ± 0.00033	0.96000 ± 0.00429	0.95590 ± 0.00522	0.95576 ± 0.00525	0.95741 ± 0.00471	0.95590 ± 0.00522	0.99921 ± 0.00013	0.16478 ± 0.01589	0.95420 ± 0.00541	9,491,778 ± 0.00000	1,668,365,064 ± 0.00000	271.27100 ± 1.03411
FeatureWrap	1.00000 ± 0.00000	0.93966 ± 0.00644	0.94188 ± 0.00234	0.94182 ± 0.00234	0.94313 ± 0.00201	0.94188 ± 0.00234	0.99876 ± 0.00019	0.24086 ± 0.01758	0.93961 ± 0.00242	6,179,626 ± 0.00000	1,341,694,848 ± 0.00000	222.70396 ± 0.22781

CNN + MLP

Method	Train Accuracy	Val Accuracy	Test Accuracy (↑)	Test F1 (↑)	Test Precision	Test Recall	Test ROC AUC	Test LogLoss	Test MCC	#Params	FLOPs	Train time (s)
TINTO	0.99993 ± 0.00016	0.93368 ± 0.00455	0.94342 ± 0.00830	0.94358 ± 0.00827	0.94552 ± 0.00767	0.94342 ± 0.00830	0.99906 ± 0.00016	0.17505 ± 0.01858	0.94123 ± 0.00860	2,007,138 ± 0.00000	719,226,464 ± 0.00000	120.77254 ± 3.81571
IGTD	1.00000 ± 0.00000	0.94820 ± 0.00754	0.96188 ± 0.00536	0.96185 ± 0.00537	0.96361 ± 0.00474	0.96188 ± 0.00536	0.99952 ± 0.00012	0.13588 ± 0.01922	0.96043 ± 0.00555	11,507,930 ± 0.00000	825,781,120 ± 0.00000	237.21687 ± 0.68981
REFINED	0.99982 ± 0.00032	0.95470 ± 0.00347	0.94735 ± 0.01016	0.94738 ± 0.01003	0.94914 ± 0.00890	0.94735 ± 0.01016	0.99862 ± 0.00057	0.22555 ± 0.04630	0.94532 ± 0.01052	10,021,506 ± 0.00000	1,669,423,560 ± 0.00000	285.32846 ± 1.41554
FeatureWrap	0.99985 ± 0.00020	0.93402 ± 0.00711	0.93060 ± 0.01016	0.93051 ± 0.01012	0.93246 ± 0.00938	0.93060 ± 0.01016	0.99841 ± 0.00035	0.30922 ± 0.04104	0.92791 ± 0.01053	6,660,074 ± 0.00000	1,342,654,912 ± 0.00000	249.84972 ± 0.92565

Mfeat-fourier

Source: OpenML · UCI

Leaderboard

Family	Best variant	Test Accuracy (↑)	Train time (s)	#Params	FLOPs
Trees	CatBoost	0.858667 ± 0.009603	—	—	—
MLP	MLP	0.820000 ± 0.012247	—	—	—
ViT	TINTO_blur	0.870000 ± 0.016997	—	—	—
ViT+MLP	TINTO_blur	0.866667 ± 0.014720	—	—	—
CNN	TINTO_blur	0.864667 ± 0.007674	—	—	—
CNN+MLP	TINTO_blur	0.844667 ± 0.021422	—	—	—

Architecture Results

Tree Baselines

Method	Test Accuracy (↑)
XGBoost	0.842000 ± 0.009603
CatBoost	0.858667 ± 0.009603
LightGBM	0.857333 ± 0.010382

MLP

Test Accuracy (↑)
0.820000 ± 0.012247

ViT

Method	Test Accuracy (↑)
TINTO_blur	0.870000 ± 0.016997
IGTD	0.831333 ± 0.013038
REFINED	0.807333 ± 0.013622
DistanceMatrix	0.821333 ± 0.022186
BarGraph	0.825333 ± 0.018348
Combination	0.836000 ± 0.017857
SuperTML	0.742667 ± 0.018166
FeatureWrap	0.674667 ± 0.022925
BIE	0.678000 ± 0.021029

ViT + MLP

Method	Test Accuracy (↑)
TINTO_blur	0.866667 ± 0.014720
IGTD	0.820667 ± 0.016228
REFINED	0.808000 ± 0.024449
DistanceMatrix	0.837333 ± 0.011879
BarGraph	0.824000 ± 0.025755
Combination	0.820667 ± 0.015528
SuperTML	0.812000 ± 0.017733
FeatureWrap	0.816000 ± 0.006831
BIE	0.805333 ± 0.011205

CNN

Method	Test Accuracy (↑)
TINTO_blur	0.864667 ± 0.007674
IGTD	0.798667 ± 0.025122
REFINED	0.804667 ± 0.009309
DistanceMatrix	0.810000 ± 0.017951
BarGraph	0.806667 ± 0.026771
Combination	0.804000 ± 0.008300
SuperTML	0.661333 ± 0.047469
FeatureWrap	0.594000 ± 0.018469
BIE	0.481333 ± 0.117109

CNN + MLP

Method	Test Accuracy (↑)
TINTO_blur	0.844667 ± 0.021422
IGTD	0.798000 ± 0.011690
REFINED	0.797333 ± 0.004346
DistanceMatrix	0.801333 ± 0.008028
BarGraph	0.817333 ± 0.013208
Combination	0.812000 ± 0.009603
SuperTML	0.692000 ± 0.058052
FeatureWrap	0.801333 ± 0.010954
BIE	0.518667 ± 0.382235