CustomGains

ランキング学習に着順を予測させるのにその評価としてCustomGainsを使ってるんだけど、単純に例えば18頭立ての1着～18着に値を設定して学習させるのに1着が当然最大で18着が最低になる必要がある。最初は1着=18点～18着=1点的な単純なところから、これだと同じ1着でも、何頭立てかで差が出るので工夫始めて1着=100点～18着=0点的なに変え、現状はその10倍で、1着=1,000点にしたり、ボーダーで例えば、500にすると半分で、10着～18着は0点、1着～9着に1～1,000点を割り振る。更にこれを乗数で、1.0ならリニア、それ以上なら弧を描く感じにするって具合です。

昨夜から本日でイテレーション数3,000、葉271、率0.01、分割数47、葉の最小50、乗数1.0という条件で、ボーダーのみ0か500で40回学習させてみました。500では、NDCG@1=0.7831 NDCG@3=0.7859が

3,455R	1点	芝(1,689R)	ダート(1,646R)	障害(120R)	8頭以下(219R)	9～12頭(809R)	13頭以上(2,427R)	多点
単勝	27.41% (78.69%)	27.18% (79.88%)	27.28% (76.25%)	32.50% (95.50%)	38.36% (90.87%)	30.04% (83.46%)	25.55% (76.00%)	57.66% (76.25%)
複勝	59.13% (84.28%)	58.97% (84.03%)	58.87% (84.39%)	65.00% (86.25%)	72.60% (90.91%)	65.27% (86.70%)	55.87% (82.87%)	89.49% (81.82%)
枠連	16.19% (84.84%)	16.61% (94.90%)	15.25% (73.89%)	25.25% (115.86%)	-- (--)	18.91% (86.75%)	14.26% (78.82%)	30.99% (77.16%)
馬連	12.71% (78.54%)	13.44% (85.81%)	11.48% (70.30%)	19.17% (89.17%)	25.11% (66.35%)	15.57% (83.82%)	10.63% (77.87%)	25.30% (70.47%)
ワイド	27.70% (89.74%)	28.95% (92.34%)	26.12% (87.78%)	31.67% (80.17%)	51.60% (84.66%)	33.25% (87.97%)	23.69% (90.79%)	49.03% (77.13%)
馬単	6.92% (70.47%)	6.87% (70.24%)	6.68% (68.07%)	10.83% (106.67%)	13.24% (60.32%)	8.53% (84.30%)	5.81% (66.78%)	25.30% (68.05%)
三連複	6.08% (58.46%)	7.16% (57.94%)	4.86% (57.79%)	7.50% (74.83%)	18.72% (67.53%)	8.03% (57.81%)	4.29% (57.85%)	16.67% (72.16%)
三連単	1.45% (55.38%)	1.89% (76.19%)	1.03% (23.44%)	0.83% (200.75%)	3.20% (34.25%)	2.47% (97.11%)	0.95% (43.38%)	16.67% (68.22%)
総合	60.35% (74.92%)	60.33% (79.85%)	59.96% (67.70%)	65.83% (105.93%)	76.26% (70.70%)	66.13% (83.49%)	56.98% (71.80%)	89.90% (71.01%)

ボーダー0だとNDCG@1=0.7840 NDCG@3=0.7880で

3,455R	1点	芝(1,689R)	ダート(1,646R)	障害(120R)	8頭以下(219R)	9～12頭(809R)	13頭以上(2,427R)	多点
単勝	26.89% (78.58%)	26.70% (79.14%)	26.91% (76.94%)	29.17% (93.00%)	37.44% (88.45%)	28.92% (83.97%)	25.26% (75.89%)	57.05% (76.95%)
複勝	58.58% (84.61%)	58.91% (84.64%)	57.96% (84.65%)	62.50% (83.67%)	70.32% (86.35%)	63.41% (84.99%)	55.91% (84.33%)	89.61% (82.32%)
枠連	15.87% (87.71%)	16.04% (95.05%)	14.99% (79.26%)	27.27% (117.47%)	-- (--)	17.06% (76.00%)	14.46% (86.04%)	30.50% (76.45%)
馬連	12.33% (81.60%)	12.91% (90.22%)	11.12% (71.56%)	20.83% (97.92%)	23.29% (68.77%)	14.34% (74.15%)	10.67% (85.24%)	24.98% (69.09%)
ワイド	26.98% (86.42%)	28.18% (85.32%)	25.15% (87.46%)	35.00% (87.67%)	51.14% (84.93%)	32.26% (82.50%)	23.03% (87.86%)	48.25% (76.69%)
馬単	6.48% (70.34%)	6.69% (80.09%)	5.89% (57.50%)	11.67% (109.25%)	12.79% (78.72%)	7.42% (69.37%)	5.60% (69.91%)	24.98% (66.19%)
三連複	6.05% (62.19%)	6.99% (55.35%)	4.98% (68.75%)	7.50% (68.50%)	19.63% (81.05%)	8.28% (69.68%)	4.08% (57.99%)	15.89% (73.60%)
三連単	1.45% (61.80%)	1.66% (75.58%)	1.22% (46.43%)	1.67% (78.75%)	4.57% (145.34%)	1.85% (48.22%)	1.03% (58.79%)	15.89% (67.54%)
総合	59.88% (76.50%)	60.27% (80.37%)	59.23% (71.53%)	63.33% (91.46%)	74.43% (90.52%)	63.78% (73.61%)	57.27% (75.76%)	89.99% (70.49%)

って感じで、指数的には上でも的中率的には下がってるけど、回収率は...微妙に良いのか悪いのか。ただ、このNDCG@1=0.7840って今回の40回で3つ出てます。NDCG@1=0.7840 NDCG@3=0.7877って事で@3が違ってますが、

3,455R	1点	芝(1,689R)	ダート(1,646R)	障害(120R)	8頭以下(219R)	9～12頭(809R)	13頭以上(2,427R)	多点
単勝	27.47% (81.98%)	26.94% (80.30%)	27.70% (82.35%)	31.67% (100.75%)	40.18% (95.84%)	28.31% (80.61%)	26.04% (81.19%)	56.82% (78.11%)
複勝	59.39% (86.25%)	59.44% (85.28%)	58.87% (87.02%)	65.83% (89.33%)	73.97% (92.51%)	64.03% (85.23%)	56.53% (86.02%)	89.84% (83.29%)
枠連	15.77% (84.33%)	15.61% (90.16%)	15.12% (76.91%)	28.28% (119.19%)	-- (--)	17.18% (76.33%)	14.30% (81.64%)	30.73% (79.09%)
馬連	12.13% (78.07%)	12.20% (79.19%)	11.36% (74.56%)	21.67% (110.42%)	22.83% (66.35%)	14.22% (73.63%)	10.47% (80.61%)	25.04% (73.50%)
ワイド	27.38% (87.37%)	28.48% (84.84%)	25.82% (90.16%)	33.33% (84.67%)	52.05% (87.12%)	32.39% (82.22%)	23.49% (89.11%)	48.63% (78.94%)
馬単	6.60% (69.54%)	6.57% (74.89%)	6.14% (59.91%)	13.33% (126.17%)	13.24% (75.75%)	7.17% (59.86%)	5.81% (72.20%)	25.04% (70.88%)
三連複	6.08% (64.78%)	6.81% (59.95%)	5.22% (68.79%)	7.50% (77.75%)	17.81% (72.05%)	8.41% (74.97%)	4.24% (60.73%)	15.77% (75.10%)
三連単	1.42% (68.11%)	1.72% (84.50%)	1.03% (49.72%)	2.50% (89.75%)	4.11% (132.42%)	1.85% (46.49%)	1.03% (69.52%)	15.77% (69.96%)
総合	60.67% (77.46%)	60.81% (79.67%)	60.09% (73.66%)	66.67% (99.32%)	78.08% (88.86%)	64.65% (72.42%)	57.77% (77.63%)	90.10% (73.07%)

更にNDCG@1=0.7840 NDCG@3=0.7879だと

3,455R	1点	芝(1,689R)	ダート(1,646R)	障害(120R)	8頭以下(219R)	9～12頭(809R)	13頭以上(2,427R)	多点
単勝	26.80% (78.25%)	26.58% (78.88%)	26.79% (76.42%)	30.00% (94.33%)	37.90% (87.76%)	28.06% (79.85%)	25.38% (76.85%)	56.73% (76.95%)
複勝	58.81% (84.77%)	58.85% (84.22%)	58.26% (85.01%)	65.83% (89.33%)	72.15% (90.41%)	63.04% (83.63%)	56.20% (84.64%)	89.41% (82.84%)
枠連	15.96% (86.59%)	16.18% (95.84%)	15.06% (76.45%)	27.27% (115.96%)	-- (--)	16.44% (70.09%)	14.79% (86.59%)	30.79% (77.77%)
馬連	12.71% (81.55%)	12.91% (85.16%)	11.85% (76.18%)	21.67% (104.50%)	24.20% (73.33%)	13.84% (65.97%)	11.29% (87.49%)	25.21% (71.42%)
ワイド	27.35% (89.19%)	28.54% (92.45%)	25.58% (86.01%)	35.00% (86.92%)	51.60% (85.25%)	31.89% (81.24%)	23.65% (92.20%)	48.71% (78.06%)
馬単	6.71% (72.77%)	6.93% (81.94%)	6.20% (62.90%)	10.83% (79.08%)	11.42% (46.71%)	6.80% (52.40%)	6.26% (81.92%)	25.21% (68.64%)
三連複	6.08% (66.08%)	7.16% (67.89%)	4.98% (64.82%)	5.83% (57.92%)	16.89% (68.95%)	8.53% (71.35%)	4.29% (64.07%)	15.95% (70.47%)
三連単	1.36% (61.66%)	1.66% (83.01%)	1.15% (44.23%)	0.00% (0.00%)	2.28% (19.63%)	1.98% (50.40%)	1.07% (69.20%)	15.95% (65.00%)
総合	60.00% (77.48%)	60.09% (83.42%)	59.42% (71.48%)	66.67% (77.67%)	76.71% (67.44%)	63.54% (69.36%)	57.31% (80.37%)	89.67% (69.61%)

同じ@1でも、@3で左右されますね。