how long before we acknowledge that releasing a model under a specific name, then replacing it with a worse one in a week without telling anyone is literally false advertising?
7
2
80
Replies
@alignment_lab
Perhaps leaderboards should report the model hash of the model at the time of testing so that we have a fingerprint to identify with.
1
0
4
@BramVanroy
any benchmark for an api model is inherently untrustable if the benchmarkers dont have access to the model
1
0
6
@alignment_lab
> get on top of leaderboard
> raise $2.75B
> swap out model and laugh all the way to the bank
2
0
19
