Back in October several phone makers were called out by Anandtech for cheating in benchmarks. At the most basic level, the site found that some Samsung, HTC, LG and Asus smartphones were optimizing their CPUs in order to run better on benchmark tests. That ended up suggesting that some phones performed better than others, but only because they were rigged to do well on the tests. Now, at least one benchmark maker is fighting back.
HotHardware recently noticed that several HTC and Samsung devices have been “delisted” from Futuremark’s 3DMark benchmark charts. Devices including the Galaxy Note III, Galaxy Note 10.1, HTC one Mini and HTC One have been delisted because Futuremark suspects that the phones (and tablet) aren’t playing fairly.
“In simple terms, a device must run our benchmarks without modification as if they were any other application,” the company said in a statement provided to HotHardware. “People rely on Futuremark benchmarks to produce accurate and unbiased results. That’s why we have clear rules for hardware manufacturers and software developers that specify how a platform can interact with our benchmark software.”
Samsung has denied cheating in benchmarks, but clearly the professionals that develop the software disagree, and want to make sure the playing field is level.