Cheating in the benchmark application is not new and we have seen a fair share of such fiasco way before the MediaTek’s shenanigans that was exposed by the much-respected team over at Anandtech.
This time, instead of phone manufacturers, MediaTek themselves have implemented the cheat in the SoC’s BSP (Board Support Package). You can either check out Anandtech’s article or our findings if you’re interested in knowing how they have done this but in this article, we’re just going to have a quick break down on the reasons why companies are still taking shortcuts to gain customers’ attention.
Higher Score = Better
That goes without saying, 100 is always better than 99, even if you’re just 1 off from the mark. That’s the harsh reality of benchmarks and it was intended to be a base comparison for different products that perform the same tasks, given if all the other variables are constant. Because of that extra points, companies would go the extra mile to make their product look good when it comes to review and benchmark scores.
When we read through the response MediaTek has given to AnandTech, it was a huge disappointment.
MediaTek Statement for AnandTech
MediaTek follows accepted industry standards and is confident that benchmarking tests accurately represent the capabilities of our chipsets. We work closely with global device makers when it comes to testing and benchmarking devices powered by our chipsets, but ultimately brands have the flexibility to configure their own devices as they see fit. Many companies design devices to run on the highest possible performance levels when benchmarking tests are running in order to show the full capabilities of the chipset. This reveals what the upper end of performance capabilities are on any given chipset.
Of course, in real-world scenarios there are a multitude of factors that will determine how chipsets perform. MediaTek’s chipsets are designed to optimize power and performance to provide the best user experience possible while maximizing battery life. If someone is running a compute-intensive program like a demanding game, the chipset will intelligently adapt to computing patterns to deliver sustained performance. This means that a user will see different levels of performance from different apps as the chipset dynamically manages the CPU, GPU and memory resources according to the power and performance that is required for a great user experience. Additionally, some brands have different types of modes turned on in different regions so device performance can vary based on regional market requirements.
We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance.
Cue the last paragraph –
We believe that showcasing the full capabilities of a chipset in benchmarking tests is in line with the practices of other companies and gives consumers an accurate picture of device performance.
They miss the entire point of why we’re doing benchmark and reviews. We are supposed to review and test the product according to how the device was designed to run. If your device somehow can identify benchmark apps and forcefully push the chip to its limits, does that means that your phone is that powerful in everyday usage?
That’s the one main reason why Huawei got so much flak when they were caught cheating in benchmarks. Users are unable to replicate those performance numbers without the built-in limit unlocker.
It is the same situation if an athlete uses performance-enhancing drugs and then tells everyone that, “Oh, I’m just showcasing my full capabilities in the Olympics by taking the drugs. It just merely giving everyone an accurate picture of my performance.”
If your SoC is so powerful, why don’t you just allow the users to run your chipset at its highest-possible performance at the user’s will – just like how Huawei did with their Performance Mode? Why deliberately hardcode the function ONLY to detect a select number of apps? It is just like how you’re behaving like a good kid when the teacher is around but immediately turn into a delinquent the moment the teacher took her eyes off from you.
MediaTek just slapped on their face with the quoted statement – “an accurate picture of device performance”.
Other companies are doing it too
The statement also indirectly points out that other manufacturers are doing the same thing as well. Sure, this is not the first time we’re seeing this and it won’t be the last. Would you be proud of saying “I’m cheating in the exams because Daniel also cheated in his papers”???
I know good benchmark scores make your device stand out when reviewers compare them side by side with other brand’s devices. What that means, higher conversion especially for some of those consumers who make their purchase decision based on benchmarks. (these people exists)
Now, since the cat is out of the box, what would they think of your brand and business ethics? I’m sure there are plenty of businesses out there won’t even flinch. Well, why worry if you’re already swimming in a pool of cash? Like, why would you be honest if you have the option to become rich?
We should blame Ourselves – Reviewers and Customers
At the end of the day, it all boils down to the very core reason – there’s a demand for good benchmark scores. Reviewers like us often compare devices performance by benchmark scores, trusting the software is the constant variable that would give us a number to represent the device. Then comes another group of people – the readers or consumers who are looking for online information regarding the devices they’re interested to purchase. And then, capitalising on this whole cycle is the companies that try to on the best show, either respectively or unethically to win customers over.
It is not a healthy situation as of now and since the benchmark score is an objective comparison, it will still be part of the device testing process. Just that we reviewers will have to take extra precaution when it comes to the test process and data presentation. Most importantly, consumers or readers should be aware of such cases and always remember that numbers don’t represent actual user experience.
What will happen next?
We have no idea. Looking at the whole situation, MediaTek may or may not have informed smartphone brands that such a thing is built into their chipset’s BSP, which would be written in their contract.
Maybe multiple brands will file a lawsuit against MediaTek for doing this and tarnishing the brands’ image. Remember – MediaTek is a huge chip supplier and a few of the biggest smartphone brands in the world are still using MediaTek chipsets, even on their flagships.
Grab some popcorn, because this is a drama that we can all spectate from the comfort of our home this RMO.
Like our Facebook Page here at NasiLemakTech.com for more news and in-depth reviews! Also, join our Facebook Group for insightful information and memes!
Subscribe to our YouTube channel too!
