r*r
2 楼
https://www.theverge.com/platform/amp/circuitbreaker/2018/9/6/17828534/
huawei-benchmarks-rigged-p20-3dmark-ai
Huawei caught cheating benchmark test for P20
By Jacob Kastrenakes on September 6, 2018 5:14 pm
Photo by Vlad Savov / The Verge
Huawei has been caught optimizing some of its top smartphones to over-
perform on benchmark tests. On Tuesday, AnandTech discovered that Huawei’s
P20 had been programmed to maximize performance specifically when running
3DMark, a popular benchmarking app. Today, the company behind 3DMark
followed up with a statement saying that it had confirmed the findings and
would delist the P20, as well as three other Huawei phones with similar
behavior, from its benchmark leaderboards.
The delisted phones include the P20, P20 Pro, Nova 3, and Honor Play. Huawei
admitted to this behavior in a statement given to Android Authority, saying
that its phones are designed to adjust their performance based on the app
that’s running.
HUAWEI SAID IT WAS AI, BUT UL FOUND IT WASN’T
But the way that Huawei implemented that behavior isn’t allowed. While
phones can adjust their performance as part of their typical behavior under
high workloads, they can’t be hard coded to maximize their behavior just
because a specific benchmark app is running. That’s what Huawei seems to
have done, according to UL, which is behind the 3DMark software.
When UL ran an internal version of 3DMark, which Huawei’s phones couldn’t
recognize the name of, the phones performed worse in the test. That
indicated that the phones weren’t actually smart enough to identify high
performance demands on their own, which meant the benchmark score wasn’t an
accurate reflection of how the phone would handle a typical app without
special attention from Huawei.
As punishment, 3DMark has removed these phones’ rankings from its
leaderboard and adorned their listings on its website with a note that the
phone’s “manufacturer has not complied with UL benchmark rules.” Many of
their results have been removed as well.
Huawei is far from the first company to get caught toying with benchmark
results. Samsung got busted for the same behavior on its flagship phones in
2013, and just last year, OnePlus was found to have done the same. This
timing is particularly unfortunate for Huawei, since just weeks ago it got
caught trying to pass off a DSLR photo as a photo from one of its phones.
OTHERS HAVE BEEN CAUGHT DOING THIS IN THE PAST
What’s funny about all of this is that benchmarks don’t really matter that
much. Tweaking a phone to optimize benchmark apps might produce some
numbers that make a small subset of nerds drool, but those numbers don’t
correlate to the actual experience of using the phone. They might speak to
how well the phone performs under heavy stress, like while gaming, but a
better test is to just play a game with it and find out what happens.
Huawei even admits this. In its statement, the company said it “always
prioritizes the user experience rather than pursuing high benchmark scores
— especially since there isn’t a direct connection between smartphone
benchmarks and user experiences.” And yet, it still coded its phone to
deliver higher performance when running a benchmark test.
But Huawei also claims that its phones’ include AI that’s smart enough to
optimize performance based on whatever app is running, and clearly that’s
not the case. If it were, then boosting performance for benchmarks would be
fair game — but as it is, it’s not a true reflection of how the phone
performs in demanding situations.
huawei-benchmarks-rigged-p20-3dmark-ai
Huawei caught cheating benchmark test for P20
By Jacob Kastrenakes on September 6, 2018 5:14 pm
Photo by Vlad Savov / The Verge
Huawei has been caught optimizing some of its top smartphones to over-
perform on benchmark tests. On Tuesday, AnandTech discovered that Huawei’s
P20 had been programmed to maximize performance specifically when running
3DMark, a popular benchmarking app. Today, the company behind 3DMark
followed up with a statement saying that it had confirmed the findings and
would delist the P20, as well as three other Huawei phones with similar
behavior, from its benchmark leaderboards.
The delisted phones include the P20, P20 Pro, Nova 3, and Honor Play. Huawei
admitted to this behavior in a statement given to Android Authority, saying
that its phones are designed to adjust their performance based on the app
that’s running.
HUAWEI SAID IT WAS AI, BUT UL FOUND IT WASN’T
But the way that Huawei implemented that behavior isn’t allowed. While
phones can adjust their performance as part of their typical behavior under
high workloads, they can’t be hard coded to maximize their behavior just
because a specific benchmark app is running. That’s what Huawei seems to
have done, according to UL, which is behind the 3DMark software.
When UL ran an internal version of 3DMark, which Huawei’s phones couldn’t
recognize the name of, the phones performed worse in the test. That
indicated that the phones weren’t actually smart enough to identify high
performance demands on their own, which meant the benchmark score wasn’t an
accurate reflection of how the phone would handle a typical app without
special attention from Huawei.
As punishment, 3DMark has removed these phones’ rankings from its
leaderboard and adorned their listings on its website with a note that the
phone’s “manufacturer has not complied with UL benchmark rules.” Many of
their results have been removed as well.
Huawei is far from the first company to get caught toying with benchmark
results. Samsung got busted for the same behavior on its flagship phones in
2013, and just last year, OnePlus was found to have done the same. This
timing is particularly unfortunate for Huawei, since just weeks ago it got
caught trying to pass off a DSLR photo as a photo from one of its phones.
OTHERS HAVE BEEN CAUGHT DOING THIS IN THE PAST
What’s funny about all of this is that benchmarks don’t really matter that
much. Tweaking a phone to optimize benchmark apps might produce some
numbers that make a small subset of nerds drool, but those numbers don’t
correlate to the actual experience of using the phone. They might speak to
how well the phone performs under heavy stress, like while gaming, but a
better test is to just play a game with it and find out what happens.
Huawei even admits this. In its statement, the company said it “always
prioritizes the user experience rather than pursuing high benchmark scores
— especially since there isn’t a direct connection between smartphone
benchmarks and user experiences.” And yet, it still coded its phone to
deliver higher performance when running a benchmark test.
But Huawei also claims that its phones’ include AI that’s smart enough to
optimize performance based on whatever app is running, and clearly that’s
not the case. If it were, then boosting performance for benchmarks would be
fair game — but as it is, it’s not a true reflection of how the phone
performs in demanding situations.
d*d
3 楼
烧了赚钱了的，给我发包子．
M*t
4 楼
啥时候不cheat发个文吧
cheat的话习惯了
s
【在 r********r 的大作中提到】
: https://www.theverge.com/platform/amp/circuitbreaker/2018/9/6/17828534/
: huawei-benchmarks-rigged-p20-3dmark-ai
: Huawei caught cheating benchmark test for P20
: By Jacob Kastrenakes on September 6, 2018 5:14 pm
:
: Photo by Vlad Savov / The Verge
: Huawei has been caught optimizing some of its top smartphones to over-
: perform on benchmark tests. On Tuesday, AnandTech discovered that Huawei’s
: P20 had been programmed to maximize performance specifically when running
: 3DMark, a popular benchmarking app. Today, the company behind 3DMark
cheat的话习惯了
s
【在 r********r 的大作中提到】
: https://www.theverge.com/platform/amp/circuitbreaker/2018/9/6/17828534/
: huawei-benchmarks-rigged-p20-3dmark-ai
: Huawei caught cheating benchmark test for P20
: By Jacob Kastrenakes on September 6, 2018 5:14 pm
:
: Photo by Vlad Savov / The Verge
: Huawei has been caught optimizing some of its top smartphones to over-
: perform on benchmark tests. On Tuesday, AnandTech discovered that Huawei’s
: P20 had been programmed to maximize performance specifically when running
: 3DMark, a popular benchmarking app. Today, the company behind 3DMark
t*n
5 楼
wuwuw~ I was late. Can I short now?
【在 d*******d 的大作中提到】
: 烧了赚钱了的，给我发包子．
【在 d*******d 的大作中提到】
: 烧了赚钱了的，给我发包子．
相关阅读
有什么可以看体育直播的app么27号preorder的iphone X最近s8或者pixel 2会有好的deal吗ZTE的可折叠电话来了！Honor 7x LTEGoogle要是搜索，Gmail出问题，还值得嚷嚷有人出不用的ddr3 desktop内存吗？跟surface pro3差不多的tablet 电脑Mix2和Mate9买哪个？做笔记工具 Nimbus Note 怎么样？pixel 2 xl burn in........Mate 9 pro完全没有升Mate 10的动力现在有啥好的轻薄本本终于要把用了四年的HTC换了。生么叫做续航尿崩我软接连搞了好几件大事Google Pixel (2) 真的很丑Samsung的S7 Edge设置了non disturb还是会震动16-Piece Precision Screwdriver $3三星哪几款手机能支持中国电信的卡啊