GPU benchmarks: how they can mislead a GPU upgrade

GPU Pricing are finally back to normal, and you may have found yourself scrolling through graphics card reviews for the past few weeks to see which ones are at the top of the charts. After all, the best graphics cards live and die based on their performance in gaming benchmarks, right?

But those benchmarks are far from a definitive answer, and in most cases they divert the conversation away from the games you actually play and the experiences they provide.

I’m not saying we should throw the baby out with the bathwater. GPU benchmarks provide a lot of value and I don’t think anything needs to change in how we (or others) conduct GPU reviews. But now that it’s actually possible to upgrade your graphics card, it’s important to put all the performance figures in context.

Games, not benchmarks

A character brandishing a sword in Lost Ark.
The most popular 2022 Steam game so far? lost arkwhich only asks for a GTX 1050.

DT’s computing evergreen coordinator Jon Martindale recently joked about GPU pricing: “I need a new GPU so I can get 9,000 frames in” Vampire Survivors.Stupid, but there is a salient point. When looking at performance, it’s important to realize that there are about four times as many people playing Terrariums or Stardew Valley as it is played Forza Horizon 5 or Cyberpunk 2077 at any time.

The best games to benchmark your pc are not the most popular games people play. In the top 25 most popular steam gamesonly two of them are regularly used in benchmarks: Grand Theft Auto V and Rainbow Six Siege. Virtually no “live” games are included in benchmark suites due to network variation, despite these games being largely at the top of the charts in player count, and recent, GPU-limited games tend to be over-represented.

The games we and others have chosen as benchmarks aren’t the problem – they provide a way to push a GPU to its limits in order to compare it to the competition and previous generations. The problem is that benchmarks hit performance around the clearest margins. And those margins can imply performance that won’t hold up outside of a graphics card rating.

Benchmarks are often misleading

A hand gripping the RTX 3090 Ti graphics card.
Jacob Roach / Digital Trends

Especially when it comes to the most recent graphics cards, benchmarks can be downright misleading. Every benchmark needs at least an average frame rate, which in itself is a problematic number. Short peaks in frame rate are overrepresented in an average of 1% lows and 0.1% lows – which average the lowest 1% and 0.1% of frames, respectively. But those numbers still don’t say much about how often those framerate dips occur — just how severe they are.

A frametime chart can show how often framerate dips occur, but even that only represents the portion of the game the benchmark focused on. I hope you see the trend here: the buck has to stop somewhere, even as more data points try to paint a picture of real-world performance. Show benchmarks family member performance, but they don’t say much about the experience of playing a game.

The RTX 3090 Ti is 8.5% faster than the RTX 3090 in Red Dead Redemption 2, for example. That’s true, and it’s important to keep in mind. But the difference between the cards while playing is all seven frames. I’d be hard-pressed to see a difference in gameplay between 77fps and 84fps without a framerate counter, so while the RTX 3090 Ti is technically faster, it doesn’t affect the gaming experience Red Dead Redemption 2 in a meaningful way.

Performance benchmarks for the RTX 3090 and RTX 3090 Ti in Red Dead Redemption 2.

the recent F1 2022 is another example. The game shows huge performance differences between resolutions with all higher level settings (as you would usually find them in a GPU review). But ditch a few GPU-intensive graphics options, and the game is so CPU-constrained that it offers nearly identical performance between 1080p and 4K. No need for a GPU upgrade there.

No one intentionally lies or misleads with benchmarks, but the strict GPU hierarchy they establish is an abstraction of how your graphics card will be used for what you bought it for in the first place. Benchmarks are important to show differences, but they don’t tell you if those differences are actually matter.

Perform an informed GPU upgrade

Installing a graphics card in a motherboard.

You should definitely look at benchmarks before upgrading your GPU as much as you can. But don’t put your money down before answering these questions:

  • Which games do I want to play?
  • What resolution do I want to play at?
  • Are there other components I need to upgrade?
  • What is my budget?

Relative performance is extremely important to understand what you’re getting for your money, but better isn’t strictly better in the PC components world. Depending on the games you play, the resolution you play at, and possible bottlenecks in your systemyou could buy a more expensive GPU and the exactly the same performance as a cheaper one.

That doesn’t mean you shouldn’t spend. There’s a lot to be said about buying something nice just because it’s fun, even if it doesn’t offer a huge benefit. If you have the means, it’s new to own something super powerful like an RTX 3090 – even if you only use it for play Vampire Survivors. Just don’t expect a difference when you’re actually playing.

This article is part of ReSpec – an ongoing biweekly column featuring discussions, advice and in-depth reporting on the technology behind PC gaming.

Editor’s Recommendations






Leave a Comment

Your email address will not be published.