Subnautica Performance and Framerates

Posted by acedude 9 years ago

Moviemakers go on and on about how hard their jobs are, don’t they? Call me insane, but I figure that if your entertainment media can be re-shot over and over again, it’s got to be easy-pesy. Compare a movie, which has linear progression, to a game, which has practically infinite experience permutations. Logically, that makes games infinitely harder to make, and Unknown Worlds could probably create a movie like Interstellar in about three days.

Obviously I’m talking garbage, but reading garbage is the risk you run when you click on a blog post with ‘hugh’ at the top. Stick with me though, there’s a point coming. Games are rendered in real time. A player’s experience is substantially influenced by the technical effectiveness with which a game presents its endless permutations. This is a way of saying ‘make your game perform better, idiot.’ This blog post is a run down on our current Subnautica performance thinking.

Performance, or the measure of how fast a game runs, has been top of the Subnautica agenda since the game’s inception. Originally, Charlie envisioned Subnautica as a low visual fidelity, aesthetically simple game that would run on everything from the Large Pixel Collider (LPC), to an iPad. At the time, we figured that blazing frame-rates would come naturally from lower visual quality. As the project grew and matured, it evolved into a high visual fidelity game.

It’s not just the detail of the individual screen: We’re dealing with the large groups of fish, dynamic lighting, multiple large creatures, and deformable, voxel based terrain. Adding all this stuff means we needed more computing power than an iPad could provide. At first, we asked PC Gamer if they would be willing to provide an LPC to every Subnautica customer. Like complete cheapskates, and despite the awesome cross-promotion potential, they refused. So instead, we started to think about what a realistic level of required computing power would be.


Breakdown of August Valve Hardware Survey with draft SN specs

Being a game developer means selling games. You can’t sell games to a target market with no-one in it. That’s why the above graph is so important: Creating it told us that if we set our minimum GPU specification as an Intel HD 4600, 39% of Steam users could play it. For now (these specs are draft!) this is the line we’ve drawn under the bottom of our target market.

Making this calculation scientifically is very important. Anyone can walk onto a forum and say ‘I think that a GTX760 is a fine recommended spec.’ Or, they might say ‘Subnautica runs like crap.’ Or, ‘Subnautica runs smooth as butter.’ These statements (and game forums are full of them!) are anecdotal, qualitative, and all but useless for making effective decisions about performance. How many people can actually play if our minimum spec is a GTX 660? As it happens, a whole lot less than if it is a GTX 550 ti!


One of the earlier Subnautica dev performance tests, from June

Setting scientific targets is the first part of the recipe. To complete the cake, we need to hit them. That means lots and lots of testing. The graph above is one of the earlier performance tests we ran at our draft minimum specification: An Intel HD 4600 GPU, two Haswell CPU cores @ 2.5Ghz with four total logical processors, 8Gb RAM. Our minimum graphics level is 1280 x 720 resolution with all eye candy off.

To read this graph, you need to stop thinking in terms of ‘Frames Per Second’ (FPS). FPS is nice to talk about but useless for analytical purposes. I won’t go into it here, but suffice to say if you measure a game’s performance with FPS, you could be missing huge issues, and analysing data is made needlessly harder. Instead, think about ‘frame time’ – The time it takes to render a complete frame, in milliseconds (ms).

Each blue dot in the graph is a single rendered frame – The y-axis showing the length of time it took to render and the x-axis the time at which it was rendered in the benchmark. In frame times, 35FPS is equivalent to 28.6ms, and 60fps is equivalent to 16.7ms. Those two thresholds are the orange and green lines you see overlayed on the graph.


A Subanutica min-spec benchmark from early October

Each week, we run comprehensive tests on both recommended and minimum specification machines that sit in the Unknown Worlds office. By keeping Steve locked in a cage for over ten months, we have significantly improved performance on our minimum and recommended spec machines. Not just in terms of mean frame rates, but also in variance. Variance is often perceived as ‘jitter’ or a general feeling of ‘inconsistency’ while playing.


An early October histogram of recommended-spec frames

We visualise performance in a few ways. The above histogram shows the proportion of frames being rendered in three critical buckets: Those less than our minimum playable threshold (28.6ms), those less than target (16.7ms) but greater than minimum playable thresholds, and those less than target. In an ideal world, all frames would be in the green bucket. But in that ideal world, there would probably be nothing on the screen.


Cumulative distribution showing varied performance over many builds

Hitting performance targets is a tricky business. On the one hand, many people on the team want to ‘add stuff’ to Subnautica. With few exceptions, ‘adding stuff’ results in the game running more slowly. Further, devoting engineering resources to keeping performance within target ranges means taking resources away from creating gameplay. And at the end of that dark corridor stands TotalBiscuit, ready to eviscerate us for not achieving good frame rates. (<3 you TB!)

Rightfully so, too. There’s no point in making games that run like PowerPoint presentations. The point of games is to interact with the game space, not watch it pass by. The slower a game runs, the greater the barrier between engagement and the player. While not everyone on the Subnautica team is obsessed with performance as I am, I don’t think any of us are in the ‘lower frame rates are more cinematic’ camp. I hear there’s dengue in that camp.


November 12th recommended spec test, with 16.7ms target removed

Where to make the tradeoff between ‘stuff’ and performance? One of the great things about PC games is they aren’t bound to the lowest-common-denominator hardware, like consoles are. The vastly greater computing power available to PC developers means we can do unique, wonderful things. For example, Subnautica’s enormous deformable high-fidelity voxel seafloor requires lots of computing power, and would be very difficult to do on very low-end hardware.

Cutting all our unique features to ensure we run blazing fast on low end hardware doesn’t make sense. The inclusion of cool stuff like our voxel sea floor means we will never hit consistent 16.7ms frame times on minimum specification while watching the fish swim by. We’ve come to that realisation recently, and stopped including 16.7ms targets in our performance evaluations. Now, we are simply targeting 28.6ms on both recommended and minimum specifications.

Fortunately, these targets compare very favorably with other indie games pushing higher visual fidelity, experimental technology, and computationally demanding gameplay elements (like a massive open world). You can watch us run our weekly performance tests by checking in our Trello board. If you find things in game that cause performance problems, you can report them to the feedback system by pressing F8 and selecting the ‘framerate’ option.

This post is very much a window onto a work-in-progress. Our performance targets and minimum/recommended specification are still drafts. However, I thought you all might like to see how seriously we are taking performance, and the steps we are taking to ensure Subnautica runs really well.

Comments are closed.