Following the Terrain

Some aircraft spend lots of time flying at low altitude and great speed. They often employ ‘terrain following’ radar: Systems that bounce electromagnetic radiation off the surrounding environment. The return signals are processed to compute a safe and optimal flight path.

Humans can do a decent job at looking out for the treetops. Crop dusters get around just fine without fancy radar systems. Yet when the sun goes down, or the aircraft gets faster, our eyes start to fail and our brains can’t keep up.

NS2 development is descending. We’re getting closer to the terrain, hiring a new development team and devoting resources to new objectives.

We’re also speeding up. There won’t be any more grand plans. No months of feature development. An idea doesn’t get resources until it shows evidence of value. Changes will happen in minutes, not months, so that ideas are forced to show that evidence.

At such great speed, and with our wings so close to the hills, there is great risk of collision. We need way to follow the terrain closely, through day and night, and ensure that we are not about to fly NS2 into a mountain.

For many years NS2 development was driven by intuition. Where scientific decision making was used, supporting evidence was often heavily time delayed and difficult to access. Unknown Worlds would make decisions around the lunch table, ideas and argument flowing back and forth.

A decision might be to make big, themed update. That update might include new abilities, maps, and graphical options. We might all nod and voice our consensus that such an update will increase sales.

The update is released. We congratulate each other. There’s relief that the product is out the door, and a sense of satisfaction and fulfilment. Did we make any extra sales? And what if the new features were released piecemeal instead of monolithically: Which ones correlated with higher sales?

That has been a very difficult question to answer. Steam does not provide detailed sales reporting tools. By business convention, not everyone in the company has access to sales data. Those that do have not always been timely or effective in organising and presenting those data to the rest of the team. As a small independent studio, we’ve never developed or purchased a comprehensive sales reporting system.

Without being able to see how our decisions affect sales, we are effectively eyeballing a moonlit mountain range at 200 metres per second. That’s not good enough. It’s already changed.

The entire NS2 development team now has access to near-real time sales data, and tools to organise it and present it as they see fit. At the same time, the system providing those data, ‘Aesa,’ is being development to present other useful information.

How many people are visiting our store page? Which changes correlated with higher player numbers? Which communication methods produce the greatest player influx? What is the proportion of active players sitting in the lobby? How often is that fancy new alien ability used?

These are all questions we should be asking constantly. We should be testing our assumptions and hypotheses against the evidence provided by the answers to these questions. Until now, getting those answers has meant jumping between a smorgasbord of different tools. Google Analytics, NS2 Sponitor2, Steamworks… And in most cases, the answers were not available at all.

Word about NS2’s development restart got out on Monday. I saw lots of comments asking ‘Ok, but what exactly are is going to be done?’ The answer is still ‘I’ve got no idea.’ Instead of thinking about the what, I’m thinking about the how. The activation of systems like Aesa, which allow us to follow the terrain of customer satisfaction, is a step towards developing the how.

And yes, I did write this blog post while travelling on a plane.  Thankfully not too close to the ground.

– Hugh

Comments are closed.