Friday Facts #401 - New terrain, new planet

Posted by Earendel on 2024-03-08

Surprise! There are 5 new planets in the expansion.

The first planet you arrive on is a strange new land, rich with iron, copper, coal, stone, oil, and uranium. Everything a starting factory needs and more. There's also water, fish, grass and trees. Yes, this is a fertile land, and we will thrive. We will rule over all this land, and we will call it... "This Land" "Nauvis".

Nauvis planet icon

So yes, the new planet Nauvis is similar to the 1.1 version of the map, but there are some substantial updates. Hopefully this is a surprise for you, because updates to the base map weren't planned at all. It just got harder and harder to not make changes after getting more experience with the other planets and seeing things that were missing or broken.

For clarity, these changes affect the 2.0 version in general and are not exclusive to the expansion.

This has been a solo side project for me for a while and it's only just come to a conclusion. Anyway, here's the story:

Stealth dunes

After working on Fulgora, the Nauvis deserts weren't looking so good. In fact, they barely seemed like a proper desert at all. I know that the game 'had' sand dunes, but they never seemed to spawn in the areas near me. It's only flatish sand-dirt with a lot of brown grass even in the driest areas. I decided to investigate.

Earlier in 1.0's development there were sand dune decoratives in the deserts. Apparently the integration with tilesets wasn't satisfactory, so they were 'temporarily' removed. Well not removed completely, because if they were really removed then if they get put back they'd need to be respawned in existing games. Instead the decorations are still there in the game state, but they look like brown grass instead, just a temporary graphics swap. Unfortunately they were never put back.

Grassy desert

I wasn't a fan of the grassy desert so I enquired about changing it, but it seemed like this was an uncomfortable topic that no one really wanted to get into.

One evening I decided to clean up the old dune decorative graphics as a secret side project. I was already familiar with the assets because I'd used them in the Alien Biomes mod and I knew I could get something good done quickly.

This version uses relief effect tricks to better integrate with terrain. Basically making it much more transparent, but then adding low transparency pure white and black on top to modify the brightness of the terrain underneath.

Old dunes. New dunes.

Old dunes | New dunes

The result looks good on all 3 of the sand types, so I could be more flexible with the placement. The old decoratives used a "peaks" autoplace format (which I won't go into, but it's an older, more limited system). I replaced it with a new noise expressions formula that's adapted from my code on other planets. That lets the dunes fade in more gradually over multiple sand types and give a nice transition effect.

Old dunes. New dunes.

Old dunes | New dunes

The PR (pull request) to get the changes made in the main game was successful (and with no drama), so my stealth dunes project ended up as a first foot in the door to changing the Nauvis map generation.

Investigating missing decoratives

I started noticing that some of the other decoratives that I was familiar with from the graphics files didn't seem to appear in the world. Looking through the decorative placement code each one seemed to be more or less correct, or at least reasonable. They had the correct graphics, they were assigned to the correct planet, and their assigned spawn regions didn't have impossible values, so nothing obvious was wrong. But then I noticed something else, a pattern, a set of similarities. Some of the settings were uncomfortably similar, and in fact there were some that were identical to other decoratives.

That might not sound like a big deal but here's why it is. The placement of one decorative excludes the placement of other decoratives that would be placed after. They block each other. If they're trying to place in different places that's fine. If they have some overlap that's okay, the 2nd one will only appear where the 1st one didn't. If the settings are identical though, the 2nd decorative will never get placed anywhere.

So that's a problem, but also an opportunity.

This is a great way to use the skills learned from the other planets to improve the Nauvis decoratives. Not only can the new system be faster, but it can just look a lot better in terms of distribution and controlled placement.

I started mapping out all the Nauvis decoratives. There are 3 axis variables that control placement: moisture, temperature and aux. Aux is the "terrain type" variable that controls differences like the Nauvis yellow desert from the Nauvis red(-ish but still mostly yellow-brown) desert. These control the wide scale distribution, but there's also a "noise layer" variable that controls the smaller scale placement and patchiness and is like a 4th axis but with discrete increments.

It's good to map this out visually to check for unwanted overlaps, desired share of the possibility space, etc. There is a lot more overlap than I was expecting. Some decoratives were missing completely, but over half of them had their range heavily restricted by other decoratives getting in the way.

Mapping the 4D state isn't so bad, I've developed some systems for Space Exploration to deal with a 12D system, so 4D is a nice break in comparison. I can use the 3 dimensional axis in blender (XYZ) to display the main volumes occupied. The 4th axis for noise layer is the colour of the box, which in this case is really convenient because the noise layer is discrete and rangeless (it has a single value, not a min-max range).

Here's the fixed distribution. The previous state was all over the place, but I don't have a copy of that (and I don't want to do the data entry to recreate it).

Decorative placement volumes The new decorative placement volumes visualised in blender.

The old system had an extra limitation in that it can't do decorative groupings, like having large variants in the middle of a cluster and small ones further out. This wasn't problematic enough on it's own to deserve an upgrade, but being able to improve this is another side benefit.

The new system has everything remapped so that everything can be placed, but that's just the tip of the iceberg. Things have moved around to accentuate certain areas of the map. Deserts are drier. Some arid areas have more red plants, other areas more white plants. In the grassy areas there are many subregions with different mixes of plants and more clustering.

Sure most of it will get paved with concrete, but it feels more meaningful if that was the only area on your map (so far) that looked this way.

Nauvis cliffs

Cliffs are annoying, right? You think you'll be able to get rid of them in a few minutes so you build like they're not even there, but they just get in the way. The concept of a cliff fort sounds cool, maybe you've experimented with placing some turrets defended by cliffs, but it never works as well as you hope. Large lakes are useful barriers, but cliffs just aren't. Why is that?

The first reason is that cliffs have loads of gaps. Every gap needs to be defended, so if you have a gap in the cliff every 6-ish cliff segments then you more or less need continuous turret coverage anyway. Yes the cliff means you need fewer walls, but the enemies only need 1 gap to get through, and following a cliff is harder to build around than a good defended wall blueprint.

The second reason is that cliffs follow elevation bands. Think about it. A lake is a good defensive obstacle. You'd probably make walls as lines between lakes if they're conveniently placed. The cliffs, though, use the same elevation data as the lake. This means that in most cases the cliff is just an extra ring around the lake, and that's not useful. A cliff parallel to a lake is mainly defending against the lake direction, but there's already a lake there doing a better job, (or it's defending the coast from inland but that's not often useful either). In most cases the cliff ends up in the way if you try to make a wall between two lakes, and often it gets in the way twice.

Cliffs in Factorio 1.1 Cliffs in 1.1. (High cliffiness to show the lines).

For Nauvis the real objective becomes clear. If we're stuck with cliffs here, then they should exist in the most useful, least annoying way possible. This 'simple' idea ended up turning into a planet generation overhaul.

Mesas terrain

The first attempt was based around the idea of having a series of defensible plateaus. They'd have some cliffs around them, but they wouldn't follow the same lines as the lakes. To do this the land above water was mostly flattened and then the hills were added to the existing land separately.

The result was better than 1.1, but still not good enough. The cliffs wouldn't be in the way as much as 1.1, but they were only marginally more useful than before. The problem was a lack of good control over where the gaps in the cliffs go. Too many gaps in the cliff mean they're not useful as barriers against enemies while still getting in the way of building.

Cliffs in Nauvis rebuild attempt 1. Cliffs in Nauvis rebuild attempt 1. (High cliffiness to show the lines).

What's really needed is longer unbroken cliffs, but fewer cliffs in total. Sadly, there was no way to do that.

Let's go on a tangent for a moment.

Precise cliffs

I usually try not to look at the C++ code because every time I do I get compelled to start changing things… (like the fluid mechanics) when really I should be focussing on other things. The call of the cliffs was just too strong, and before I knew it I was rewriting some of the core cliff placement code.

The first time this happened was when I was working on Fulgora. It was impossible to make the cliffs following the intricately placed island edges. It just didn't seem to be possible to make the cliffs and tiles match up. I found that the reason is because the cliffs are "smoothed".

Smoothing is an attempt to make cliffs coherent even when the elevation data is noisy. It was needed on old Nauvis because the elevation gets messy sometimes.

The smoothing was a nightmare for planets that need more precise cliff placement like Fulgora. Fulgora also doesn't have an erratic elevation so the smoothing wouldn't do anything useful. My first change to the cliffs C++ code was adding a way to disable smoothing on some surfaces. Suddenly all the cliffs snapped into their proper place. The cliff problem on Fulgora had been plaguing me for weeks, so to say this was a relief is an understatement.

Fulgora with cliff smoothing. Fulgora without cliff smoothing.

With smoothing | Without smoothing
Please slide the white bar left and right. So satisfying.

Freeform cliffs

Back to the plan for Nauvis. I'd been thinking about the cliff situation for a while. Now that I was more familiar with the core cliff spawning code I saw a clear way of making some minor changes that would let the cliffs use a completely different noise expression for their placement.

This means that cliffs can go in directions completely independent of the planet's water. They'd use the water elevation by default, but a planet could specify a different expression if they want to.

Ideal cliff lines. Left: Lots of cliffs but none are useful.
Right: Fewer cliffs and has the potential to be useful.

The main plan was to make a small number of long cliffs as mentioned earlier. Fewer of them means they wouldn't get in the way as much. Long unbroken cliffs would be a better barrier against enemies, but it's also a better barrier to your base expansion, and that's a good thing.

If there's a huge cliff you're less likely to ignore it and build into it, you're more likely to build away from it. By affecting your large-scale building more, it affects your small-scale building less, and that's much less annoying.

Fewer cliffs inherently means they're also in the way less, but this is a secondary concern because you can always use the terrain controls to affect this as you like. The shape when they are there is the most important thing to justify their existence.

The plateau shapes from earlier were still a good starting place for cliff placement. It's a good starting shape for larger plateaus, but can also be used for effects like canyons. The biggest difference is scale, the features are just much larger.

Debug visualisation of cliff plateaus. Debug visualisation of cliff plateaus. Dark is a low area, bright is a high area.

This leads into another nice opportunity. The cliff "elevation" at this point is completely separate from the water elevation, but when you have a line of cliffs heading to the water, you expect the coastline to be different showing that the area above the cliff would extend further into the water as that area is higher.

So that's what I did. The cliff elevation is added to the water elevation. This creates some nice terrain features. River-like sections that lead to canyons, peninsulas guarded by cliffs, and some nice new islands.

A river-like formation. A river-like formation.

A canyon-like formation. A canyon-like formation.

Remaking the terrain

The problem is that adding the cliff elevation to water elevation decreases the amount of water. The water level can be adjusted to add more water back, but that mostly floods the canyons areas. It's the main disadvantage of the plateau shapes being surrounded by lower land, if the water level is higher it all ends up as disconnected islands. That's not ideal for Factorio because you get cut off from biters and need landfill or elevated rail to connect areas. It's better if most of the land stays connected.

Problem: Plateaus tend to create islands. Problem: Plateaus tend to create islands.

You may remember from FFF-390 that I already have a solution to this problem. Ridge-based elevation where you have narrow strips of connected land while the rest is mostly water in disconnected lakes. It's basically the opposite of the plateaus. Using this sort of noise to make more reliable land bridges, mixing in the base terrain, and then adding the raised cliff areas, gives a really nice result for gameplay, and it looks great too.

Plateaus Mixed terrain.

The same raw plateaus as before. | The actual terrain of plateaus mixed with other layers.
Notice that on the mixed terrain, the coastline gets affected by the plateau shapes, and inland areas get cliffs in the same pattern.

But wait, this has gone beyond just tweaking the terrain for better cliffs. This is getting towards overhaul territory. If it's already going to change that much, let's just make the Nauvis map the best it can be.

So that's what I did.

Let's start with the water slider. The minimum amount of water is almost all land, but there are occasional small lakes that are few and far between.

Min water, 100% scale Max water, 100% scale.

Min water, 100% scale. | Max water, 100% scale.

On the other extreme, maxed-out water results in nice-looking clusters of islands, an archipelago.

Min water, 25% scale Max water, 25% scale.

Min water, 25% scale. | Max water, 25% scale.

Once the terrain is set up to handle these extreme water values well, it allows for some macro-variation if the map is set to normal values too. On a "medium water" setting (the default), then as you get further from the surface middle the land starts to oscillate between these extremes to make some massive lakes and wide plains. It's much more varied than before.

Starting in deserts has been a common problem. I added an additional slider so that you can customise the starting area moisture without affecting the rest of the world. A forest start is significantly easier.

The terrain types for above and below cliffs have been adjusted very slightly so that sometimes there's a slight terrain difference. It's a small point, but makes it feel more 3d and integrated into the world.

A dead end

A lot of things were working really well. The terrain looked much better, it was better to play on, but it wasn't perfect.

A canyon path blocked by water. A canyon path blocked by water.

When exploring by car, a lot of the old problems with exploration and navigation were still present. You follow a gap between forests but there's a small lake blocking the path. You follow a line of cliffs for a gap, but it's blocked by trees. But now there's a new problem where you can follow a land bridge, but then there's a perpendicular cliff blocking your path.

A tree path blocked by a cliff. A tree path blocked by a cliff.

The problem is the cliff system, the land bridge system, and the trees system don't know anything about each other. That's not really solvable though, because the tree system still used the same 'peaks' definitions as the decoratives mentioned earlier.

A landbridge blocked by trees. A landbridge blocked by trees.

So I changed that too.

Forest noise

Like with the decoratives, the more I looked into the trees system the more problems I found. In general it was a bit better because 'most' of the trees were capable of spawning, but there was one tree that was completely excluded because it required a temperature that didn't exist on the planet.

Tree placement volumes The cage boxes show the new tree placement volumes, except the box for 1 tree has been put back to it's original place. Can you guess which one it is?

These sorts of problems are very hard to prevent. It takes a huge amount of organisation and understanding to find a lot of these problems. Even then it takes a while to build a complete picture of what's going on, so when some small thing needs to be adjusted, like adjusting the noise that generates temperature, it's difficult to tell if you've accidentally made a particular tree never spawn. If you change the noise layer of a decorative, you check it and that decorative looks fine, but it's hard to tell if you've made some other random decorative never spawn. We do have tests to warn against accidental changes but sometimes these negatives get mixed in with other changes where it's expected that map generation will have changed so they don't get noticed.

Anyway, as with the decoratives, the tree placement has been changed to make areas more characterful.

As before dry areas generally have reduced tree density while moist areas have more but the difference is more extreme and more consistent. The absolute zero-moisture desert now doesn't have any trees at all.

Do any of these trees look new to you?

All of this looks nice, which is great, but it's really all for the end goal of integrating the trees with the other systems for better gameplay.

Natural paths

This is what really ties everything together.

The land bridges are defined by ridges, meaning there's a continuous ribbon that determines the high point. When there's a land bridge, this ribbon line travels roughly down the middle of the strip of land. These ribbons are areas that should be kept clear, so they're added to a list of paths.

The plateaus actually use a very similar system to the ridges but it's inverted. The ribbon lines instead of being high points are low points, essentially the "canyon" areas between the hills. If you're following a cliff edge from below you'll often get funnelled into these areas, so it's important that these areas are not blocked by trees, so they're also added to paths.

Dense forests are also a problem for moving around especially if you set the tree density slider to the max and try to move in a moist section of the landscape. An extra set of lines for forest trails was added to the set of paths too.

All of these path lines are combined and subtracted from both cliff and tree placement. The result is extremely convenient and feels natural. Forest trails would be the result of animal movement, and it makes sense for these to be connected to the gaps in cliffs because that's where they can get through. If you're following a path over a land bridge then there's always a way through any cliffs and trees you might encounter.

Debug visualisation of paths. Max-trees landscape showing paths.

Left: Red = Canyon path, Blue = Landbridge path, Green = tree path.
Right: A landscape with maximum tree settings to more clearly show the paths.

There are of course exceptions where things block each other, but these tend to be coincidences. For example, terrain just happens to connect between two lakes making what is effectively a land bridge, but because it was not created by the special land bridge system, the route is not protected and it has a chance to get clogged with trees.

The true test of this for me, was to set trees and cliffs to maximum settings and try to navigate the map by car. And you know what? That's actually a really cool map to move around on now.

Finishing touches

Here are the results:

Map at 100% scale. Map at 100% scale.

Map at 50% scale. Map at 50% scale. Note: When changing scale only elevation-style features are resized. Things like ore stay in the same place.

Map at 25% scale. Map at 25% scale.

Map at 25% scale. Another map at 25% scale.

I've put a lot of work into this and I've been testing it heavily through the entire process, it's finally working really well. There's still more testing to do because it's such a big change and it affects everyone in a very significant way. There are some more minor things to do a bit later, like more patchy forest edges, but that's not a significant change to gameplay.

The safety net for all these changes is that the new terrain is extremely customisable. Even if you don't like some setting it's easy to adjust the map the way you like it. For example, I made some extra changes to the cliff frequency slider so that a value below 100% will remove cliffs from huge regions of the map.

Also if you really like the old terrain it's still there along with the single-island mapgen option.

As always, follow the trails to the usual places.