Fluid mixing saga Dominik Hello Factorians, Today I would like to talk to you about my favourite subject - fluid mixing and its prevention. It is a new mechanic introduced in 0.17 that seemed quite simple at first, but has been giving me nightmares ever since. A while ago I took on the task of updating the fluid system (FFF-260). The way it works in 0.16 is that the fluid boxes (the thing that holds the fluid and is contained in entities like pipes or refineries) had no organisation whatsoever except their connections. They would sit there and do nothing and then once per update send fluid somewhere. It had it's issues especially with symmetry, and when you happened to put some fluid where it did not belong, you could end up requiring major demolition works. My task was to develop a better algorithm to move the fluids, and a very very optional side task I had in mind was to do something about the fluids getting to wrong places and mixing. We started by organizing the fluid boxes into systems (connected FBs make a system) managed by a special fluid manager, and optimizing the heck out of it (FFF-271). Soon after, I started working on the new algorithm and anti fluid-mixing. Until then nobody really considered preventing the fluid mixing seriously as it did not seem possible. But when I thought about it, it did not seem that hard. The idea is to simply not allow any action that would lead to fluids getting where they are not supposed to go. When you build a steam pipeline, you should not need nor want it to touch another fluid. In principle, it would only require a few quite realistic mechanisms: A connected block of fluid boxes would either be fluid-free, or it would be locked to one fluid. Two such blocks can connect only if they are compatible - either one has no fluid lock, or they have the same fluid. An assembler can only set a recipe if it is compatible with the fluid locks on its fluid boxes. Have a migration from old saves that can contain mixing. By creating the fluid systems right before, number 1 was almost finished and we only needed to assign the lock. It would be set either by a fluid, or by a 'filter', which is a fluidbox that is set to use a fluid - such as a water pump using water, or assembler having some inputs/outputs given by a recipe. After a while I had both the fluid algorithm and the mixing done (FFF-274). The mixing was not that easy (like 5x more complicated) but it worked pretty well. As for the fluid algorithm, V453000 and Twinsen found some issues with waves on a macro scale, and because it was right before releasing the 0.17 experimental, we decided to hold it off on it for the time being (we have a new version now that seems okay, but has to wait for 0.17 becoming stable first). The mixing made it through though and seemed quite finished. It turned out that the work on it was at most one tenth complete. Some difficulties appeared already before the initial release, but that was only a hint of what would come later. One by one, then ten by ten, bugs started coming. The problem is that as often as not, they were not just little issues with simple fixes. Instead, over and over, they turned the current solution on its head and the code had to be completely rewritten in order to account for the new case. As of now, almost exactly half a year and many rewrites later, it is still not fully bug free. The number one issue was with Assembling machines. It turned out that their code was pretty messy and does not simply allow the checks I needed, so it had to be refactored. The next problem was that the check was not only required when setting a recipe on an existing assembler - as I naively thought - but in about one million different situations I would not dream existed. Building a fixed recipe assembler. Reviving an assembler. Rotating. Blueprint of an assembler. Blueprint with a recipe and a rotation placed over a ghost with a non-default rotation during a full moon. Teleport. Script building. Copy-paste settings. Any combination of these. Pretty much every case would go through different paths in the code and behave entirely differently. Fixing one would break another and so tons of tests had to be written. When these cases finally worked, it turned out that doing these operations when they fail (e.g. can’t rotate because it would cause mixing) brings another wave of issues. And then mods come and the complexity multiplies. All this, although in a more simple form, had to be done for all other entities that can use fluid too, such as inserters (mods...). Another number one issue are underground connections. When playing, you would not think twice about them but on a closer look they are all but simple. They have this (cough) uncomfortable feature (want to shoot myself) that you can build another underground pipe between them (or take it out) and a complex reconnection logic jumps in. It means that based on building order, connections are established and reestablished differently. This is a big thing when building a blueprint with many pipes, or loading a save game. But the largest issue is one tiny corner case with huge implications. Have two underground pipes that have different fluids, but are split by a third, and it gets removed. Until now mixing was theoretically completely avoidable, but here it was and there was nothing to stop it. As a result, a new concept is introduced - a blocked connection. An underground connection that would normally connect but can’t. Establishing it is the easier part. But when does it get fixed? An entity miles away gets rotated, that splits a fluid system, disconnecting the blocked connection from a fluid filter on the other side, and the connection should unblock. But even something as simple as a rotation contains fluid fixes and re-establishing of fluid boxes and if it fails, it still fixes the blocked connection in the process and it can’t be undone... You get the picture. And we are still talking about underground pipes, while anything can have underground connections, including said assemblers, which the previous code did not consider at all. The complexity just goes deeper and deeper, and Rseding is probably right that we should have never gone this way at all. So the bug fixing has been basically running in circles - clearing one batch of bugs, thinking that those were the last ones and the ordeal is finally over, only to find another batch (ideally from some bizzare modder idea) a few days later. Many times I wished that mods never existed. Nor multiplayer, or any players at all for that matter. About two months ago all seemed really fine, with basically no reports and just a few crashes in the automated crash reports. At that moment another nightmare materialized in the form of a talented volunteer bug tester named boskid, who took on a personal crusade to break the fluids to dust (I actually imagine him sitting in his dark room with evil laughter, dreaming about making my next day worse than the previous). In all seriousness, he has done a great deal of work with his bug testing, creating bizzare modded cases and testing scripts, and deserves a big thanks. He is actually coming to our offices next week, so follow the news for reports of developers falling out of windows. An example of a nice configuration he found (and I can’t fix). The whole time we were taking the offensive approach to mixing - crash the game when it happens - so that we know about bugs to fix them. Recently we have decided to put a stop to it and have the mixing automatically fix itself once it is detected, eyeing the end of this episode. Right now I have 3 mixing bugs on the list and I am sure those are the last, and mixing will be done (lying to myself is a way to cope with it) and the new fluid algorithm can come soon after :).

Hello, Another week has elapsed, which brings us another week deeper into the declining weather of autumn.

A few bugs left... WoW classic has been released, which means several of our top men have taken time off to spend hours raiding and having fun in Azeroth. This isn't great timing, as a few new bugs related to train signals appeared. We want to get these bugs solved before we do another release (another stable candidate). As it turns out, the only person with the skillset to fix these specific train signal bugs is also deep into levelling his Priest... (Kov on Pyrewood Village) We are still making the rest of the preparations for the stable release. We have started writing up the stable annoucement blog post, and have produced a 0.17 postcard image. Other than the few more critical rail bugs, there doesn't look like there is much else to block the stable release, on the forum we are down to 27 bugs. Since there are so few bugs left to deal with, some of the team has starting working on 'post-stable' features. Wheybags is working on the new campaign, Oxyd is diving into some detailed pathfinder improvements, and Rseding has started work on some particle optimizations. We will delve deeper into these topics and more in future FFFs, as we always love to do.

Crash site: First state Ernestas, V453000, Albert For many weeks now the GFX department has been focused on preparing replacements for the placeholder graphics of the campaign crash site. The subject as usual is not that easy because we had to first solve the main concept of the crash site. It happens that those new entities belong to the Factorio universe, but they come from a different reality than the usual DIY/diesel punk of the game. So we had to invent a new way to design machines that look like Factorio but that are not too familiar. Here a proof of concept of the look: The concept is that the big (medium) spaceship broke into pieces as it crash landed, and lost many components that the player, during the introduction, will repair and use for his own profit. The look of the spaceship remnants are a little bit based on the designs of the 60’s/70’s pulp Sci-Fi. Fortunately we can keep the look of Factorio due the accident, which allows us to destroy and dirty up the machines show many inner mechanical details. It is also part of the concept that all the machines that the player builds, are based on an existing technology from his home planet. So the machines that we see in the regular game are like 'cheap' versions of the original ones. For the lab, we keep the dome shape and the beams inside in order to keep consistency with the regular laboratory. So slowly the player gets used to the meaning of the shapes. The generator is similar to a substation -more or less-, connectable like a pole but it also produces electricity. Sometimes we really have to invent. This works like an assembling machine. The design is more based in the (yet unshown) redesign of the assembling machines, rather than on the actual 'classic' ones. These cylinders are like chests. We decided to make them cylindrical instead of a box for this difference in technology level that we are speaking about. The player will recognize them for the shape, color, and they also always have a number printed. You don’t really want to know the meaning of the numbers. All this new content is a work in progress, and we made these new entities first for the testing of the campaign. Based on feedback with testers we will have the chance to tweak and adapt whatever is necessary. In the case of the introduction, the positioning of the entities can have a large impact on the flow of the gameplay. Once we are more sure of the final placement, we can see how all the pieces will fit together. The next round for the crash site is the main crashed spaceship, and some other assets that converts the scene into a full composition, more proper for the introduction of the game.

Hello, the bugfixing period boringly continues, we got down to 159 active bug reports, so in few weeks we should be finally down with this burden. But at least the graphics department has something new to show:

More remnants Dom, albert As for the conclusion of the topic opened in the FFF-288 we finally decided to go for a balanced solution in which the player can recognize what entity was destroyed and also walk through them. The solution of going up in the Z-axis is really attractive for designing the destroyed models, but it gives a lot of headaches to the game designers due its need of a bounding box. We are planning also a generic set of remnants for the modders who don’t feel like making the destroyed version of their own machines. Here an example of how a factory can look after a biter attack: These remnants are still a work in progress, many of them are not yet finished and we are planning more iterations with this subject. By now you'll find a sneak peek in today's release (0.17.36).

New Campaign Abregado Have you ever been playing a Freeplay game and realised you don't know what your next big goal is? And then, once you decide to pick a new goal, you realise while you worked on automating the last goal, there were 10 new technologies unlocked and now you don't know which to pick next. These are the situations we hope to address with the new full Campaign. A guided Freeplay, in which the player plays through the whole tech tree, without being overloaded with choice, while still having the permanence and unidirectional progression of Factorio. The permanence problem has already been solved using the new map expansion technique which is playable in the Introduction scenario. Over the last year we have been working on the bigger design task of unravelling the tech tree and breaking it into a set of choices for the player. This task has been made all the more Interesting as the tech tree is also constantly getting tweaks and revisions over that time as well. I look forward to providing more insights but for now I will leave you with one example (read: spoiler): Just to note, we won't be changing the freeplay tech tree, which will still have all the choice and diverging paths as it does now.

Rail building changes kovarex The problem with rail building is that it has too many states. It depends whether you start building the rail with shift, to use the ghost mode or not, and then it also matters whether you still hold shift, to ignore trees or not. Moving from manual building to ghost rail building means cancelling the whole rail building and starting it again with the correct modifier. The problems were reaching the surface from time to time, and Twinsen even drew a nice little state diagram of the rail building system. It kind of peaked with this bug report. After some time, it became clear that we should simplify it. This is a nice example, where we can talk about change we just released (0.17.29) in a FFF. From now on, instead of 3 modes (building manually, ghost building, ghost building + tree/rock removal), we have only 2 modes, where the ghost building without obstacle deconstruction is not available anymore. So it doesn't matter anymore how you start the building, it just matters whether you are holding shift or not at the moment, which makes it consistent with the normal entity building and the ghost icon. Once the topic is opened, it is hard to leave it so easily, so we agreed that the rail building could be streamlined even more. The current model is, that you have to build one straight piece of rail first, and then you can use the rail builder on the edge of that rail. It is annoying generally, and especially for the new players, because he might miss the way how it is being used, as the straight rails are built exactly the same way as other entities. The reason we did it this way was mainly, because when we first introduced the (new at that time) rail builder, we weren't confident enough that it could replace even the basic straight rail building. But since the rail building is stable enough, we might go one more step forward. The plan is that the rail won't be used to build straight rails normally the way you can now. The rail item would be always used to build by the rail planner only. Instead of having to snap to an existing rail, you could just start building anywhere on the ground. Instead of showing the rail preview when holding the item, you would see the arrow as when starting the rail building, and you could rotate it with the R key. It would make sense, to make the arrow color/size different for starting a rail planner on the ground versus snapping to an existing rail, but other than that, it would be the same.

Hello, we are still focusing most of our resources towards fixing as many bugs as possible so we have stable release in reasonable time. In the meantime, the preparation for the continuation of the work on the GUI rewrite is still happening:

Removing RTL language translations Wheybags I'm sorry to say that we have removed the RTL language translations (Hebrew and Arabic) in 0.17.20. Until this point we've had a half implementation of RTL languages, where the text is simply flipped when we download it from Crowdin. This 'works' for a decent proportion of things, but not nearly 100%. In order to attain the level of polish we want for the 1.0 release, we would need to spend a lot of time implementing proper support for RTL layouts. This just doesn't make sense for us given our current goals, and the proportion of our player base which uses these languages (less than 0.1%). We decided that instead of completely gutting the translations, we could leave them in for those who enjoy them, but not to offer them in the GUI as defaults. The languages will remain up on Crowdin, and the locale files will still be present in game, but there will be no option in the in-game language options dialog to choose them. If you want to use an RTL language, you will have to manually edit your config file to set your locale. Detailed instructions are available on our forum. What this also means, is that we won't be investigating any bug reports about RTL issues.