HKMP Godseeker and Steel Soul saves

It's been a couple of weeks, but I have another development update on the new HKMP 3.0. I've been working on some changes regarding the save file system that will make my life easier for future development and in turn make your lives easier. In this post I'll describe the changes and why they are impactful on saves.

If you've been following along with HKMP development recently (and perhaps have tried out the builds), you'll know that HKMP 3.0 features save file synchronisation. This is a system that allow synchronisation of select save data across clients. For example, whenever player A opens a breakable wall somewhere, this wall will be broken player B as well. This allows players to experience the exact same world wherever they are along with synchronised entities.

The system is split up into two distinct parts. One part is the so-called global save data, which includes broken walls, defeated bosses, hit levers, etc. The other part is the player save data, which stores (for each player individually) nail upgrades, obtained charms, mask fragments, relics, etc. When you host a server in-game, the player save data will be stored with the hosting player's save file. Meaning that if you join that same host in another session on that save file, your player save data will be loaded again. The global save data is already stored in Hollow Knight by default, so there's no need to store it separately. For standalone servers, all save data (global and player save data) is stored in a separate file.

This save data synchronisation system has seen some updates recently, and unfortunately, with each update this meant that these save files would become outdated and unusable. The reason for this was that the values in these save files were encoded in a way that was changed each time I made even slight changes to its functionality. Let me explain how the implementation works.

If you've read some of the other development posts on here, you'll know how important it is to limit network bandwidth as much as possible. The save data in Hollow Knight consists of a bunch variable names with corresponding values. Naturally, it won't be very efficient to simply transit this data as is over the network. We can do much better than this, namely by assigning each of these variables to a numeric index. For example, the variable named "maxHealth" would get the index "1", the variable "geo" would get the index "2", and so on. Now, whenever the value for one of these variables changes, we simply network to the server: variable with index 2 changed to 32 for example.

Now on the server-side, whenever we receive such an index, there's no need to decode it back to the variable name, because we only send it back to other players or save it in our save file. This previously meant that in order to store this save data, we would store the indices and values as raw bytes into the save file. Then, when I made an update to these indices, for example by either adding or removing a variable that needed to be synchronised, the stored save file contained incorrect indices for some of the values and couldn't be read.

The next logical step was to turn these indices back into their variable names on the server as well, and only store them in that form. That way, the stored save files always contain the correct values, even if the indices are updated. Unfortunately, there is one problem with this approach. The values of the variables are encoded before they are sent to the server. This encoding is based on the type of value. For example, the variable "geo" is an integer (a numeric value), but the variable "hasDash" is a boolean (either true of false). In order to decode these values again on the server-side, we'd need to know what their type. But keep in mind that the standalone has no reference to the game at all (that's why it is called "standalone"). So while on the client-side we can simply lookup the type of the variable if we have its name, on the server-side we can't do this. And if we can't decode the values, we can't easily store the save files in a readable format.

To remedy this, I've added a mapping that stores for each variable name the corresponding type and modified the encoding/decoding implementation to use this mapping. This means that the server-side can now simply lookup the type for each variable, without needing access to the Hollow Knight code.

There were a few minor hiccups along the way of this implementation, but I'll spare you the details for now. In summary, this change means that save files are now stored identically to how vanilla Hollow Knight stores save files, and enables users to easily modify them if needed. Moreover, future updates should no longer invalidate old save files.

Given that I was updating the save file related implementation, I also tackled some other outstanding issues regarding Godseeker and Steel Soul saves. The gamemode of the save files was previously not synchronised, as such joining players would not be start in Godhome with abilities or have the Steel Soul UI. With the new update, these gamemodes are working properly. For Godseeker mode, this makes it so that newly joining players start with full abilities and start in Godhome, unable to leave. For Steel Soul the UI is enabled for players and dying will wipe player-specific data. Moreover, if a player hosts in-game and dies, the save file will still be deleted and everyone disconnected, so beware!

As usual, I've attached the latest development build to this post. Feel free to try it out and let me know in the Discord server (https://discord.gg/KbgxvDyzHP) if you have any questions or concerns. I will also be hosting a Patreon-exclusive standalone server running on Steel Soul so we can try to beat the game together. Let me know if the game is beaten, so I can reset the server. More information in the Patreon-exclusive Discord channel.