One profile per player and a copy that follows it.

Warning

Rod and Reef hasn’t been released yet, and videos or GIFs of this system can’t be shown until it launches.

Abstract. A Rod and Reef player’s data (coins, fish, rods, quests, luck counters, achievements and season) lives in a ProfileStore profile, a library that stores profiles in DataStores behind a session lock, so only one server at a time can write to each. On top of it sits a thin layer, DataManager, where every write changes the profile’s live table and notifies the client in the same call. The client keeps a copy built from a full snapshot on join and a sequence of per-path changes afterwards, which converges to the server’s state as long as it receives the changes in order. This article also describes three decisions around that layer: idempotent migrations that run on every join, preferences stored outside the profile so the menu place can write them, and messages between servers that let a player gift something to someone who is on another server or offline.

1. The problem

Player data is the one thing in the game that can’t be lost or duplicated. A fish that disappears is a bug report, and duplicated coins are a broken economy. Roblox runs every match on a different server, a player can hop from one to another in seconds, and a server can go down at any moment. On top of that, the client needs its own data to draw almost the entire UI, from the inventory to the quest journal, and it needs it as soon as it joins, racing everything else that starts at the same time.

2. Sessions

2.1 The lock

When a player joins, the server opens a ProfileStore [3] session on their profile, stored under the key Player_<UserId>. A session is a lock: while a server holds it, no other server can write to that profile. If another server asks for it, it waits, and if the holder doesn’t release it within 40 s, it steals it. A profile that hasn’t been updated for more than 630 s is assumed to belong to a crashed server. It’s essentially a lease, a permission with an expiry [1]: the lock doesn’t depend on the holder staying alive to say it’s letting go. Changes are saved every 300 s and when the session ends.

The session request is cancelled if the player leaves while it’s waiting, so it doesn’t end up holding a lock for someone who’s already gone. If the session ends for any reason while the player is still in the game, for example because another server stole it, the player is kicked, because playing on would mean writing to a profile that’s no longer theirs and losing everything they do from then on. The profile is also associated with the player’s UserId, which is what makes it possible to erase their data if they ask.

2.2 A template and a generation

The profile’s shape is defined by a template of about 460 lines. On load, Reconcile adds any template key the profile is missing, so a new field shows up in old profiles without migrating anything. The store’s name is a generation, 20260814, and Studio uses a different one, Debug, so tests never touch real data. Changing the generation is how all data gets wiped, for example between a beta and launch.

3. Idempotent migrations

Reconcile fills in missing keys, but it can’t move a value that’s sitting in the wrong key, which is exactly what a change of shape leaves behind. That’s what migrate is for: it runs on every join, after Reconcile, so every key it reads exists.

The rule is that every step is idempotent, $m (m (x)) = m (x)$ . With that, there’s no need to store a version number or remember which profile was already migrated: applying the migration to a migrated profile does nothing, because the conditions that trigger it no longer hold. Today it has three steps.

The first comes from bait. There used to be one slot, and the three paid baits now go into a second one. A profile saved before the change can have one of those baits in the primary slot. It still works there, but the inventory draws it as unequipped and offers to equip it again, so the step moves it to the second slot, only if that slot is empty, so nothing is thrown away.

The second comes from an item type that disappeared. Gear (the diving mask and the cold-water suit) became quest items, and the mask was folded into the goggles. The step walks the old list, moves it into the new one with the rename and without duplicating anything already there, and deletes the old key.

The third comes from emotes, which lost the spaces in their names (Rat Dance became RatDance), and nobody had renamed the ones already saved. Not even the template’s starter emote, so every profile created since then owned an emote under a name that no longer existed. The step renames every saved name whose space-less version exists, or drops it if the player already owns the new spelling.

In all three cases, after being applied once the condition that triggers the step is false, and that’s what makes them idempotent.

4. DataManager

4.1 Writing is replicating

DataManager keeps a reference to each player’s profile data table, the same table ProfileStore saves, not a copy. It has four main operations: read and write a key, and read and write a nested path, like {"Achievements", "Titles", "Owned", key}. Writing changes the table, and since it’s the profile’s table, the change is saved on the next autosave. In the same call, the client is sent an event with the path joined by dots and the new value:

set (path, v) \Rightarrow S [path] \leftarrow v \land send ("𝙰.𝙱.𝙲", v)

From that comes a convention every piece of server code follows: copy, modify, write. The season service, for example, clones the season block, changes the counters and writes the whole block back. Modifying a profile table in place would still save the change, but it wouldn’t tell the client, because nothing went through the function that replicates.

4.2 Waiting for the right thing

There are two waits that look the same and aren’t. GetProfile waits for the player’s session to exist, and WaitForData waits for their data table to exist. There’s a window between the two, because after opening the session the server still loads the player’s preferences from another store (section 6), and only then publishes the table. A system that waited for the session and then read a saved value found it empty, and for a progress value “empty” means “this player never had one”, which is very different from “it hasn’t arrived yet”. That’s what happened to the routine that releases disabled quests: it saw a profile with no active quests and did nothing. Anything that reads saved data has to wait for the data, not the session.

5. The client’s copy

5.1 Snapshot plus changes

The client keeps a copy of its data built in two parts. On join it receives a full snapshot, $S_{j}$ , and afterwards it receives each change $o_{i} = ({path}_{i}, v_{i})$ in the order the server made them. Applying a change means assigning the value at the path, creating any missing intermediate tables:

S_{k} = o_{k} \circ o_{k - 1} \circ \dots \circ o_{j + 1} (S_{j})

If the client has the snapshot and every later change in order, its copy equals the server’s state. Roblox remote events are reliable and ordered per player, so the condition holds. It’s the same scheme as a replicated database that starts from a snapshot and then replays a change log [2]. Since the server is the only writer, there are no conflicts to resolve: the last value at each path is the right one.

5.2 The join race

The problem is the snapshot. The server pushes it as soon as the data is ready, but the client script listening for it can connect after the push has already gone by. For that, the client can ask for it explicitly. If the server doesn’t have the data yet, it waits up to 15 s and replies, and if it can’t get it in that time it kicks the player, because a client without its data can’t play.

On the client side, PlayerDataClient offers the copy through a promise. If it already has it, it resolves immediately. If not, it registers the function that will resolve it before sending the request, so a reply that comes back very fast isn’t lost, and gives it a 10 s timeout. A DataChanged signal announces every change, with the path and the value, or with no path when a full snapshot arrives. Consumers filter by path prefix, and many of them only flag that they need to redraw and do it on the next frame, rather than reacting to every change.

5.3 Two channels

Player data travels over regular remote events, because the values are arbitrary tables. Game events with a known shape (a catch, quest progress, a season board entry) travel separately through ByteNet, which serializes them in binary with a schema per packet. So the data channel is generic and the event channel is compact.

6. Preferences outside the profile

Player preferences (volume, graphics, subtitles and so on) used to live inside the profile. That meant the menu place, which is a separate Roblox place from the game, could only display them. To write them it would have had to open a session on the profile, and then the game would have had to wait for the menu to release the lock every time the player pressed “Play”.

Now preferences live in a separate store, with no ProfileStore and no lock, written by both the game and the menu. Neither asks the other for permission, because the player is only ever in one of them at a time. The accepted cost is that a write can be lost: without a session, two servers holding the same player for a moment, during a teleport, can overwrite each other. That’s a slider position, and it’s a fair price for being writable from anywhere. The rule is that nothing that can be earned, spent or stolen goes in there.

Writes go through a throttle with a leading write. The first change is written immediately, which is what lets a setting changed seconds before pressing “Play” survive the trip, and after that at most one write every 6 s per player, with the last state always written on the way out. Dragging a slider produces changes much faster than the write budget allows, and without the throttle it would spend all of it. On joining the game, preferences are copied into the data sent to the client, but as a copy, so the profile doesn’t end up saving the live preferences table.

7. Messages between servers

A player can gift something to someone who is on another server or offline. Writing into the other player’s profile isn’t possible, because the profile is locked and only its current holder can write. ProfileStore has a message queue per profile: sending a message adds it to the recipient’s queue and pings MessagingService, so if the recipient is online they get it within a second or two, and if not, they get it the next time their session starts.

Delivery is at least once. A handler receives each message and returns whether it consumed it, and it must only return that after actually writing whatever the message asked for. A handler that marks a gift as consumed and then fails before granting it has thrown it away. A message nobody consumes stays in the queue and is offered again, which is what guarantees a gift is never lost. For the same reason, purchases record the id of every receipt already processed, so a repeated delivery doesn’t grant twice.

Registration order matters. ProfileStore delivers queued messages as soon as a handler exists, including the ones that arrived while the player was offline. Handlers write through DataManager, which reads the published table, so if the handler were registered before the table was published, the gift would be lost and also marked as consumed. That’s why it’s registered afterwards. A message that isn’t a table is consumed immediately, so it doesn’t sit in the queue being offered on every join forever, and every handler runs protected, so one that fails doesn’t take the others down.

8. Leaving

When the player leaves, the time of their last session is written and then the session is closed. The order makes “last seen” mean the end of a full session and not the time of the last autosave, because the stamp goes into the final save.

9. Limitations

Changes send the full value at the path that was written, so writing a large block to change one number sends the whole block. The copy-modify-write convention depends on nobody forgetting it: an in-place change is saved but not replicated, and the client stays out of date until the next change at that path. DataManager has no strict types, because a path is a list of strings and Luau can’t check it against the template’s shape. And the quest system keeps its own copy on the client from the same stream of changes, so there are two copies of the same data that have to follow the same protocol.

10. What I took away

Client-side state built from a snapshot and an ordered sequence of changes is easy to reason about, because the question of whether it’s right reduces to whether everything arrived in order. Idempotent migrations saved me the version bookkeeping, which is where half-migrated profiles usually come from. And the preferences taught me that not all data deserves the same guarantees. Splitting it by how much it costs to lose made it possible to write some of it from anywhere without putting the rest at risk.

References

C. G. Gray and D. R. Cheriton. “Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency”. Proceedings of the 12th ACM Symposium on Operating Systems Principles, 1989.
M. Kleppmann. Designing Data-Intensive Applications. O’Reilly, 2017.
loleris (MAD STUDIO). ProfileStore. github.com/MadStudioRoblox/ProfileStore.