Snapshots: Actor and entity synchronization considerations

An entity in the SpatialOS Runtime represents an Actor in a game. There may be small discrepancies between an Actor’s state and an entity’s state which can affect the accuracy of a snapshot.

The SpatialOS Runtime updates data in an entity to reflect updates to data in an Actor in the game. The entity data updates very slightly later than the Actor data updates. Because snapshots represent entity data and not Actor data, there may be small discrepancies between an Actor’s state and the state of the entity which represents it in a snapshot.

Because of this, you must take account of several considerations when you use snapshots.

Terminology on this page

While a game is running, a server-worker instance sends requests to the SpatialOS Runtime for it to make changes to the game’s entity database. It’s the SpatialOS Runtime which makes the changes. The changes could be: update an entity’s component data, delete an entity, or replicate an Actor.

For the sake of simplicity, on this page we say that:

  • a server-worker instance makes the changes in place of: a server-worker instance requests the SpatialOS Runtime makes the changes.
  • a server-worker instance updates an entity, in place of: a server-worker instance requests the SpatialOS Runtime to update an entity’s component data.
  • a server-worker instance replicates an Actor, in place of: a server-worker instance requests the SpatialOS Runtime to replicate an Actor.

Entity creation after Actor creation

Issue:
When a server-worker instance creates an Actor, the entity which represents it in the SpatialOS world does not exist until the server-worker instance replicates the Actor.

Because an entity does not exist in the entity database until the Runtime receives a replicate Actor request, the replicate-Actor request is really a create-entity request.

If you take a snapshot after the server-worker instance creates the Actor but before the server-worker instance creates the corresponding entity, then the snapshot cannot store the entity. When you later use the snapshot, the entity doesn’t exist in the game and therefore the corresponding Actor cannot exist. This means that you must take care when you reference an Actor before its corresponding entity exists.

Solution:

To mitigate this, set up your game so Actors do not interact with a new Actor before the SpatialOS Runtime gives a server-worker instance authority over the new Actor.

For example, if a parent Actor needs to control the life-cycle of a child Actor, ensure that the child Actor sends a message to its parent that it now exists only after a server-worker instance has gained authority over it. Do not assume that the child Actor is ready to interact with other Actors until that point.

Entity deletion after Actor deletion

Issue:
When a server-worker instance deletes an Actor, it also deletes the corresponding entity.

If you take a snapshot after the server-worker deletes the Actor, but before it deletes the entity, then the Actor still exists in the game when you use the snapshot.

Solution:
The following are best-practice suggestions:

  • In your game design, when a server-worker instance deletes an Actor, it might need to update another Actor. For example, a rabbit dies when a wolf eats it, and the wolf gains health points. To avoid synchronization issues, make sure that your game design does not assume that a server-worker instance updates the other Actor (the wolf) only once.
  • If possible, delete Actors based on the values of their replicated fields (that is; their corresponding entity’s component properties). For example, it might be appropriate to delete an Actor if its Health field reaches 0. Then, when you use a snapshot, you can delete that Actor stored in the snapshot if the Health field is 0

State consistency

Issue:
If a server-worker instance replicates an Actor close to the time that you take a snapshot, the Runtime might save the Actor in the snapshot in a half-replicated state.

Solution:
The following are best-practice suggestions:

  • Try to take a snapshot at a time when the game is running as few processes as possible.
  • Consider adding a “Shutting Down” state to your Actors, where they no longer initiate data updates but do still respond to in-flight data updates. This drains processes from the game before it shuts down, enabling you to take a reliable snapshot before it shuts down.
  • In your snapshots, do not store Actors that you generate from other Actors. For example, if you have a wolf-spawner Actor and individual wolves, store the spawner in the snapshot and not the wolves.

For more information on how the Runtime replicates an Actor, see State consistency details.

State consistency details

When a server-worker instance replicates an Actor, it splits the operation into one or more component updates. The Runtime can take a snapshot before or after one of these component updates, but not during one.

This means that the timing of a snapshot has the following effects:

  • If the Runtime takes the snapshot at approximately the same time that it receives a component update, then the snapshot contains either all of the changes in the component update or none of them.
  • If the Runtime takes the snapshot before the last component update for an Actor replication, then the snapshot does not reflect the results of the component updates that happen after this point.

The GDK for Unreal determines the order in which server-worker instances send component updates to the SpatialOS Runtime. You cannot predict the order in which server-worker instances send these updates.

How are state changes split into components?

The GDK creates a set of SpatialOS components for the following:

  • Each Actor hierarchy
  • Each Actor Component hierarchy

For each Actor hierarchy or Actor Component hierarchy, the GDK creates up to three SpatialOS components as follows:

  • One component for properties marked with UPROPERTY(Handover)
  • One component for properties with the replication condition COND_OwnerOnly
  • One component for all other replicated fields

Let's take an example Actor to see what components the GDK creates.

UCLASS()
Class MYPROJECT_API UExampleActorComponent : public UActorComponent
{
  GENERATED_UCLASS_BODY()

public:
  UPROPERTY()
  int ComponentReplicatedProperty;

  UPROPERTY(Handover)
  int ComponentHandoverProperty;
};

UCLASS()
Class MYPROJECT_API ABaseExampleActor : public AActor
{
  GENERATED_UCLASS_BODY()

public:
  
  UPROPERTY()
  int BaseReplicatedProperty;
};

UCLASS()
class MYPROJECT_API ADerivedExampleActor : public ABaseExampleActor
{
  
  GENERATED_UCLASS_BODY()
    
public:
  //Set up lifetimes etc
  // …
  
  UPROPERTY()
  int DerivedReplicatedProperty;
  
  UPROPERTY()  // COND_OwnerOnly
  int OwnerOnlyReplicatedProperty;
  
  UPROPERTY()
  UExampleActorComponent* Component;
};

In this example, if a server-worker instance has authority over an instance of ADerivedExampleActor and changes the replicated fields, it sends the following:

  • BaseReplicatedProperty and DerivedReplicatedProperty in the same component update
  • OwnerOnlyReplicatedProperty in a dedicated component update
  • Component->ComponentReplicatedProperty in a dedicated component update
  • Component->ComponentHandoverProperty in a dedicated component update

If you take a snapshot shortly after the server-worker instance sets these properties, then your snapshot might contain only some of the properties, not necessarily all of them. You cannot predict which properties are included in the snapshot. The only exception is that the server-worker instance must write both DerivedReplicatedProperty and BaseReplicatedProperty or neither, because they are part of the same component.

Determining whether an Actor state is stored in the SpatialOS Runtime

You can determine that an Actor state has been stored when another client or server-worker instance has already observed the state. The only way to replicate changes from a server-worker instance to other client or server-worker instances is to write the changes to the SpatialOS Runtime. The Runtime then sends the changes to the relevant client or server-worker instances.

Therefore, if a worker instance observes a replicated field change on an Actor that it does not have authority over, then that field must have been written to the SpatialOS Runtime. Any snapshot taken after that point contains those changes, unless a server-worker instance has since replicated the Actor again and overwritten the old state.

Multiple Actors

When a server-worker instance replicates an Actor, it sends the components in an ordered sequence. However, when it replicates multiple Actors, it sends the ordered sequences concurrently for each Actor. Therefore, you cannot assume anything about the state of one Actor given the state of another Actor, unless writing to one Actor can only happen as a result of observing a state change to a different Actor.

In other words, suppose that a state can only be written when Actor A receives state changes from the Runtime about Actor B (for which a different server-worker must have authority). You can assume that if the snapshot contains these state changes to Actor A then the snapshot must also contain the state changes to Actor B.

Known issues

The following known issues relate to snapshots:

  • If you take a snapshot when a placed-in-level Actor has not been replicated yet, then this Actor is never replicated after you load this snapshot
  • If a server-worker instance deletes a placed-in-level Actor that does not have bNetLoadOnClient set, then when you reload this snapshot, the Runtime recreates the Actor
  • Suppose that a server-worker instance creates Actor A. No server-worker instance has received authority over Actor B yet, and Actor B holds a replicated reference to Actor A. If you take a snapshot at this point, then start a deployment from the snapshot, there is a small chance that the Runtime replaces the reference to A with a reference to a new Actor that was created after you restarted the deployment

2020-09-24 Page updated with editorial review
2020-06-22 Page created with editorial review

Updated about a year ago


Snapshots: Actor and entity synchronization considerations


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.