Heartbeating is a technique used to ensure that client-workers are still connected. Unresponsive or disconnected clients are periodically removed from the game world.
Client-workers typically create a new player entity at some point after connecting to SpatialOS. Without the heartbeats, it is impossible for a server-worker to tell whether the client-worker is still connected and responsive.
By introducing heartbeats, we ensure that a player entity belonging to a disconnected or unresponsive client-worker is deleted from the world. This assures us that in a stable deployment each player entity in the world is "owned" by a client-worker that is connected or has been connected in the last N seconds, where N is the maximum number of failed heartbeats allowed multiplied by the heartbeat interval.
A server sends a heartbeat request at regular intervals to the clients. If the server continually receives failures or the request times out, the server assumes that the client has disconnected. The server then deletes the player associated with that client.
On the server-worker, the
PlayerHeartbeatInitializationSystem checks for ECS entities in the worker's view that have a
PlayerHeartbeatClient component, but do not yet have the
HeartbeatData component. For all these entities, a
HeartbeatData component is added to the ECS entity at the end of the current frame.
HeartbeatData component is used to determine which entities need to be sent
PlayerHeartbeat requests and to track how many consecutive failed heartbeats a player has.
At intervals set by
SendPlayerHeartbeatRequestSystem finds all entities fulfilling the following constraints:
- the server-worker has authority over the
- the entity has the
It sends a
PlayerHeartbeat request to each of those entities. These requests need to be handled by the client-worker that the player entity belongs to.
Note that there are numerous ways that a request may fail, as outlined here.
The client-worker runs the
HandlePlayerHeartbeatRequestSystem, which sends a response back whenever it receives a
HandlePlayerHeartbeatResponseSystem on the server-worker iterates through all
PlayerHeartbeat responses received. If at least one successful response was received from a player entity, the
NumFailedHeartbeats inside this player entity's
HeartbeatData component is set to 0.
If no responses were received with a
Success status code from a Player entity, the
NumFailedHeartbeats field is incremented. If an entity has more failed heartbeats than the number configured in
PlayerLifecycleConfig.MaxNumFailedPlayerHeartbeats, a request to delete the SpatialOS entity is sent to the SpatialOS Runtime.
Updated about a year ago