Skip to content

refactor: scene object serialization consolidation and improved handling#756

Merged
NoelStephensUnity merged 37 commits intodevelopfrom
refactor/mtt640-sceneobjects
Apr 28, 2021
Merged

refactor: scene object serialization consolidation and improved handling#756
NoelStephensUnity merged 37 commits intodevelopfrom
refactor/mtt640-sceneobjects

Conversation

@NoelStephensUnity
Copy link
Copy Markdown
Member

@NoelStephensUnity NoelStephensUnity commented Apr 21, 2021

This consolidates the NetworkObject serialization process for unified serialization and deserialization scene objects during both the approval and scene transitioning process.
Primary files to review over:
NetworkObject.cs
InternalMessageHandler.cs
NetworkSceneManager.cs
NetworkManager.cs --> HandleApproval method

Acceptance Test:
NetworkObjectSceneSerializationTests
This tests the primary functionality aspect of this PR which is that an invalid NetworkObject in a stream being de-serialized during approval or scene transitioning will not impact all NetworkObjects being de-serialized but just skip the invalid NetworkObject (including its NetworkVariable data) and continue processing the rest of the remaining NetworkObjects.

This is the first pass changes required to implement MTT-640.
The default player prefab is now a directly assigned value as opposed to a checkbox in the network prefab list.
The default player prefab has to also be included in the NetworkPrefab list (this is so the default player prefab can be overridden too)
NetworkPrefab entries can be overridden and directly assigned alternate prefabs for clients to spawn.
The source prefab (what the server uses) can be a prefab reference or a prefab hash.
This update is the first pass of making this update as painless as possible to the user by "auto-handling" the v0.1.0 NetworkPrefab pattern as well as there is additional "user-bumper" code that assures the assigned default player prefab is always in the NetworkPrefab list.
Removing the auto insert within the editor and doing this at runtime.
Added additional check to handle assigning the OverridingSourcePrefab if it is null and the Prefab is not.
This will prevent the scenario where a user might want to delete the default player prefab but also has it assigned within the NetworkPrefab list.

This assures that a newly created network prefab list entry is blank and that there are not two entries within the list that have the same global object id hash.
just adding a few comments
Adjusted names of the new NetworkPrefab properties and added additional comments to better clarify some of the "trickery" happening that will help make this a "non-breaking" feature when it comes to existing v0.1.0 NetworkConfigs and the already selected default player prefab.
This will prevent any future client side soft synchronization issues or related issues where only one (or a few) NetworkObjects are misconfigured and the client fails to synchronize (find or instantiate).  As opposed to completely breaking the rest of the pending NetworkObjects to be synchronized/instantiated, it will log an error and continue loading.

This also has some additional comments and updates to comments for better clarity.
minor change in comments
This removes the CreatePlayerPrefab property from the NetworkConfig and from the NetworkManagerEditor.  It is no longer needed as the NetworkConfig.PlayerPrefab property being something other than null tells us that the user wants to have MLAPI automatically spawn the players for them.
removed unused namespace references.
Added a tooltip for the new NetworkConfig.PlayerPrefab property
Added some spaces between the forward slashes and comment's text body
This fixes an issue that can occur with serialization and NetworkPrefabOverrideLinks.
The policy is to ALWAYS clear this dictionary before building the links dictionary.
This removes the NetworkConfig compatibility checks that auto-upgraded a previous NetworkConfig setting to the new pattern.
Updated NetworkPrefab according to this change, isPlayer is no longer needed.
Added additional code to NetworkPrefab that will return a hash reflective of the override state.
Adjusted some comments.
This refactors how scene object synchronization is handled.  Primarily this puts the object synchronization found within the approval and scene transition process into one location which should help to assure that both cases are handled in the same way.

This also includes additional header information for network variable data written into the outbound stream as well as reading the added information on the client side from the inbound stream.
NoelStephensUnity added a commit that referenced this pull request Apr 22, 2021
Removing two checks for the failure to spawn a NetworkObject locally.
This will be handled via PR-756:
#756
m_EnableTimeResyncProperty = m_NetworkConfigProperty.FindPropertyRelative("EnableTimeResync");
m_TimeResyncIntervalProperty = m_NetworkConfigProperty.FindPropertyRelative("TimeResyncInterval");
m_EnableNetworkVariableProperty = m_NetworkConfigProperty.FindPropertyRelative("EnableNetworkVariable");
m_EnsureNetworkVariableLengthSafetyProperty = m_NetworkConfigProperty.FindPropertyRelative("EnsureNetworkVariableLengthSafety");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, trailing space

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer part of the PR (it was an artifact from an earlier merge of #749.

var networkOverrideProp = networkPrefab.FindPropertyRelative(nameof(NetworkPrefab.Override));
var networkOverrideInt = networkOverrideProp.enumValueIndex;

return 10 + (networkOverrideInt == 0 ? EditorGUIUtility.singleLineHeight : (EditorGUIUtility.singleLineHeight * 2) + 5);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always seems odd to me how in Unity editor code these raw offset numbers show up. I wonder, are the various uses of '5' the same '5'? As in could those be constants? And is the '10' this constant * 2?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer part of the PR (it was an artifact from an earlier merge of #749.

0xFA11 added a commit that referenced this pull request Apr 23, 2021
…er Prefab (#749)

* NetworkPrefab registration and default PlayerPrefab

This is the first pass changes required to implement MTT-640.
The default player prefab is now a directly assigned value as opposed to a checkbox in the network prefab list.
The default player prefab has to also be included in the NetworkPrefab list (this is so the default player prefab can be overridden too)
NetworkPrefab entries can be overridden and directly assigned alternate prefabs for clients to spawn.
The source prefab (what the server uses) can be a prefab reference or a prefab hash.

* refactor: Naming, Migration, and PlayerPrefab

This update is the first pass of making this update as painless as possible to the user by "auto-handling" the v0.1.0 NetworkPrefab pattern as well as there is additional "user-bumper" code that assures the assigned default player prefab is always in the NetworkPrefab list.

* refactor: PlayerPrefab assignment

Removing the auto insert within the editor and doing this at runtime.
Added additional check to handle assigning the OverridingSourcePrefab if it is null and the Prefab is not.

* refactor: player prefab selection and blank new entries

This will prevent the scenario where a user might want to delete the default player prefab but also has it assigned within the NetworkPrefab list.

This assures that a newly created network prefab list entry is blank and that there are not two entries within the list that have the same global object id hash.

* style: comments

just adding a few comments

* style: property names and comments

Adjusted names of the new NetworkPrefab properties and added additional comments to better clarify some of the "trickery" happening that will help make this a "non-breaking" feature when it comes to existing v0.1.0 NetworkConfigs and the already selected default player prefab.

* refactor: Client Sync and Comments

This will prevent any future client side soft synchronization issues or related issues where only one (or a few) NetworkObjects are misconfigured and the client fails to synchronize (find or instantiate).  As opposed to completely breaking the rest of the pending NetworkObjects to be synchronized/instantiated, it will log an error and continue loading.

This also has some additional comments and updates to comments for better clarity.

* style: minor change

minor change in comments

* refactor: remove CreatePlayerPrefab

This removes the CreatePlayerPrefab property from the NetworkConfig and from the NetworkManagerEditor.  It is no longer needed as the NetworkConfig.PlayerPrefab property being something other than null tells us that the user wants to have MLAPI automatically spawn the players for them.

* refactor: cleaning up using

removed unused namespace references.

* style: PlayerPrefab tooltip

Added a tooltip for the new NetworkConfig.PlayerPrefab property

* style: spaces between slashes and comment text body

Added some spaces between the forward slashes and comment's text body

* fix: clearing network prefab links

This fixes an issue that can occur with serialization and NetworkPrefabOverrideLinks.
The policy is to ALWAYS clear this dictionary before building the links dictionary.

* refactor: NetworkConfig compatibility, NetworkPrefab, and Style

This removes the NetworkConfig compatibility checks that auto-upgraded a previous NetworkConfig setting to the new pattern.
Updated NetworkPrefab according to this change, isPlayer is no longer needed.
Added additional code to NetworkPrefab that will return a hash reflective of the override state.
Adjusted some comments.

* fix: wrong hash reference

Minor adjustment for the right GlobalObjectIdHash check.

* refactor and style

Made the NetworkPrefab internal as well as NetworkPrefabOverride.
Removed GetNetworkPrefabIndexOfHash and GetPrefabHashFromIndex as they were used in the old hash generation pattern.
Updated comments a bit.

* refactor: removing NetworkObject check

Removing two checks for the failure to spawn a NetworkObject locally.
This will be handled via PR-756:
#756

* refactor: removing NetworkPrefab.Hash

This property is no longer being used and where it was used was replaced by the NetworkConfig.NetworkPrefabOverrideLinks

Co-authored-by: M. Fatih MAR <mfatihmar@gmail.com>
Removing some code that got duplicated during a merge.
@NoelStephensUnity NoelStephensUnity marked this pull request as ready for review April 26, 2021 17:30
@NoelStephensUnity NoelStephensUnity marked this pull request as draft April 26, 2021 17:36
Comment thread com.unity.multiplayer.mlapi/Runtime/Core/NetworkObject.cs Outdated
Comment on lines +671 to +672
//Write how much data was written
writer.WriteUInt32((uint)(buffer.Position - currentStreamPosition));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//Write how much data was written
writer.WriteUInt32((uint)(buffer.Position - currentStreamPosition));
// Replace the placeholder 0 with how much data `WriteNetworkVariableData` wrote into the stream
writer.WriteUInt32((uint)(buffer.Position - currentStreamPosition));

/// <param name="objectStream">inbound stream</param>
/// <param name="reader">reader for the stream</param>
/// <param name="networkManager">NetworkManager instance</param>
static internal void DeserializeSceneObject(Serialization.NetworkBuffer objectStream, PooledNetworkReader reader, NetworkManager networkManager)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, the suggested order is internal static void instead of static internal void (minor)

if (networkObject == null)
{
// This will prevent one misconfigured issue (or more) from breaking the entire loading process.
Debug.LogError($"Failed to spawn NetowrkObject for Hash {prefabHash}.");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Debug.LogError($"Failed to spawn NetowrkObject for Hash {prefabHash}.");
Debug.LogError($"Failed to spawn {nameof(NetworkObject)} for Hash {prefabHash}.");

Fixed method declaration standard issue.
Improved comments for network variable data size placeholder.
Using only NetworkWriter and NetworkReader as parameter values.
Copy link
Copy Markdown
Contributor

@0xFA11 0xFA11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very cool, ship it!


if (parentNetworkObject == null)
{
// We don't have a parent
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the wonderful comments

Comment thread com.unity.multiplayer.mlapi/Runtime/Core/NetworkObject.cs Outdated
Comment thread com.unity.multiplayer.mlapi/Runtime/Core/NetworkManager.cs
//As long as something was written
if (buffer.Position > currentStreamPosition)
{
//Get the new updated stream buffer position
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code wasn't here before, seems a bit in an addition to the refactor. Is this just an optimization to not send data if there's nothing to send? Could there be a comment on what this overall is doing?

Copy link
Copy Markdown
Member Author

@NoelStephensUnity NoelStephensUnity Apr 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire block of code associated with this:

 //Write whether we are including network variable data
  writer.WriteBool(NetworkManager.NetworkConfig.EnableNetworkVariable);
  
if (NetworkManager.NetworkConfig.EnableNetworkVariable)
{
    var buffer = writer.GetStream() as NetworkBuffer;
    // Mark our current position and set the network variable data size initially to zero (NOT as a packed value)
    var currentStreamPosition = buffer.Position;

    // Write variable data size placeholder initially as zero
    writer.WriteUInt32(0);

    //Write the network variable data
    WriteNetworkVariableData(buffer, targetClientId);

    //As long as something was written
    if (buffer.Position > currentStreamPosition)
    {
        //Get the new updated stream buffer position
        var updatedPosition = buffer.Position;

        //Move back to the position just before we wrote our size
        buffer.Position = currentStreamPosition;

        // Replace the zero value placeholder with the size of data written into the stream by WriteNetworkVariableData
        writer.WriteUInt32((uint)(buffer.Position - currentStreamPosition));

        //Revert the buffer position back to the end of the network variable data written
        buffer.Position = updatedPosition;
    }
}

Basically, if EnableNetworkVariable is true then we will be including the size of the NetworkVariable data written into the stream for the NetworkObject in question. We basically write a place holder (unpacked) UINT32 value, mark the position of the stream, then invoke WriteNetworkVariableData, and if the stream position changed we want to write the delta between the position prior to calling WriteNetworkVariableData and the current stream position. This delta we will then write into the place holder position (i.e. before the WriteNetworkVariableData was invoked) so upon de-serializing this entry we will know the exact size of the Network Variable data in the event the NetworkObject in question fails to be instantiated on the de-serialization end of things (typically the client).

This is the "improved handling" of scene object serialization such that if one NetworkObject fails to be instantiated we can simply "log a warning" and skip to the next NetworkObject in the scene.

I think I provide enough comments for this, but if you have any additional suggestions on how to better improve the comments I am certainly open to suggestions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so it sounds like as part of the improved scene object handling, now when we provide the serialization for the object we also include the state of the its associated network variables. I assume this is to avoid timing problems such as we talked about previously?

If so, great; I'd put in a comment to the effect of "force synchronize the state of this object's variables immediately to avoid consistency errors, etc."

Lastly, should make sure @jeffreyrainy sees this for context when we go to snapshot sync

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only assists in the prevents completely asserting out of the current call-stack scope during scene synchronization. This would happen during the InternalMessageHandler.HandleConnectionApproved and during the NetworkSceneManager.OnSceneUnloadClient. The issue was that we couldn't just "skip an entry" in the serialized stream because we didn't know the size of the NetworkVariable data that was written into the stream for the NetworkObject in question that failed to instantiate on the client side.


var networkObject = networkManager.SpawnManager.CreateLocalNetworkObject(isSceneObject, prefabHash, ownerClientId, parentNetworkId, position, rotation);

//Determine if we have network variable data to read
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this section seems new. That's cool, I just don't know what's going on to review it.

Copy link
Copy Markdown
Member Author

@NoelStephensUnity NoelStephensUnity Apr 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sure that if one NetworkObject fails to construct (for whatever reason) then we can "skip past" that specific NetworkObject but continue processing any remaining serialized NetworkObjects as opposed to just throwing an exception and skipping the remaining (if any) NetworkObjects which was the original behavior.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess what I'm going to ask you to do :)

let's add the paragaph you just typed for me into the code

Copy link
Copy Markdown
Contributor

@mattwalsh-unity mattwalsh-unity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had some questions, nice cleanup and comments tho

Suggested update by mattwalsh-untity.
This just passes in the already obtained buffer as opposed to calling a method that returns the same buffer.
This is a cleaner implementation adjustment.

Co-authored-by: Matt Walsh <69258106+mattwalsh-unity@users.noreply.github.com>
Refactored some of the names used to hopefully better clarify what is happening in this process.  Updated the logic and flow slightly to assure stream position alignment is accurate and for better clarity of what is happening.
This includes the test to validate that the primary purpose behind this PR, to skip past an invalid NetworkObject during scene object synchronization, is functioning as expected.
This is an improved NetworkObjectSceneSerializationFailure unit test.
It adds random NetworkVariable data sizes to assure all stream offsets are being calculated properly.  It randomly adds invalid NetworkObjects.
Minor style related changes. Adjusted comment and removed LF.
comment update for invalidNetworkObjectFrequency
Fixed some spelling issues.
@NoelStephensUnity NoelStephensUnity merged commit ea30795 into develop Apr 28, 2021
@NoelStephensUnity NoelStephensUnity deleted the refactor/mtt640-sceneobjects branch April 28, 2021 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants