feat: quaternion synchronization, delta position, and NetworkTransform bandwidth optimizations by NoelStephensUnity · Pull Request #2388 · Unity-Technologies/com.unity.netcode.gameobjects

NoelStephensUnity · 2023-01-19T00:43:52Z

This update provides users with optional bandwidth optimizations (at the cost of precision) of up to 50% reduction in bandwidth costs per NetworkTransform instances. It also includes several changes that address issues related to rotation and parented NetworkTransforms through the enabling of quaternion synchronization.

`NetworkTransform` Additions and Changes

Quaternion Synchronization: Added the ability to synchronize by quaternions (full precision). Using quaternion synchronization is recommended for scenarios where multiple axis of rotation can change and the transform in question is within a parent-child hierarchy. When enabled, per axis rotation flags are no longer visible in the editor as the entire quaternion is synchronized when any change in rotation occurs.
Quaternion Compression: Added the ability to compress quaternions (requires Quaternion Synchronization to be enabled). Reduces the cost of quaternion synchronization down to 4 bytes at the cost of additional loss in precision (a bit less precise than half float precision).
Half Float Precision: Added the ability to enable half float precision which reduces payload size per instance by ~50% at the cost of a decrease in precision.
Delta Position Synchronization: When using half float precision NetworkTransform uses delta position synchronization to assure any loss in precision is recovered/handled in the next state update.
Added additional virtual methods to be used in prediction calculation scenarios where knowing the exact values being updated are required. This addition includes changing NetworkTransformState to public.
Added a bandwidth optimization related property, SynchronizeTransform, to NetworkObject that allows users to disable the automatic synchronization of the associated transform to reduce the total bandwidth cost of spawning and late joining client synchronization. This is particularly useful if the NetworkObject is used more for management related tasks and has no spatial synchronization needs.

Internal Companion Project: This project demonstrates the bandwidth cost/savings based on which properties are set, the average precision loss over time (editor console log required), and can be used to visual the latency between the authoritative and non-authoritative instances.

Fixes

MTT-5252
MTT-5082

Related GitHub Issues Resolved:
#2353
#2299
#1977

Changelog

Added

Added: NetworkTransform.UseHalfFloatPrecision property that, when enabled, will use half float values for position, rotation, and scale. This yields a 50% bandwidth savings a the cost of precision.
Added: NetworkTransform.UseQuaternionSynchronization property that, when enabled, will synchronize the entire quaternion.
Added: NetworkTransform.UseQuaternionCompression property that, when enabled, will use a smallest three implementation reducing a full quaternion synchronization update to the size of an unsigned integer.
Added: NetworkTransform.SlerpPosition property that, when enabled along with interpolation being enabled, will interpolate using Vector3.Slerp.
Added: BufferedLinearInterpolatorVector3 that replaces the float version, is now used by NetworkTransform, and provides the ability to enable or disable Slerp.
Added: HalfVector3 used for scale when half float precision is enabled.
Added: HalfVector4 used for rotation when half float precision and quaternion synchronization is enabled.
Added: HalfVector3DeltaPosition used for position when half float precision is enabled. This handles loss in position precision by updating only the delta position as opposed to the full position.
Added: NetworkTransform.GetSpaceRelativePosition and NetworkTransform.GetSpaceRelativeRotation helper methods to return the proper values depending upon whether local or world space.
Added: NetworkTransform.OnAuthorityPushTransformState virtual method that is invoked just prior to sending the NetworkTransformState to non-authoritative instances. This provides users with the ability to obtain more precise delta values for prediction related calculations.
Added: NetworkTransform.OnNetworkTransformStateUpdated virtual method that is invoked just after the authoritative NetworkTransformState is applied. This provides users with the ability to obtain more precise delta values for prediction related calculations.
Added: NetworkTransform.OnInitialize virtual method that is invoked after the NetworkTransform has been initialized or re-initialized when ownership changes. This provides for a way to make adjustments when NetworkTransform is initialized (i.e. resetting client prediction etc.)
Added: NetworkObject.SynchronizeTransform property (default is true) that provides users with another way to help with bandwidth optimizations where, when set to false, the NetworkObject's associated transform will not be included when spawning and/or synchronizing late joining players.

Changed

Changed: NetworkTransform authority handles delta checks on each new network tick and no longer consumes processing cycles checking for deltas for all frames in-between ticks.
Changed: The NetworkTransformState structure is now public and now has public methods that provide access to key properties of the NetworkTransformState structure.
Changed: NetworkTransform interpolation adjusts its interpolation "ticks ago" to be 2 ticks latent if it is owner authoritative and the instance is not the server or 1 tick latent if the instance is the server and/or is server authoritative.

Fixed

Fixed: issue with a child's rotation rolling over when interpolation is enabled on a NetworkTransform. Now using half precision or full quaternion synchronization will always update all axis.
Fixed: issue where NetworkTransform was not setting the teleport flag when the NetworkTransform.InLocalSpace value changed. This issue only impacted NetworkTransform when interpolation was enabled.

Testing and Documentation

Includes updates to existing integration tests.
Includes a long-duration delta position manual test to check for precision drift over time.
Includes edits to existing public API documentation.
Requires additional public documentation updates.

Work in progress.

WIP Closer but to get reasonable precision accuracy we utilize 6 bytes with only a 50% reduction in size. Deltas cannot exceed +/- 128.0f (so 30x per second rate of change would be +/- 3840 unity space units per second) The fault tolerance is +/- 0.20f

Doing a comparison of the original vs the new implementation (precision relative). Looks like 10 bit precision stored in a UINT and USHORT (6 bytes) is about the best combination between byte savings vs precision.

Converted the original delta position compressor over to use 10 bit precision. We lose 2 bytes of savings (like the adjustable precision version) but: Deltas cannot exceed +/- 128.0f (so 30x per second rate of change would be +/- 3840 unity space units per second) The decompressed value(s) fault tolerance is +/- 0.10f. (50% reduction in size is the best precision vs compression ratio we can get with this approach)

Improved the original DeltaPositionCompressor so it now has 11 bit precision for the smallest axial values. Added some additional remarks about usage along with a precision accuracy table. This version can handle +/-15,330 UWS units per second rate of change.

These changes allow for increased precision at smaller delta position values. It reaches its floor at +/- 0.001f delta (i.e. the minimum NetworkTransform position threshold)

Renamed DeltaPositionCompressor to Vector3DeltaCompressor and made all necessary adjustments. Integrated Vector3DeltaCompressor into NetworkTransform for position delta compression.

…rk-transform

This gets NetworkTransform passing all tests with delta position compression disabled. This fixes a few minor test related issues.

Adjustments for the nested network transform test. Preparing for the precision correction addition with delta position compression.

Added adjustments to handle delta position precision loss.

Added a manual test for long-duration testing of delta position compression with precision loss compensation.

some additional minor tweaks.

Forgot to check in the editor build settings.

Now the non-authoritative side (client for this manual test) shows the absolute position of the client so when you stop motion it should be either the exact value on the server side or will have a very small loss in precision (depending upon when you stop moving).

Adding a define for the delta position manual test and some cleaning up.

updating the project settings again

Adding quaternion synchronization without compression or precision loss compensation. Some axis specific NetworkTransform integration tests fail due to the quaternion synchronizing all axis. These tests will be adjusted once everything else (for quaternion compression and precision loss compensation) is in place.

This fixes an issue with the NestedNetworkTransform test. The children NetworkTransforms were still using world space synchronization which caused issues when the scale changed. When that was corrected, it then exposed a bug in the delta position implementation where the predicted client position was not taking local space into consideration. Also making sure k_QuaternionSync is not cleared after updating on the authoritative side.

Removing the sphere collider from the NetworkTransformOwnershipTests because the test basically ignores collision with other players and collision is not a factor of this test. Since the spheres are effectively "doing nothing", it didn't make sense to try and add them when they weren't being used for anything and more importantly they were causing the client side to have skewed results when OwnerAuthoritativeTests was run under the HostStartsAsOwner pass.

Temporarily adding updates to various NetworkTransform tests to account for current changes. (primarily to run stand alone testrunner builds on consoles to verify current changes still work on them)

Added HalfVector3 After doing some tests, it looks like using HalfVector3 yields the exact same size in bytes and actually has better over-all precision.

All NetworkTransform integration tests pass using HalfVector3. Still WIP, but looking like this could be the best path.

Final tweaks to NetworkTransform using HalfVector3 and adjustments to the DeltaPositionNetworkTransform manual test to be able to compare both.

Several adjustments to tests that take the HalfVector3 into consideration as well as some debug hooks for testing purposes.

Added the quaternion compressor and the HalfVector4 (quat) implementation. Updated some tests and NetworkTransform.

This update: - Increases the precision of HalfVector4 when using quaternions - Adds UseHalfFloatPrecision property that when enabled will synchronize in half float values as well as synchronize the change in status on the non-authoritative sides. - Includes some debug mode adjustments (TBD if we will keep all of this)

Updating tests for previous PR's changes.

Slowly backing out some legacy changes and debug info from NetworkTransform Updating some of the NetworkTransform tests to more recent changes.

This applies delta position updates to HalfVector3 and includes adjustments to NetworkTransform to account for this. This also increases half float quaternion precision.

…ransform

Reverting out the changes to UnityTransport as this was fixed in PR-2433.

…ransform

Updating the vector implementations to reduce memory allocation. Renaming some structs and methods for better clarity. Updated XML API for clarity and formatting.

…ransform

Updating some wording in the XML API.

adding suggested comments about leaving remarks regarding the contents of the ushort values are used to store the 16bits of the half float values.

wackoisgod · 2023-03-13T19:36:30Z

I am curious why these new types were added when Unity already has these types in the Unity Mathematics package? Like many things within NGO it seems that you just end up creating your own version of a thing? Why not just make the types in a package that is used all over the place support serialization? and furthermore are compatible with Engine Types when required?

NoelStephensUnity · 2023-03-15T01:12:08Z

@wackoisgod
Which types are you referring to?
The half, half3, and half4 types?

wackoisgod · 2023-03-15T01:36:40Z

That would be correct; I just think introducing another new type seems kind of wasteful and confusing to people because now they can use the Unity.Mathmatics package which converts to Unity Engine types where required but then now they may have to convert to these NGO types when they want to use them on the network. I just don't see a lot of value in creating another type.

I think VectorT is not a great type either because while yes it can be limited to an unmanaged type its really only value is primitive number types but nothing stops someone from putting any sort of struct in the type.

NoelStephensUnity · 2023-03-15T03:45:39Z

@wackoisgod
I see your point. We still have the release/1.3.0 update to push out before any of the changes that occurred after the v1.3.0 branch...so there is plenty of time to look into any fine tuning/re-factoring. All input is 100% welcome (especially yours).
The Unity.Mathmatics package would need to be added to the assemblies referenced in the components, runtime tests, and other NGO asmdef files (currently it is not referenced) which would make NGO dependent upon that assembly. I will most definitely pass it by @ThusSpokeNomad to get his take on adding that as a dependency.

Please always feel free to bring up anything you see that needs a second pass "re-think" as I highly value your insights into things of this very nature and the more eyes on things the better (imo). 👍

0xFA11 · 2023-03-15T10:36:22Z

We already indirectly depend on Unity.Mathematics package, so it'd be just fine :)

Would this depgraph be useful? @NoelStephensUnity

NoelStephensUnity · 2023-03-15T18:51:12Z

@ThusSpokeNomad Very cool graph...
@wackoisgod
#2456 implements the refactoring with Unity.Mathematics types.
Most likely I will need to update the FastBufferReader/Write with all types included in Unity.Mathematics, so the PR is still in draft form (just doing a pass on desktop platforms to make sure nothing else is impacted).

NoelStephensUnity added 30 commits December 2, 2022 14:55

update

5571be6

Work in progress.

update

a7cde50

WIP Closer but to get reasonable precision accuracy we utilize 6 bytes with only a 50% reduction in size. Deltas cannot exceed +/- 128.0f (so 30x per second rate of change would be +/- 3840 unity space units per second) The fault tolerance is +/- 0.20f

update

343e52b

Doing a comparison of the original vs the new implementation (precision relative). Looks like 10 bit precision stored in a UINT and USHORT (6 bytes) is about the best combination between byte savings vs precision.

update

44b03cb

Improved the original DeltaPositionCompressor so it now has 11 bit precision for the smallest axial values. Added some additional remarks about usage along with a precision accuracy table. This version can handle +/-15,330 UWS units per second rate of change.

update

6be04c5

These changes allow for increased precision at smaller delta position values. It reaches its floor at +/- 0.001f delta (i.e. the minimum NetworkTransform position threshold)

update

64855d6

Renamed DeltaPositionCompressor to Vector3DeltaCompressor and made all necessary adjustments. Integrated Vector3DeltaCompressor into NetworkTransform for position delta compression.

Merge branch 'develop' into refactor/quaternion-synchronization-netwo…

0168cc9

…rk-transform

update

a093a06

This gets NetworkTransform passing all tests with delta position compression disabled. This fixes a few minor test related issues.

update

0a03b9f

Adjustments for the nested network transform test. Preparing for the precision correction addition with delta position compression.

update

877acc1

Added adjustments to handle delta position precision loss.

test manual

73c73db

Added a manual test for long-duration testing of delta position compression with precision loss compensation.

update

e3dc079

some additional minor tweaks.

update

7183d38

Forgot to check in the editor build settings.

update

ce145ce

Now the non-authoritative side (client for this manual test) shows the absolute position of the client so when you stop motion it should be either the exact value on the server side or will have a very small loss in precision (depending upon when you stop moving).

update

220e30a

Adding a define for the delta position manual test and some cleaning up.

update

09359bd

updating the project settings again

test updates

2b254f1

Temporarily adding updates to various NetworkTransform tests to account for current changes. (primarily to run stand alone testrunner builds on consoles to verify current changes still work on them)

update

fe68e84

Added HalfVector3 After doing some tests, it looks like using HalfVector3 yields the exact same size in bytes and actually has better over-all precision.

update

d232e7e

All NetworkTransform integration tests pass using HalfVector3. Still WIP, but looking like this could be the best path.

update

419010d

Final tweaks to NetworkTransform using HalfVector3 and adjustments to the DeltaPositionNetworkTransform manual test to be able to compare both.

tests update

b71f7d2

Several adjustments to tests that take the HalfVector3 into consideration as well as some debug hooks for testing purposes.

update

466bbaa

Added the quaternion compressor and the HalfVector4 (quat) implementation. Updated some tests and NetworkTransform.

Test

9ec0e4c

Updating tests for previous PR's changes.

update and test

fd3dcd1

Slowly backing out some legacy changes and debug info from NetworkTransform Updating some of the NetworkTransform tests to more recent changes.

update

a96d12b

This applies delta position updates to HalfVector3 and includes adjustments to NetworkTransform to account for this. This also increases half float quaternion precision.

NoelStephensUnity requested review from ShadauxCat and samlucas-unity March 3, 2023 22:30

NoelStephensUnity added 2 commits March 6, 2023 09:46

Merge branch 'develop' into feat/quaternion-synchronization-network-t…

2b7822b

…ransform

revert

3a906ae

Reverting out the changes to UnityTransport as this was fixed in PR-2433.