RTC Hybrid: Table with awareness in object cache when available.

[peterwilsoncc]

This combines to two best performing approaches following feedback from various stakeholders.

• Single table for collaboration updates
• Awareness stored in object cache only when a persistent cache is available, no attempt at DB writes is made.
• Awareness stored in table when a persistent cache is not available, writes to DB.

The awareness approach is taken from that used by transients but uses the new table as the fallback rather than storing the data in the object cache. This is to allow for the table optimized for collaboration to be used.

Forked from #11256
Trac ticket: https://core.trac.wordpress.org/ticket/64696

Props

The following WP.org accounts should be included in the props list in addition to any added by the bot below.

jorbin, czarate, desrosj, zieladam, joefusco, paulkevan, mindctrl, mukesh27, dmonad, westonruter

Use of AI Tools

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[westonruter]

Problem: $existing['type'] is not guaranteed to exist because ::get_updates_after_cursor() returns array<int, mixed>. So $existing here is mixed. It should be updated to account for that somehow.

[westonruter]

for example:

					if ( is_array( $existing ) && isset( $existing['type'] ) && self::UPDATE_TYPE_COMPACTION === $existing['type'] ) {

[alecgeatches]

@peterwilsoncc Howdy! We found a syncing issue fix which has a PR out in WordPress/gutenberg#77980 as well as a backport into the existing sync server in #11716. We can apply the same fix to this PR as well. I had trouble creating a fork based on your fork, so for now here's a diff to show the changes:

For src/wp-includes/collaboration/class-wp-http-polling-collaboration-server.php:

@@ -529,10 +529,13 @@ class WP_HTTP_Polling_Collaboration_Server {
 				}
 
 				/*
-				 * Reaching this point means there's a newer compaction,
-				 * so we can silently ignore this one.
+				 * A newer compaction already advanced the cursor, but we
+				 * can not safely drop an update. The incoming bytes still encode
+				 * operations other clients may not have seen, so store them as a
+				 * regular update. Y.applyUpdateV2 merges state-as-update blobs
+				 * idempotently, so overlap with the existing compaction is safe.
 				 */
-				return true;
+				return $this->add_update( $room, $client_id, self::UPDATE_TYPE_UPDATE, $data );
 
 			case self::UPDATE_TYPE_SYNC_STEP1:
 			case self::UPDATE_TYPE_SYNC_STEP2:

That's the main fix. Additionally, there's a test fix for the compaction change in tests/phpunit/tests/rest-api/rest-collaboration-server.php:

@@ -1094,7 +1094,7 @@ class WP_Test_REST_Collaboration_Server extends WP_Test_REST_Controller_Testcase
 	/**
 	 * @ticket 64696
 	 */
-	public function test_collaboration_stale_compaction_succeeds_when_newer_compaction_exists(): void {
+	public function test_collaboration_stale_compaction_is_stored_as_update_when_newer_compaction_exists(): void {
 		wp_set_current_user( self::$editor_id );
 
 		$room   = $this->get_post_room();
@@ -1124,9 +1124,12 @@ class WP_Test_REST_Collaboration_Server extends WP_Test_REST_Controller_Testcase
 			)
 		);
 
-		// Client 3 sends a stale compaction at cursor 0. The server should find
-		// client 2's compaction in the updates after cursor 0 and silently discard
-		// this one.
+		// Client 3 sends a stale compaction at cursor 0 (mirroring two offline
+		// clients that reconnect from the same baseline cursor). The server
+		// cannot run remove_updates_through_cursor because client 2 has already
+		// advanced the frontier, but the bytes must still be stored as a
+		// regular update so client 3's operations can propagate to other
+		// clients via Yjs state-as-update merging.
 		$stale_compaction = array(
 			'type' => 'compaction',
 			'data' => 'c3RhbGU=',
@@ -1139,16 +1142,29 @@ class WP_Test_REST_Collaboration_Server extends WP_Test_REST_Controller_Testcase
 
 		$this->assertSame( 200, $response->get_status() );
 
-		// Verify the newer compaction is preserved and the stale one was not stored.
-		$response    = $this->dispatch_collaboration(
+		// Verify the newer compaction is preserved AND the stale compaction's
+		// bytes were persisted (now as type=update so subsequent compactions
+		// don't trip the has_newer_compaction check).
+		$response = $this->dispatch_collaboration(
 			array(
 				$this->build_room( $room, '4', 0, array( 'user' => 'c4' ) ),
 			)
 		);
-		$update_data = wp_list_pluck( $response->get_data()['rooms'][0]['updates'], 'data' );
+		$updates  = $response->get_data()['rooms'][0]['updates'];
 
+		$update_data = wp_list_pluck( $updates, 'data' );
 		$this->assertContains( 'Y29tcGFjdGVk', $update_data, 'The newer compaction should be preserved.' );
-		$this->assertNotContains( 'c3RhbGU=', $update_data, 'The stale compaction should not be stored.' );
+		$this->assertContains( 'c3RhbGU=', $update_data, 'The stale compaction bytes should be stored so client 3\'s operations propagate.' );
+
+		$stale_entry = null;
+		foreach ( $updates as $entry ) {
+			if ( 'c3RhbGU=' === $entry['data'] ) {
+				$stale_entry = $entry;
+				break;
+			}
+		}
+		$this->assertNotNull( $stale_entry, 'The stale compaction entry should be present in the room.' );
+		$this->assertSame( 'update', $stale_entry['type'], 'The stale compaction should be stored as type=update, not type=compaction.' );
 	}
 
 	/*

Please add these changes to this PR if you can, and my apologies for not finding a better way to PR them to you.

[peterwilsoncc]

@alecgeatches I've committed the changes to trunk and merged them in to this branch using b7bc9f1

Are you able to verify the changes to the two files on the relevant commit?