ddd
This commit is contained in:
parent
0a343ecbc5
commit
c3a6f5ff37
80 changed files with 2512 additions and 1625 deletions
Binary file not shown.
|
|
@ -1,676 +0,0 @@
|
|||
# Bend Field: Efficient Field Save and Load Design
|
||||
|
||||
## Purpose
|
||||
|
||||
This document proposes a scalable persistence system for a complete Bend Field
|
||||
world. It covers two related but separate problems:
|
||||
|
||||
1. Loading a large local field quickly enough that the player can begin using it
|
||||
before every puzzle has been read.
|
||||
2. Exporting and restoring the complete field without creating several
|
||||
whole-field copies in memory.
|
||||
|
||||
The recommended design preserves the existing incremental autosave behavior,
|
||||
introduces an index-first local repository, and adds a streamed portable archive.
|
||||
|
||||
## Current system assessment
|
||||
|
||||
IndexedDB is already the durable local source of truth. Normal gameplay saves are
|
||||
incremental: only dirty board metadata, dirty board states, global data,
|
||||
tombstones, recovery information, and cloud-outbox changes are committed. This
|
||||
part should be retained.
|
||||
|
||||
The main scaling problems are elsewhere:
|
||||
|
||||
- Startup calls `getAll()` for all metadata and state rows, constructs a complete
|
||||
snapshot, and normalizes every puzzle before gameplay begins.
|
||||
- `hydrateMeta()` does not currently load anything from storage. It only checks
|
||||
that an already loaded puzzle exists.
|
||||
- The compact localStorage mirror is useful for small-field recovery but is
|
||||
intentionally limited to about 4.5 MiB and cannot represent a very large
|
||||
field.
|
||||
- JSON export constructs a complete snapshot, pretty-printed JSON string, and
|
||||
Blob in memory.
|
||||
- The current export payload can contain cloud credentials, cloud transport
|
||||
state, session identity, and recovery data. These are local persistence
|
||||
details and must not be portable.
|
||||
- JSON import reads and parses the complete file in memory, then clears and
|
||||
repopulates the active stores.
|
||||
- The current metadata and state stores are keyed only by board ID. A second
|
||||
complete field cannot be staged beside the active field, so an atomic pointer
|
||||
switch is not possible with the present keys.
|
||||
- The original cloud implementation used a complete snapshot for pulls and one
|
||||
whole-world JSON file on the server.
|
||||
|
||||
The local repository and portable backup were implemented first. The follow-up
|
||||
cloud implementation now batches pushes, pages pulls, returns recent revision
|
||||
deltas, and stores board revisions in separate crash-safe files.
|
||||
|
||||
## Design goals
|
||||
|
||||
- Preserve the existing low-cost dirty-row autosave path.
|
||||
- Make the origin board or current viewport usable before a full field scan
|
||||
completes.
|
||||
- Avoid loading every puzzle definition and path collection during startup.
|
||||
- Keep archive memory use independent of total field size.
|
||||
- Make restore strict, cancelable, and all-or-nothing.
|
||||
- Never place cloud credentials, recovery journals, or session identity in a
|
||||
portable backup.
|
||||
- Make reset, import, migration, and rollback use the same safe epoch-switching
|
||||
primitive.
|
||||
- Keep old JSON saves importable through explicit compatibility rules.
|
||||
- Preserve multi-tab protection and reject stale writers after a field switch.
|
||||
|
||||
## Version boundaries
|
||||
|
||||
The following versions serve different purposes and must not share a counter:
|
||||
|
||||
- **Archive version** describes the portable `.bfsave` wire format.
|
||||
- **Save schema** describes gameplay data and its migrations.
|
||||
- **IndexedDB layout version** describes object stores and indexes.
|
||||
- **World generation** identifies compatible generated-world rules.
|
||||
- **Application version** identifies the build that produced an archive.
|
||||
|
||||
The proposed portable archive is version 2. The proposed IndexedDB layout is
|
||||
version 6. Upgrading either one must not implicitly change the other.
|
||||
|
||||
The existing physical database name should be reused for the version-6
|
||||
migration. Future database names should not be derived from the save-schema
|
||||
number.
|
||||
|
||||
## Local repository design
|
||||
|
||||
### Object stores
|
||||
|
||||
The version-6 upgrade creates the following stores while retaining the current
|
||||
version-5 stores for migration:
|
||||
|
||||
- `control`, keyed by a singleton key.
|
||||
- `worlds`, keyed by `epoch`.
|
||||
- `boardIndex`, `boardPuzzles`, `boardStates`, and `tombstonesV2`, keyed by
|
||||
`[epoch, id]`.
|
||||
- `recoveryV2` and `outboxV2`, keyed by `[epoch, key]`.
|
||||
|
||||
Every epoch-scoped store has an `epoch` index so an abandoned field can be
|
||||
deleted in bounded batches.
|
||||
|
||||
Conceptual records:
|
||||
|
||||
```ts
|
||||
interface WorldControl {
|
||||
key: "active";
|
||||
activeFormat: 1 | 2;
|
||||
activeEpoch: string;
|
||||
previousEpoch?: string;
|
||||
activationId: string;
|
||||
activationVerified: boolean;
|
||||
switchedAt: number;
|
||||
}
|
||||
|
||||
interface WorldRecord {
|
||||
epoch: string;
|
||||
status: "staging" | "ready" | "active" | "rollback" | "garbage";
|
||||
schema: number;
|
||||
worldGeneration: string;
|
||||
global: PortableAndLocalGlobalState;
|
||||
boardCount: number;
|
||||
solvedCount: number;
|
||||
score: number;
|
||||
bounds: { minX: number; minY: number; maxX: number; maxY: number };
|
||||
approximateBytes: number;
|
||||
createdAt: number;
|
||||
activatedAt?: number;
|
||||
source?: {
|
||||
kind: "migration" | "import" | "reset";
|
||||
archiveCrc32?: string;
|
||||
};
|
||||
progress?: {
|
||||
phase: string;
|
||||
lastKey?: IDBValidKey;
|
||||
rows: number;
|
||||
};
|
||||
}
|
||||
|
||||
interface BoardIndexRecord {
|
||||
epoch: string;
|
||||
id: string;
|
||||
x: number;
|
||||
y: number;
|
||||
chunks: [number, number][];
|
||||
level: number;
|
||||
targetLevel: number;
|
||||
seed: number;
|
||||
axis: string;
|
||||
entrySide: "N" | "S" | "W" | "E" | null;
|
||||
metaRev: number;
|
||||
stateRev: number;
|
||||
solved: boolean;
|
||||
expanded: boolean;
|
||||
scoreAwarded: number;
|
||||
hasProgress: boolean;
|
||||
specialFlags: {
|
||||
crossing: boolean;
|
||||
warp: boolean;
|
||||
lock: boolean;
|
||||
};
|
||||
shop: null | {
|
||||
cell: [number, number] | null;
|
||||
itemIds: string[];
|
||||
purchases: CompactPurchaseSummary[];
|
||||
};
|
||||
}
|
||||
|
||||
interface BoardPuzzleRecord {
|
||||
epoch: string;
|
||||
id: string;
|
||||
metaRev: number;
|
||||
revAuthor: string;
|
||||
generatorVersion: number;
|
||||
puzzle: StoredPuzzle;
|
||||
}
|
||||
|
||||
interface BoardStateRecord {
|
||||
epoch: string;
|
||||
id: string;
|
||||
stateRev: number;
|
||||
revAuthor: string;
|
||||
value: StoredBoardState;
|
||||
}
|
||||
```
|
||||
|
||||
`BoardIndexRecord` is the source for occupancy, distant rendering, solved
|
||||
totals, shop icons, and inventory summaries. `summarizeBoard(meta, state)` is
|
||||
the only function that creates it. Any transaction that changes a field used by
|
||||
the summary must update the index in the same transaction.
|
||||
|
||||
Puzzle records remain separate from state records so drawing a path does not
|
||||
rewrite the static puzzle definition.
|
||||
|
||||
### Startup and hydration
|
||||
|
||||
Startup proceeds in this order:
|
||||
|
||||
1. Open IndexedDB and read `WorldControl`, the active `WorldRecord`, recovery
|
||||
coverage, and the cloud outbox header.
|
||||
2. Read B0 and the saved camera/selected-board area first. If no camera anchor is
|
||||
available, use B0.
|
||||
3. Build initial occupancy and presentation from the first index page.
|
||||
4. Mark the application ready once B0 and the initial viewport details are
|
||||
hydrated.
|
||||
5. Continue scanning `boardIndex` in 512-record pages and progressively add
|
||||
occupancy and overview summaries.
|
||||
|
||||
Puzzle and state stores must never use an unbounded `getAll()` during startup.
|
||||
Page scans resume from the final compound key of the previous page.
|
||||
|
||||
The current hydration contract is replaced by:
|
||||
|
||||
```ts
|
||||
async function hydrateBoardDetails(
|
||||
epoch: string,
|
||||
id: string,
|
||||
expected?: { metaRev: number; stateRev: number }
|
||||
): Promise<HydratedBoard>
|
||||
```
|
||||
|
||||
It reads the index, puzzle, and state in one readonly transaction. Missing rows
|
||||
or revision mismatches are rejected rather than combined.
|
||||
|
||||
Details are hydrated for:
|
||||
|
||||
- Boards entering the interactive viewport and its prefetch margin.
|
||||
- The selected or keyboard-focused board.
|
||||
- Boards on both sides of an active gate connection.
|
||||
- Boards involved in pending expansion or repair work.
|
||||
- A shop before it opens or an inventory item before it is consumed.
|
||||
|
||||
The detail cache is an LRU limited to 128 boards or 32 MiB, whichever is reached
|
||||
first. Active pointer boards, dirty boards, time-attack boards, and boards still
|
||||
referenced by a renderer or editor are pinned. A dirty board must be committed
|
||||
before it can be evicted.
|
||||
|
||||
Full-field score, inventory, minimap, shop-marker, and distant-overview loops
|
||||
must use index summaries. They must not hydrate every board as a side effect.
|
||||
|
||||
### Commit and concurrency rules
|
||||
|
||||
All canonical field writes use one repository entry point:
|
||||
|
||||
```ts
|
||||
async function commitFieldDelta(delta: FieldDelta): Promise<CommitResult>
|
||||
```
|
||||
|
||||
The transaction:
|
||||
|
||||
1. Reads `WorldControl`.
|
||||
2. Verifies both the expected active epoch and active format.
|
||||
3. Applies puzzle, state, summary, global, recovery, tombstone, and outbox
|
||||
changes.
|
||||
4. Commits the updated revisions together.
|
||||
|
||||
An epoch or format mismatch aborts with `STALE_WORLD_EPOCH`.
|
||||
|
||||
The existing Web Lock and storage-lease fallback remain the cross-tab mutation
|
||||
guard. BroadcastChannel and storage events are only wake-up signals; receivers
|
||||
must re-read authoritative control and revision records from IndexedDB.
|
||||
|
||||
Compression, file writes, fetches, timers, worker replies, and rendering yields
|
||||
must not be awaited inside an IndexedDB transaction. Browsers may auto-commit a
|
||||
transaction when it has no pending IndexedDB request. Data should be parsed,
|
||||
normalized, summarized, and encoded before its bounded transaction is opened.
|
||||
|
||||
### Recovery
|
||||
|
||||
IndexedDB remains authoritative. LocalStorage retains only:
|
||||
|
||||
- The active-epoch hint.
|
||||
- A bounded dirty-row recovery journal.
|
||||
- Cross-tab wake-up data.
|
||||
|
||||
It no longer attempts to mirror the entire version-2 field. Recovery records
|
||||
remain epoch-scoped and cannot be applied after an import, reset, or rollback
|
||||
switches to another epoch.
|
||||
|
||||
## Portable archive
|
||||
|
||||
### Format
|
||||
|
||||
The default filename is:
|
||||
|
||||
```text
|
||||
bend-field-save-YYYY-MM-DD.bfsave
|
||||
```
|
||||
|
||||
The payload is UTF-8 NDJSON. It is wrapped in one gzip stream when
|
||||
`CompressionStream("gzip")` is available and remains uncompressed otherwise.
|
||||
The importer detects the gzip magic bytes and verifies that the decoded
|
||||
manifest declares the same compression mode.
|
||||
|
||||
Decoded records have this exact order:
|
||||
|
||||
```json
|
||||
{"type":"manifest","format":"bend-field-save","archiveVersion":2,"saveSchema":31,"gameplayVersion":3,"worldGeneration":"...","appVersion":"...","generatorVersion":1,"exportedAt":"...","boardCount":42,"estimatedRawBytes":123456,"encoding":"ndjson","compression":"gzip"}
|
||||
{"type":"global","value":{}}
|
||||
{"type":"board","id":"B0","meta":{},"state":{}}
|
||||
{"type":"board","id":"B1","meta":{},"state":{}}
|
||||
{"type":"end","boardCount":42,"rawBytes":123456,"crc32":"12ab34cd"}
|
||||
```
|
||||
|
||||
Board records are sorted by numeric board ID. Exactly one global record and one
|
||||
footer are required. No record is pretty-printed.
|
||||
|
||||
The footer CRC32 covers every decoded UTF-8 byte before the footer, including
|
||||
record newlines. Gzip validation, CRC32, deterministic ordering, byte counts,
|
||||
and board counts detect accidental corruption and truncation. They do not
|
||||
authenticate the archive's author.
|
||||
|
||||
Version 2 intentionally has no random-access index, encryption, signature, or
|
||||
per-block hash. Import consumes every record in order, so a custom container
|
||||
would add complexity without improving the current use case.
|
||||
|
||||
### Portable allowlist
|
||||
|
||||
Portable board metadata includes geometry, levels, seed, puzzle definition,
|
||||
generator version, and connection-related data. Portable board state includes
|
||||
paths, special progress, solved/expanded status, rewards, and shop purchases.
|
||||
|
||||
Portable global state includes gameplay version, bonus events,
|
||||
encountered mechanics, last-solve information, time-attack gameplay state, and
|
||||
the trusted clock floor.
|
||||
|
||||
The importer recomputes `nextId`, solved count, total score, bonus score, field
|
||||
bounds, board summaries, and storage-size estimates.
|
||||
|
||||
The following data is always excluded:
|
||||
|
||||
- `cloudProfile`, player ID, token, cloud revision, cloud pending data, and
|
||||
outbox rows.
|
||||
- Recovery envelopes, journals, coverage markers, and tombstones.
|
||||
- `worldEpoch`, `sessionId`, revisions' author identities, and transport
|
||||
revision state.
|
||||
- Origin URL.
|
||||
- Quarantine diagnostics, debug flags, performance data, and transient caches.
|
||||
- Cosmetic preferences that are not part of the field.
|
||||
|
||||
Portable records omit local revisions and authors. Import assigns a fresh epoch
|
||||
and fresh local revisions.
|
||||
|
||||
### Export pipeline
|
||||
|
||||
The export button obtains its destination synchronously while it still has user
|
||||
activation. Destination priority is:
|
||||
|
||||
1. `showSaveFilePicker()` and a direct `FileSystemWritableFileStream`.
|
||||
2. An OPFS temporary file whose resulting `File` is downloaded.
|
||||
3. An in-memory Blob only when the projected archive is at most 64 MiB.
|
||||
|
||||
All capabilities are feature-detected. Lack of gzip support selects identity
|
||||
encoding; it does not change the archive record format.
|
||||
|
||||
After a destination is available, export:
|
||||
|
||||
1. Flushes pending field writes and aborts if the flush fails.
|
||||
2. Acquires the exclusive world lock.
|
||||
3. Captures the active epoch, global record, board count, and size estimate.
|
||||
4. Reads board IDs in deterministic pages using short readonly transactions.
|
||||
5. Encodes records through a worker and writes them with stream backpressure.
|
||||
6. Writes the footer, closes the destination, and releases the lock.
|
||||
|
||||
Canonical database writes are queued while the lock is held. The archive
|
||||
therefore represents the committed field at export start. The UI may allow
|
||||
navigation but must pause puzzle edits, purchases, field placement, reset, and
|
||||
time-attack mutations until export finishes or is canceled.
|
||||
|
||||
Worker input uses approximately 1 MiB chunks with at most two chunks in flight.
|
||||
The worker performs JSON serialization, parsing, and CRC work. Native streams
|
||||
perform compression and decompression.
|
||||
|
||||
## Import and activation
|
||||
|
||||
### Inspection
|
||||
|
||||
`inspectFieldArchive(file, { signal })` reads enough of the file to display:
|
||||
|
||||
- Archive and save-schema versions.
|
||||
- Export date.
|
||||
- World generation.
|
||||
- Board count.
|
||||
- Estimated raw size.
|
||||
- Compression mode.
|
||||
|
||||
Inspection does not trust the declared values and does not activate anything.
|
||||
The complete stream is validated during staging.
|
||||
|
||||
### Limits
|
||||
|
||||
Version-2 import enforces:
|
||||
|
||||
- Compressed file size at most 1 GiB.
|
||||
- Decoded content at most 2 GiB.
|
||||
- A single decoded record at most 8 MiB.
|
||||
- Expansion ratio at most 100:1.
|
||||
- At most `MAX_BOARDS`, currently 200,000.
|
||||
- Existing board-coordinate, chunk-shape, puzzle-cell, and path-count limits.
|
||||
- Write batches of at most 256 boards or 4 MiB.
|
||||
|
||||
`navigator.storage.estimate()` provides an advisory preflight. Import requests
|
||||
approximately twice the estimated staged size plus 32 MiB of headroom because
|
||||
the old and new fields coexist during validation. The actual IndexedDB result
|
||||
remains authoritative: `QuotaExceededError` aborts staging without deleting the
|
||||
active field.
|
||||
|
||||
Large or version-2 restores require IndexedDB. There is no destructive
|
||||
localStorage-only fallback.
|
||||
|
||||
### Strict validation
|
||||
|
||||
Each record is treated as untrusted data. Version-2 restore fails closed for:
|
||||
|
||||
- Invalid archive identity, ordering, or unsupported version.
|
||||
- A schema/world-generation pair without an explicit migration.
|
||||
- Duplicate or noncanonical board IDs.
|
||||
- Missing or invalid B0.
|
||||
- Missing metadata or state.
|
||||
- Invalid puzzle bounds, gates, cells, paths, stores, or references.
|
||||
- Overlapping world chunks.
|
||||
- Board, byte, line, cell, path, or expansion-ratio limits.
|
||||
- A missing footer or incorrect board count, byte count, or CRC.
|
||||
|
||||
Unlike current tolerant snapshot normalization, a damaged version-2 archive
|
||||
never silently drops a board.
|
||||
|
||||
Validation maintains occupied chunks and derived totals while records are
|
||||
staged. After the final record, a database pass verifies staged counts,
|
||||
revisions, summary/detail agreement, B0 hydration, and global aggregates.
|
||||
|
||||
`importFieldArchive()` stops after producing a completely validated `ready`
|
||||
epoch. It never changes the active pointer; activation is a separate operation
|
||||
after the replacement confirmation.
|
||||
|
||||
### Staging and atomic activation
|
||||
|
||||
Import creates a new local epoch with `status: "staging"`. The epoch from the
|
||||
archive is never reused. Each validated batch is committed independently and
|
||||
updates resumable progress on its `WorldRecord`.
|
||||
|
||||
After complete validation, the epoch becomes `ready`. Activation then acquires
|
||||
the world lock and uses one short transaction to:
|
||||
|
||||
1. Re-read the expected active epoch and global revision.
|
||||
2. Move the current epoch to `rollback`.
|
||||
3. Mark the ready epoch `active`.
|
||||
4. Set `previousEpoch`.
|
||||
5. Flip `WorldControl.activeEpoch`.
|
||||
6. Mark the activation as unverified.
|
||||
|
||||
Only after this transaction commits may localStorage hints and cross-tab
|
||||
replacement messages be updated.
|
||||
|
||||
The application reloads into the new epoch. It verifies the header, first index
|
||||
page, B0 puzzle/state, and occupancy before enabling mutations. Failure at this
|
||||
point atomically restores `previousEpoch`.
|
||||
|
||||
After the new epoch completes its first durable gameplay checkpoint,
|
||||
`activationVerified` becomes true and the old epoch becomes garbage. At most
|
||||
one rollback epoch is retained, and it is not presented as save history.
|
||||
|
||||
Canceled, interrupted, corrupt, quota-failing, or worker-failing imports leave
|
||||
the active pointer unchanged. Abandoned staging epochs are marked for cleanup
|
||||
and removed after 24 hours.
|
||||
|
||||
Restored archives start local-only with no cloud profile or outbox. Reconnecting
|
||||
cloud sync requires an explicit player action and a separately designed paged
|
||||
full-sync protocol. The current 10,000-change push path must not be used to
|
||||
silently upload a very large restored world.
|
||||
|
||||
### Legacy JSON
|
||||
|
||||
Existing JSON envelopes and raw snapshots remain importable and retain the
|
||||
current 100 MiB file limit. They are sanitized before conversion and pass
|
||||
through the same epoch-staging and activation pipeline.
|
||||
|
||||
Compatibility is handled through explicit
|
||||
`migrateArchiveRecord(fromSchema, record)` registrations. An unknown schema or
|
||||
world generation is rejected instead of silently creating a fresh field.
|
||||
|
||||
Legacy normalization may repair or discard malformed data. When that occurs,
|
||||
the importer displays a lossy-import report with totals and the first affected
|
||||
board IDs. Activation requires separate confirmation of that report. New
|
||||
version-2 archives never use lossy repair.
|
||||
|
||||
## Migration from the current database
|
||||
|
||||
The version-change transaction creates stores and indexes only. It must not copy
|
||||
an entire field inside `onupgradeneeded`.
|
||||
|
||||
While `activeFormat` is 1:
|
||||
|
||||
1. Startup continues using the existing stores.
|
||||
2. One tab becomes migration leader through the existing world lock.
|
||||
3. A new version-2 staging epoch is created.
|
||||
4. Legacy rows are copied in resumable batches.
|
||||
5. Normal commits dual-write legacy and version-2 records in the same
|
||||
transaction.
|
||||
6. Backfill uses revision-conditional writes so it cannot overwrite newer
|
||||
dual-written data.
|
||||
7. The last legacy cursor key and copied counts are persisted after every
|
||||
batch.
|
||||
|
||||
After board, global, tombstone, recovery, and outbox counts and revisions
|
||||
reconcile, the persistence queue is flushed and `activeFormat` switches to 2 in
|
||||
one transaction. Dual-writing continues until the first version-2 activation
|
||||
verification checkpoint.
|
||||
|
||||
Quota failure, a blocked upgrade, or interruption leaves format 1 authoritative
|
||||
and retryable. The old stores remain for one application release and are
|
||||
removed only by a later IndexedDB version upgrade.
|
||||
|
||||
## Garbage collection
|
||||
|
||||
Garbage collection:
|
||||
|
||||
- Never deletes the active, previous, or recovery-pinned epoch.
|
||||
- Deletes no more than 500 rows per idle batch.
|
||||
- Persists the current store and key after every batch.
|
||||
- Resumes after reload or interruption.
|
||||
- Removes abandoned staging epochs older than 24 hours.
|
||||
- Removes a verified rollback epoch after the new field's first checkpoint.
|
||||
- Cleans failed OPFS export files.
|
||||
|
||||
## Interfaces
|
||||
|
||||
The repository and archive modules expose these conceptual operations:
|
||||
|
||||
```ts
|
||||
loadActiveWorldHeader(): Promise<{
|
||||
control: WorldControl;
|
||||
world: WorldRecord;
|
||||
}>;
|
||||
|
||||
scanBoardIndex(
|
||||
epoch: string,
|
||||
afterKey?: IDBValidKey,
|
||||
limit?: number
|
||||
): Promise<BoardIndexRecord[]>;
|
||||
|
||||
hydrateBoardDetails(
|
||||
epoch: string,
|
||||
id: string,
|
||||
expectedRevisions?: { metaRev: number; stateRev: number }
|
||||
): Promise<HydratedBoard>;
|
||||
|
||||
commitFieldDelta(delta: FieldDelta): Promise<CommitResult>;
|
||||
|
||||
inspectFieldArchive(
|
||||
file: File,
|
||||
options?: { signal?: AbortSignal }
|
||||
): Promise<ArchiveSummary>;
|
||||
|
||||
exportFieldArchive(options: {
|
||||
writable: WritableStream<Uint8Array>;
|
||||
signal?: AbortSignal;
|
||||
onProgress?: (progress: FieldArchiveProgress) => void;
|
||||
}): Promise<ExportSummary>;
|
||||
|
||||
importFieldArchive(options: {
|
||||
file: File;
|
||||
signal?: AbortSignal;
|
||||
onProgress?: (progress: FieldArchiveProgress) => void;
|
||||
}): Promise<ImportSummary>;
|
||||
|
||||
activateStagedWorld(
|
||||
stagedEpoch: string,
|
||||
expectedActive: { epoch: string; globalRev: number }
|
||||
): Promise<void>;
|
||||
|
||||
rollbackUnverifiedActivation(expectedEpoch: string): Promise<void>;
|
||||
migrateLegacyWorld(): Promise<void>;
|
||||
collectWorldGarbage(): Promise<void>;
|
||||
```
|
||||
|
||||
Progress has the following stable shape:
|
||||
|
||||
```ts
|
||||
interface FieldArchiveProgress {
|
||||
phase:
|
||||
| "prepare"
|
||||
| "encode"
|
||||
| "write"
|
||||
| "validate"
|
||||
| "stage"
|
||||
| "activate"
|
||||
| "cleanup";
|
||||
boardsDone: number;
|
||||
boardsTotal: number;
|
||||
bytesRead: number;
|
||||
bytesWritten: number;
|
||||
}
|
||||
```
|
||||
|
||||
Progress is reported at least every 250 ms while work is advancing.
|
||||
Cancellation is acknowledged within one worker chunk or one database batch.
|
||||
|
||||
## Performance budgets
|
||||
|
||||
- Initial gameplay readiness requires only the field header, first index page,
|
||||
and initial viewport details.
|
||||
- Startup does not load every puzzle or state.
|
||||
- Archive import/export adds at most 32 MiB of JavaScript heap beyond the
|
||||
resident field summaries and browser stream buffers.
|
||||
- At most two approximately 1 MiB worker chunks are in flight.
|
||||
- No archive or migration task occupies the main thread for 50 ms or longer.
|
||||
- IndexedDB writes contain at most 256 boards or 4 MiB.
|
||||
- Index scans use at most 512 records per transaction.
|
||||
- Import and export UI remains cancelable throughout encode, write, validation,
|
||||
and staging.
|
||||
|
||||
## Verification
|
||||
|
||||
### Semantic and privacy tests
|
||||
|
||||
- Gzip and identity-encoded round trips preserve the complete portable gameplay
|
||||
field.
|
||||
- Derived totals after import equal totals recomputed from the original field.
|
||||
- Archive text and decoded records contain none of the excluded credential,
|
||||
recovery, epoch, session, debug, or transport fields.
|
||||
- Numeric board ordering and CRC output are deterministic for identical
|
||||
portable input.
|
||||
|
||||
### Invalid archive tests
|
||||
|
||||
- Corrupt gzip data.
|
||||
- Truncated records or missing footer.
|
||||
- Incorrect CRC, byte count, or board count.
|
||||
- Reordered, duplicated, or unknown records.
|
||||
- Duplicate IDs, missing B0, and overlapping geometry.
|
||||
- Malformed puzzles, paths, stores, and references.
|
||||
- Oversized file, decoded stream, line, board count, and expansion ratio.
|
||||
- Unsupported archive, save schema, or world-generation combination.
|
||||
|
||||
### Failure and concurrency tests
|
||||
|
||||
- Cancellation and injected failure before and after every staging batch.
|
||||
- Quota failure during preflight and during a write transaction.
|
||||
- Worker termination and destination-write failure.
|
||||
- Crash before activation, during pointer activation, and before verification.
|
||||
- Automatic rollback after failed B0 or first-index validation.
|
||||
- Stale-tab writes after import, reset, rollback, or migration cutover.
|
||||
- Cross-tab field replacement and blocked database upgrade.
|
||||
- Migration resume, dual-write conflict, and conditional backfill.
|
||||
- Garbage-collection resume and protection of active/rollback epochs.
|
||||
- Cloud remains disconnected after portable restore.
|
||||
|
||||
### Lazy-load tests
|
||||
|
||||
- B0 becomes interactive before the complete index scan finishes.
|
||||
- Details hydrate for viewport, gate adjacency, repair, inventory, and shop
|
||||
access.
|
||||
- Missing or mismatched revisions fail hydration.
|
||||
- Dirty, active, and referenced boards cannot be evicted.
|
||||
- Full-field summaries do not cause detail hydration.
|
||||
|
||||
### Browser and scale tests
|
||||
|
||||
- Direct file picker, OPFS, and bounded Blob output paths.
|
||||
- Gzip and identity compression paths.
|
||||
- Web Lock and storage-lease concurrency paths.
|
||||
- 10,000-board round trip and startup coverage in continuous integration.
|
||||
- 100,000-board and 200,000-board browser benchmarks as scheduled tests.
|
||||
- Long-task, heap, progress-frequency, cancellation-latency, transaction-size,
|
||||
and startup-readiness budgets.
|
||||
|
||||
Existing local `BEND_PERF` instrumentation should record only aggregate
|
||||
duration, byte, board, cancellation, quota, rollback, and cleanup metrics. No
|
||||
field contents, board IDs, player identity, or remote telemetry are added.
|
||||
|
||||
## Current implementation requirements
|
||||
|
||||
1. Add the version-6 stores, repository interfaces, epoch checks, and resumable
|
||||
migration without changing the active read path.
|
||||
2. Enable summary-first startup, real detail hydration, summary-based
|
||||
full-field operations, and the bounded LRU.
|
||||
3. Add the streamed `.bfsave` exporter and all output sinks.
|
||||
4. Add strict staged import, atomic activation, rollback, and legacy conversion.
|
||||
5. Route reset through the same epoch activation path.
|
||||
6. Garbage collection removes inactive epochs and obsolete stores only after verified activation.
|
||||
7. Cloud synchronization uses paged pulls, bounded push batches, recent-revision deltas, and per-board versioned server storage.
|
||||
52
docs/interaction-performance.md
Normal file
52
docs/interaction-performance.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
# Bend Field interaction performance and persistence
|
||||
|
||||
## Problems addressed
|
||||
|
||||
- Cursor movement felt uneven.
|
||||
- Camera panning could appear to stop redrawing during a long held gesture.
|
||||
- Pickup dragging needed a stable upper frame-rate limit.
|
||||
- Zoomed-out play could exhaust its cached field image while the camera was still moving.
|
||||
- A solved puzzle could later return to an unsolved state.
|
||||
- Puzzle gates still had a dormant debug scheduling switch.
|
||||
|
||||
## Implemented solutions
|
||||
|
||||
### Cursor, camera, and pickup cadence
|
||||
|
||||
- Cursor rendering now keeps only the newest pointer sample and commits it on a display frame.
|
||||
- Cursor commits use a 16.67 ms minimum interval, limiting presentation to 60 FPS even on high-refresh displays.
|
||||
- Camera commits use the same 60 Hz ceiling with a separate missed-frame fallback.
|
||||
- Pickup dragging remains on its bounded 60 Hz scheduler and keeps logical catch-up work separate from visual pointer tracking.
|
||||
|
||||
### Continuous overview panning
|
||||
|
||||
- The zoomed-out field still pans by transforming a cached bitmap, avoiding a full map redraw on every pointer event.
|
||||
- When a held pan reaches the bitmap's overscan boundary, the game now requests a cache rebuild during the gesture.
|
||||
- These rebuilds run through an idle callback, are separated by at least 180 ms, and are capped by the benchmark. Camera transforms continue while the new bitmap is prepared.
|
||||
|
||||
### Durable puzzle completion
|
||||
|
||||
- A durable `solved: true` value is now monotonic for the same puzzle.
|
||||
- Route sanitization may remove damaged path data, but it no longer revokes the clear, score, solver identity, or store state.
|
||||
- Existing cloud-pull, cross-tab, recovery-journal, and board-hydration merges continue to preserve compatible solved records.
|
||||
|
||||
### Production-only puzzle gates
|
||||
|
||||
- The `SPECIAL_CELL_DEBUG_ALL_LEVELS` switch was removed.
|
||||
- Internal gates and other special cells now use only the normal production level schedule, beginning at level 5.
|
||||
|
||||
## Verification
|
||||
|
||||
- Focused regression coverage executes the in-gesture cache refresh, durable-clear sanitization, and production-only gate schedule.
|
||||
- The complete fast test suite passes.
|
||||
- A real Edge normal-speed scenario measured:
|
||||
- display cadence: 16.70 ms median;
|
||||
- cursor cadence: 16.67 ms median;
|
||||
- cursor input age: 16.70 ms p95;
|
||||
- camera work: 0.20 ms p95;
|
||||
- no uncapped pickup presentation.
|
||||
- Edge and benchmark server processes are checked after every browser run; the final count is zero.
|
||||
|
||||
## Remaining stress-test observation
|
||||
|
||||
The optional 4× CPU-throttled Edge scenario recorded one 106.70 ms functional-drag outlier. Its steady cadence (8.40 ms p95), model work (2.80 ms p95), visual work (4.90 ms p95), and render work (2.60 ms p95) stayed within their individual budgets. This outlier should remain visible in future profiling rather than being hidden by a relaxed acceptance threshold.
|
||||
45
docs/test-policy.md
Normal file
45
docs/test-policy.md
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
# Test tiers and source-guard inventory
|
||||
|
||||
The release pipeline has four explicit tiers:
|
||||
|
||||
- `npm run test:fast` — deterministic unit, contract, persistence, server, and
|
||||
source-policy-compatible regression tests; no browser is launched.
|
||||
- `npm run test:browser` — the required real-browser interaction/performance
|
||||
matrix plus the store UI flow. Each runner owns one temporary profile and one
|
||||
process tree, closes it in `finally`, and never targets an unrelated Edge
|
||||
process. The UI runner uses `playwright-core` only as a driver for the
|
||||
system-provided Edge binary; it does not download a second browser.
|
||||
- `npm run test:storage` — opt-in large IndexedDB/archive scale coverage.
|
||||
- `npm run test:ci` — source policy, fast suite, and required browser release
|
||||
behavior. CI sets the small bounded browser profile and runs one job at a
|
||||
time.
|
||||
|
||||
## Source-shape guard inventory
|
||||
|
||||
The historical `source-smoke` and versioned `v47xx` files contain temporary
|
||||
implementation-shape tripwires. They remain only where no stable public seam
|
||||
exists yet. Their common reason is to prevent a known expensive or unsafe path
|
||||
from being accidentally restored. Their removal condition is one of:
|
||||
|
||||
1. a pure module has a behavioral unit test;
|
||||
2. a browser test measures the user-visible DOM, timing, or computed style;
|
||||
3. a protocol/storage integration test covers the invariant; or
|
||||
4. a generated artifact equality test covers the contract.
|
||||
|
||||
The following guards have already moved to public seams:
|
||||
|
||||
| Area | Public seam | Replacement coverage |
|
||||
|---|---|---|
|
||||
| Pointer ownership | `createGestureCoordinator` | `interaction-ownership-test.js` |
|
||||
| Interaction scopes | `createInteractionState` | `interaction-ownership-test.js` |
|
||||
| 60 Hz latest-value scheduling | `createFrameScheduler` | `frame-drag-scheduler-test.js` and browser cadence probes |
|
||||
| Pickup lifecycle/queue | `createDragScheduler` | `frame-drag-scheduler-test.js` and release-drain behavior |
|
||||
| Cursor identity/presentation | `createCursorModel` | `architecture-boundaries-test.js` and browser cursor probes |
|
||||
| HTTP dispatch | `createHttpRouter` | `architecture-boundaries-test.js` and server integration |
|
||||
| Authentication | `createAuthenticator` | `architecture-boundaries-test.js` and server security integration |
|
||||
| Atomic JSON storage | `createJsonRepository` | recovery and server integration tests |
|
||||
|
||||
When touching a remaining source assertion, migrate it to the nearest seam and
|
||||
delete the old assertion in the same change. New tests must not parse function
|
||||
source unless they enforce a documented repository policy that cannot be
|
||||
expressed as behavior.
|
||||
Loading…
Add table
Add a link
Reference in a new issue