Faster map with large feeds #27

New issue

Open

opened 2026-03-26 22:14:07 +00:00 by maxtkc · 0 comments

maxtkc commented

2026-03-26 22:14:07 +00:00

Owner

Summary

Large GTFS feeds frequently have 10k+ trips spread across only a few hundred unique shapes. The current RouteRenderer emits one GeoJSON feature per trip, so MapLibre tessellates and rasterizes the same line dozens of times per route — both feed-load time and per-frame paint scale with trip count instead of shape count. We will deduplicate features down to one per (route_id, geometry_key), where geometry_key is the trip's shape_id in shapes mode and a hash of the ordered stop_id sequence in stops mode (kept structurally parallel to shape handling). On top of dedupe, we will replace today's "rebuild from scratch on undo/redo only" model with patch-driven incremental updates: subscribe to the patch manager, route by table, and surgically rebuild only the affected features. Click semantics simplify — clicks resolve to a route, which is all the UI needs today. Trip-shape edits and (future) shape edits will rerender selectively without touching the rest of the map.

The likely 10–100× drop in feature count is the primary win; selective invalidation prevents the same cost from being paid on every edit. A Web Worker offload is documented as a fallback but not implemented in this plan unless dedupe + incremental updates fall short.

Relevant context

Files

src/modules/route-renderer.ts — owns the routes GeoJSON source and three layers (routes-background, routes-clickarea, routes-highlight). createRouteFeatures() (lines 171–270) is the hot path. Geometry is computed by createRouteGeometryFromShape() (275) or createRouteGeometryFromStops() (294). setRenderMode('shapes' | 'stops') (360) toggles the source for geometry.
src/modules/map-controller.ts — owns RouteRenderer and LayerManager. updateMap() (259) clears and full-rebuilds. Basemap-change handler (180) does the same. This module is the right place to subscribe to patch events and dispatch surgical rerenders.
src/modules/patch-manager.ts — emits 'change' | 'undo' | 'redo' | 'jump' via emit() (476). Patches are SingleGTFSPatch (op: 'insert' | 'update' | 'delete', with source: { table, id }) or BatchGTFSPatch (op: 'batch', ops: SingleGTFSPatch[]). Patch types live at src/types/patch.ts.
src/modules/layer-manager.ts — highlightTrip() (313) computes its own line from stop_times independently of RouteRenderer, so dedupe does not affect trip-highlight behavior.
src/modules/gtfs-parser.ts — getFileDataSyncTyped<T>(filename) returns shallow-copied row arrays (copy-on-read invariant per CLAUDE.md). Cached indexes derived from these arrays must be invalidated on patches; we cannot rely on aliasing.
src/index.ts (170–219) — currently calls mapController.updateMap() only on undo/redo/jump; change only refreshes browseNavigation. We will keep that wiring but MapController will additionally subscribe to all four events for selective rerender.
src/types/gtfs-entities.ts — Trips, Shapes, StopTimes, Stops, Routes row types.

Architectural patterns

All user edits go through patchManager.record*(). Direct DB writes happen only for import / patch replay / snapshot restore. The map can therefore listen to a single source — the patch manager — and trust it to fire for every user-initiated change.
Modules are wired by constructor injection from GTFSEditor in src/index.ts. MapController already receives gtfsParser; we will additionally pass patchManager so it can subscribe.
console.log with a [ModuleName] prefix is the convention for state-transition logging; keep adding them at every invalidation/rebuild for debuggability.

Highlight layer compatibility

routes-highlight filters by ['==', 'route_id', X] or ['in', 'route_id', ...]. After dedupe, multiple features per route share the same route_id — the filter still selects all of them, which is the desired behavior. No change needed.

Click handling

Confirmed: clicks only need to resolve to a route. Trip-level resolution from a route click is not used. Feature id becomes ${route_id}::${geometry_key} (string), properties keep route_id (the only field InteractionHandler reads off route features).

Phase 1 — Dedupe by `(route_id, geometry_key)`

Goal. Cut feature count from O(trips) to O(unique route×geometry pairs). Refactor RouteRenderer so geometry indexing is cached (not rebuilt on every render) and so geometry derivation is uniform across 'shapes' and 'stops' modes — both produce a geometry_key plus a coordinate array, which createRouteFeatures consumes identically.

Why first. This is the bulk of the perf win and is independent of the patch wiring. It also creates the data structures (geometry-keyed feature map, indexes) that Phase 2 mutates surgically.

Steps.

In route-renderer.ts, change routeFeatures from RouteFeature[] to Map<string, RouteFeature> keyed by ${route_id}::${geometry_key}, with stable iteration order maintained by insertion order. Update getRouteFeatures() to return [...map.values()].
Change RouteFeature.id from ${route_id}-${trip.trip_id} to ${route_id}::${geometry_key} (use :: to avoid collisions with - inside ids). Drop nothing from properties — keep route_id, route_data, color, route_short_name, route_long_name. Add trip_ids: string[] (the list of trip ids contributing to this feature) so future debugging and possible UI expansion has it; do not expose in click handling yet.
Add three private cached indexes on RouteRenderer:
- private shapeIndex: Map<string, [number, number][]> | null = null; — built from shapes.txt, identical to today's local index.
- private stopSeqIndex: Map<string, [number, number][]> | null = null; — keys are geometry_key for stop-sequence-derived geometries (see next bullet), values are coordinate arrays.
- private tripsByGeomKey: Map<string, { route_id: string; trip_ids: Set<string> }> | null = null; — bucket membership: which trips contribute to which feature. Used in Phase 2 to decide when a feature should be added/removed.
Define geometry_key derivation per trip:
- In shapes mode: if trip.shape_id is set and present in shapeIndex, key is shape:${trip.shape_id}.
- Otherwise (no shape, or stops mode): build the ordered stop_id sequence from stop_times.txt for that trip (sorted by stop_sequence). Key is stops:${stop_ids.join('|')}. Cache the resulting coordinate array in stopSeqIndex keyed by the same string. The full join is the key (no hashing) — content is the identity, collisions impossible. If a trip has fewer than 2 valid stop coords, no feature is created.
- This is the only place stops mode and shapes mode diverge. Everything downstream — feature creation, highlighting, click handling — is identical.
Replace createRouteFeatures() with a method that:
1. Lazily builds shapeIndex, stopSeqIndex, tripsByGeomKey if null.
2. Iterates trips once, computes geometry_key, and either creates a new feature for that (route_id, geometry_key) or appends trip.trip_id to the existing bucket. The route's color/name come from the routes lookup, computed once per route.
3. Stores results in routeFeatures (the Map) and tripsByGeomKey.
Add private invalidateAll(): void that nulls the three caches and clears routeFeatures. Call from clearRoutes() and from a basemap re-init.
Delete the old non-dedup feature creation code path. There is no backwards-compat to maintain.
Verify routes-highlight still works visually (filter by route_id matches multiple features with the same route_id — exactly what we want).
Add [RouteRenderer]-prefixed logs at: cache build start/end (with row counts), dedupe ratio (features emitted vs trips processed), and source setData call.

Gotchas.

Copy-on-read: gtfsParser.getFileDataSyncTyped returns shallow copies. Do not assume the row arrays we cached are still up-to-date after a patch — that's exactly what Phase 2's invalidation is for. Phase 1 alone still rebuilds caches every renderRoutes() call (preserving today's full-rebuild behavior on undo/redo/jump); the lazy caching only matters once Phase 2 controls invalidation.
A trip can have a shape_id that does not exist in shapes.txt. Today the code falls back to stop connections with a warning. Preserve that fallback in the new geometry_key derivation: missing-shape trips fall through to the stops-derived key.
The stops:<joined> key strings can be long for routes with many stops (~50 stops × ~10 chars = ~500 bytes). With a few hundred unique sequences this is negligible RAM. Do not pre-hash; hashing introduces a collision-handling burden for no measurable benefit.
Feature insertion order matters for paint order (later features draw on top). Today the implicit order is route order × trip order. After dedupe, preserve route iteration order; within a route, the order is whichever geometry_key we see first. This is acceptable — the user has not relied on cross-route stacking.

Phase 2 — Patch-driven incremental updates

Goal. Replace "full rebuild on undo/redo only" with "surgical rebuild on every patch event." MapController subscribes to all four patch events; for each patch, it routes by source.table to a targeted invalidation method on RouteRenderer. After invalidation, a single coalesced setData() is fired on the next animation frame.

Why second. Phase 1 establishes the bucket/index data structures this phase mutates. Phase 2 also lets us remove the updateMap() call from index.ts's undo/redo wiring (MapController will handle it via its own subscription), unifying the two paths.

Steps.

Add patchManager: PatchManager to MapController constructor. Wire it from src/index.ts where MapController is instantiated.
In MapController, after routeRenderer is constructed, subscribe to patchManager.on('change' | 'undo' | 'redo' | 'jump', cb). The handler receives a PatchRecord ({ version, patch, timestamp }).
In the handler, normalize patch to a flat list of SingleGTFSPatch: if patch.op === 'batch', iterate patch.ops; otherwise wrap in a single-element array. Then dispatch each by patch.source.table to one of:
- 'routes' → routeRenderer.invalidateRoute(id, op)
- 'trips' → routeRenderer.invalidateTrip(id, op, before, after) (we need both before/after to know the old vs new shape_id; for update, derive from forward.changes + the current row; for delete, the inverse has the row; for insert, the forward has it)
- 'shapes' → routeRenderer.invalidateShape(id, op) (note: id here is shape_id — see gotcha below)
- 'stop_times' → routeRenderer.invalidateStopTimes(trip_id, op) (the patch id for stop_times is the composite ${trip_id}_${stop_sequence}; extract trip_id from it)
- 'stops' → routeRenderer.invalidateStop(stop_id, op)
- any other table → no-op.
For each invalidate* method on RouteRenderer, the contract is: update the relevant cached index, recompute affected (route_id, geometry_key) buckets, mutate routeFeatures (add/remove/replace specific entries), and enqueue a setData() on the next frame. Concrete behaviors:
- invalidateRoute(route_id, op):
  - On update (color/name): mutate the properties.color / name fields in-place on every feature with this route_id. No geometry work.
  - On insert: build features for any trips already in tripsByRoute (rare on insert, but handle it).
  - On delete: remove every feature whose route_id matches; drop from tripsByGeomKey.
- invalidateTrip(trip_id, op, beforeShapeId, afterShapeId):
  - Compute oldKey from beforeShapeId (or stop sequence if missing); compute newKey from afterShapeId. Remove trip_id from the old bucket; if the bucket becomes empty, delete the feature. Add trip_id to the new bucket; if the bucket is new, build the feature. If oldKey === newKey, no-op.
  - On insert: only newKey side runs; oldKey is null.
  - On delete: only oldKey side runs.
- invalidateShape(shape_id, op):
  - Recompute shapeIndex.get(shape_id) from current shapes.txt rows for that id. (For delete: remove the entry.) Then for every feature with geometry_key === 'shape:' + shape_id, replace its geometry.coordinates. If the new coords are <2 points, drop the feature and reassign its trips to the stop-sequence fallback (this keeps semantics consistent with initial build).
- invalidateStopTimes(trip_id, op):
  - The trip's stop sequence may have changed → its geometry_key (in stops mode, or in shapes mode if it had no shape) may have moved. Recompute and call the same bucket-move logic as invalidateTrip. If shapes-mode and the trip has a valid shape_id, no-op (stop_times don't affect shape-mode geometry).
- invalidateStop(stop_id, op):
  - Recompute coords for every stops: key in stopSeqIndex that contains this stop_id. Replace geometry.coordinates on every affected feature. (We can keep a reverse index stopId → Set<geometry_key> to make this O(features-touching-stop) instead of O(all stop-mode features); add it if profiling shows the linear scan hurts.)
Coalesce: each invalidate* call sets a dirty flag and schedules a single requestAnimationFrame that calls source.setData(asFeatureCollection(routeFeatures)) and clears the flag. A burst of patches in the same task collapses to one paint.
Remove the updateMap() call from the undo/redo/jump handler in src/index.ts. The MapController patch subscription replaces it. (updateMap() itself stays — initial load and basemap change still call it as a hard reset.)
Add [MapController]-prefixed logs at every dispatch decision: [MapController] patch routes:abc op=update → invalidateRoute, etc.
Edge case: a patch's before value for a trip's shape_id is not directly provided in update patches (only the changed fields). The patch handler must read the row's current state from the parser before the DB write would have applied — but patchManager.recordUpdate already calls applyPatchForward before emitting 'change', so by the time we receive the event the new value is live. Solution: compute oldKey from patch.inverse.changes (the recorded before-values) rather than re-reading from the parser. For insert, before is null. For delete, before is the full record in patch.inverse.record.

Gotchas.

The stop_times primary key shape: confirm the id format in this codebase (probably ${trip_id}_${stop_sequence}) before writing the extraction logic. If it differs, adjust invalidateStopTimes.
A single user action (e.g. "duplicate trip") may emit a batch patch covering trips + stop_times. The coalesced setData ensures both invalidations land in one paint, but the order of invalidate* calls matters: if stop_times is invalidated before its trip exists in tripsByRoute, the bucket-move runs on a non-existent trip. Process the batch's ops in order (insert before update before delete by default already, but follow the order the patch records) and tolerate "trip not yet known" by checking presence.
tripsByGeomKey is the source of truth for which features should exist. After every invalidation, the invariant routeFeatures.has(key) ⇔ tripsByGeomKey.get(key)?.trip_ids.size > 0 must hold. Add an assertion-style check (gated behind a debug flag) that runs in dev to catch leaks.
Snapshot restore (loading from a snapshot during undo) bypasses individual patches and replaces the whole DB. After a snapshot restore the patch manager fires a 'jump' event but no per-row patches. Treat 'jump' as a hard reset: invalidate all caches and trigger a full rebuild (essentially what updateMap() does today). Same for the change-but-from-import-replay scenarios — but change is only emitted from appendAndPush, which is only user-initiated, so we should be safe.

Phase 3 (Possible improvement — not implemented now)

Goal if needed. Move feature building off the main thread when feed is very large.

When to do it. After Phase 1+2 ship, profile on a feed with 100k+ trips. If createRouteFeatures (the cached-build path) blocks the main thread for >100 ms, do this. Patch-driven invalidation is already incremental, so the main-thread cost on edit should be tiny — Worker is only useful for the cold-start build.

Implementation sketch (not in this PR).

Create src/workers/route-feature-builder.worker.ts. Input message: { routes, trips, shapes, stop_times, stops } (raw row arrays, transferable). Output: { features: RouteFeature[], tripsByGeomKey: ...serialized..., shapeIndex: ..., stopSeqIndex: ... }.
In RouteRenderer, replace the synchronous initial build with a Worker postMessage; await the response inside renderRoutes(). Fall back to the synchronous path if Worker is unavailable (SSR/test contexts).
Keep all invalidate* methods on the main thread — they operate on already-built indexes and are O(affected features), no benefit from workerization.
Use structuredClone-friendly types throughout the worker boundary; do not pass class instances. The existing row types are plain objects, so this is mostly free.
Memory cost: row arrays are duplicated across the boundary. For a 100k-trip feed this is on the order of 50 MB; acceptable but not free. Consider Transferable ArrayBuffers if it becomes a concern.
Cancellation: if a user triggers another reload before the previous build completes, drop the stale message. Track the latest "build id" and ignore responses with mismatched ids.
No need to share the Worker with other modules — keep it dedicated to route building. One Worker, owned by RouteRenderer, lifecycle tied to RouteRenderer.destroy().

Original Issue

Extension to #21. Now that we can load the large feeds, the map is very slow. I think it might be because we're drawing every trip/every shape? Maybe we can dedup them or simplify them.

## Summary Large GTFS feeds frequently have 10k+ trips spread across only a few hundred unique shapes. The current `RouteRenderer` emits one GeoJSON feature per trip, so MapLibre tessellates and rasterizes the same line dozens of times per route — both feed-load time and per-frame paint scale with trip count instead of shape count. We will deduplicate features down to one per `(route_id, geometry_key)`, where `geometry_key` is the trip's `shape_id` in shapes mode and a hash of the ordered `stop_id` sequence in stops mode (kept structurally parallel to shape handling). On top of dedupe, we will replace today's "rebuild from scratch on undo/redo only" model with patch-driven incremental updates: subscribe to the patch manager, route by table, and surgically rebuild only the affected features. Click semantics simplify — clicks resolve to a route, which is all the UI needs today. Trip-shape edits and (future) shape edits will rerender selectively without touching the rest of the map. The likely 10–100× drop in feature count is the primary win; selective invalidation prevents the same cost from being paid on every edit. A Web Worker offload is documented as a fallback but not implemented in this plan unless dedupe + incremental updates fall short. ## Relevant context **Files** - `src/modules/route-renderer.ts` — owns the `routes` GeoJSON source and three layers (`routes-background`, `routes-clickarea`, `routes-highlight`). `createRouteFeatures()` (lines 171–270) is the hot path. Geometry is computed by `createRouteGeometryFromShape()` (275) or `createRouteGeometryFromStops()` (294). `setRenderMode('shapes' | 'stops')` (360) toggles the source for geometry. - `src/modules/map-controller.ts` — owns `RouteRenderer` and `LayerManager`. `updateMap()` (259) clears and full-rebuilds. Basemap-change handler (180) does the same. This module is the right place to subscribe to patch events and dispatch surgical rerenders. - `src/modules/patch-manager.ts` — emits `'change' | 'undo' | 'redo' | 'jump'` via `emit()` (476). Patches are `SingleGTFSPatch` (`op: 'insert' | 'update' | 'delete'`, with `source: { table, id }`) or `BatchGTFSPatch` (`op: 'batch'`, `ops: SingleGTFSPatch[]`). Patch types live at `src/types/patch.ts`. - `src/modules/layer-manager.ts` — `highlightTrip()` (313) computes its own line from `stop_times` independently of `RouteRenderer`, so dedupe does not affect trip-highlight behavior. - `src/modules/gtfs-parser.ts` — `getFileDataSyncTyped<T>(filename)` returns shallow-copied row arrays (copy-on-read invariant per CLAUDE.md). Cached indexes derived from these arrays must be invalidated on patches; we cannot rely on aliasing. - `src/index.ts` (170–219) — currently calls `mapController.updateMap()` only on undo/redo/jump; `change` only refreshes `browseNavigation`. We will keep that wiring but `MapController` will additionally subscribe to all four events for selective rerender. - `src/types/gtfs-entities.ts` — `Trips`, `Shapes`, `StopTimes`, `Stops`, `Routes` row types. **Architectural patterns** - All user edits go through `patchManager.record*()`. Direct DB writes happen only for import / patch replay / snapshot restore. The map can therefore listen to a single source — the patch manager — and trust it to fire for every user-initiated change. - Modules are wired by constructor injection from `GTFSEditor` in `src/index.ts`. `MapController` already receives `gtfsParser`; we will additionally pass `patchManager` so it can subscribe. - `console.log` with a `[ModuleName]` prefix is the convention for state-transition logging; keep adding them at every invalidation/rebuild for debuggability. **Highlight layer compatibility** - `routes-highlight` filters by `['==', 'route_id', X]` or `['in', 'route_id', ...]`. After dedupe, multiple features per route share the same `route_id` — the filter still selects all of them, which is the desired behavior. No change needed. **Click handling** - Confirmed: clicks only need to resolve to a route. Trip-level resolution from a route click is not used. Feature `id` becomes `${route_id}::${geometry_key}` (string), `properties` keep `route_id` (the only field `InteractionHandler` reads off route features). ## Phase 1 — Dedupe by `(route_id, geometry_key)` **Goal.** Cut feature count from O(trips) to O(unique route×geometry pairs). Refactor `RouteRenderer` so geometry indexing is cached (not rebuilt on every render) and so geometry derivation is uniform across `'shapes'` and `'stops'` modes — both produce a `geometry_key` plus a coordinate array, which `createRouteFeatures` consumes identically. **Why first.** This is the bulk of the perf win and is independent of the patch wiring. It also creates the data structures (geometry-keyed feature map, indexes) that Phase 2 mutates surgically. **Steps.** - [x] In `route-renderer.ts`, change `routeFeatures` from `RouteFeature[]` to `Map<string, RouteFeature>` keyed by `${route_id}::${geometry_key}`, with stable iteration order maintained by insertion order. Update `getRouteFeatures()` to return `[...map.values()]`. - [x] Change `RouteFeature.id` from `${route_id}-${trip.trip_id}` to `${route_id}::${geometry_key}` (use `::` to avoid collisions with `-` inside ids). Drop nothing from properties — keep `route_id`, `route_data`, `color`, `route_short_name`, `route_long_name`. Add `trip_ids: string[]` (the list of trip ids contributing to this feature) so future debugging and possible UI expansion has it; do not expose in click handling yet. - [x] Add three private cached indexes on `RouteRenderer`: - `private shapeIndex: Map<string, [number, number][]> | null = null;` — built from `shapes.txt`, identical to today's local index. - `private stopSeqIndex: Map<string, [number, number][]> | null = null;` — keys are `geometry_key` for stop-sequence-derived geometries (see next bullet), values are coordinate arrays. - `private tripsByGeomKey: Map<string, { route_id: string; trip_ids: Set<string> }> | null = null;` — bucket membership: which trips contribute to which feature. Used in Phase 2 to decide when a feature should be added/removed. - [x] Define `geometry_key` derivation per trip: - In shapes mode: if `trip.shape_id` is set and present in `shapeIndex`, key is `shape:${trip.shape_id}`. - Otherwise (no shape, or stops mode): build the ordered `stop_id` sequence from `stop_times.txt` for that trip (sorted by `stop_sequence`). Key is `stops:${stop_ids.join('|')}`. Cache the resulting coordinate array in `stopSeqIndex` keyed by the same string. The full join is the key (no hashing) — content is the identity, collisions impossible. If a trip has fewer than 2 valid stop coords, no feature is created. - This is the only place stops mode and shapes mode diverge. Everything downstream — feature creation, highlighting, click handling — is identical. - [x] Replace `createRouteFeatures()` with a method that: 1. Lazily builds `shapeIndex`, `stopSeqIndex`, `tripsByGeomKey` if `null`. 2. Iterates trips once, computes `geometry_key`, and either creates a new feature for that `(route_id, geometry_key)` or appends `trip.trip_id` to the existing bucket. The route's color/name come from the routes lookup, computed once per route. 3. Stores results in `routeFeatures` (the Map) and `tripsByGeomKey`. - [x] Add `private invalidateAll(): void` that nulls the three caches and clears `routeFeatures`. Call from `clearRoutes()` and from a basemap re-init. - [x] Delete the old non-dedup feature creation code path. There is no backwards-compat to maintain. - [ ] Verify `routes-highlight` still works visually (filter by `route_id` matches multiple features with the same `route_id` — exactly what we want). - [x] Add `[RouteRenderer]`-prefixed logs at: cache build start/end (with row counts), dedupe ratio (features emitted vs trips processed), and source `setData` call. **Gotchas.** - Copy-on-read: `gtfsParser.getFileDataSyncTyped` returns shallow copies. Do not assume the row arrays we cached are still up-to-date after a patch — that's exactly what Phase 2's invalidation is for. Phase 1 alone still rebuilds caches every `renderRoutes()` call (preserving today's full-rebuild behavior on undo/redo/jump); the lazy caching only matters once Phase 2 controls invalidation. - A trip can have a `shape_id` that does not exist in `shapes.txt`. Today the code falls back to stop connections with a warning. Preserve that fallback in the new `geometry_key` derivation: missing-shape trips fall through to the stops-derived key. - The `stops:<joined>` key strings can be long for routes with many stops (~50 stops × ~10 chars = ~500 bytes). With a few hundred unique sequences this is negligible RAM. Do not pre-hash; hashing introduces a collision-handling burden for no measurable benefit. - Feature insertion order matters for paint order (later features draw on top). Today the implicit order is route order × trip order. After dedupe, preserve route iteration order; within a route, the order is whichever `geometry_key` we see first. This is acceptable — the user has not relied on cross-route stacking. ## Phase 2 — Patch-driven incremental updates **Goal.** Replace "full rebuild on undo/redo only" with "surgical rebuild on every patch event." `MapController` subscribes to all four patch events; for each patch, it routes by `source.table` to a targeted invalidation method on `RouteRenderer`. After invalidation, a single coalesced `setData()` is fired on the next animation frame. **Why second.** Phase 1 establishes the bucket/index data structures this phase mutates. Phase 2 also lets us remove the `updateMap()` call from `index.ts`'s undo/redo wiring (`MapController` will handle it via its own subscription), unifying the two paths. **Steps.** - [ ] Add `patchManager: PatchManager` to `MapController` constructor. Wire it from `src/index.ts` where `MapController` is instantiated. - [ ] In `MapController`, after `routeRenderer` is constructed, subscribe to `patchManager.on('change' | 'undo' | 'redo' | 'jump', cb)`. The handler receives a `PatchRecord` (`{ version, patch, timestamp }`). - [ ] In the handler, normalize `patch` to a flat list of `SingleGTFSPatch`: if `patch.op === 'batch'`, iterate `patch.ops`; otherwise wrap in a single-element array. Then dispatch each by `patch.source.table` to one of: - `'routes'` → `routeRenderer.invalidateRoute(id, op)` - `'trips'` → `routeRenderer.invalidateTrip(id, op, before, after)` (we need both before/after to know the old vs new `shape_id`; for `update`, derive from `forward.changes` + the current row; for `delete`, the inverse has the row; for `insert`, the forward has it) - `'shapes'` → `routeRenderer.invalidateShape(id, op)` (note: `id` here is `shape_id` — see gotcha below) - `'stop_times'` → `routeRenderer.invalidateStopTimes(trip_id, op)` (the patch `id` for stop_times is the composite `${trip_id}_${stop_sequence}`; extract `trip_id` from it) - `'stops'` → `routeRenderer.invalidateStop(stop_id, op)` - any other table → no-op. - [ ] For each `invalidate*` method on `RouteRenderer`, the contract is: update the relevant cached index, recompute affected `(route_id, geometry_key)` buckets, mutate `routeFeatures` (add/remove/replace specific entries), and enqueue a `setData()` on the next frame. Concrete behaviors: - `invalidateRoute(route_id, op)`: - On `update` (color/name): mutate the `properties.color` / name fields in-place on every feature with this `route_id`. No geometry work. - On `insert`: build features for any trips already in `tripsByRoute` (rare on insert, but handle it). - On `delete`: remove every feature whose `route_id` matches; drop from `tripsByGeomKey`. - `invalidateTrip(trip_id, op, beforeShapeId, afterShapeId)`: - Compute `oldKey` from `beforeShapeId` (or stop sequence if missing); compute `newKey` from `afterShapeId`. Remove `trip_id` from the old bucket; if the bucket becomes empty, delete the feature. Add `trip_id` to the new bucket; if the bucket is new, build the feature. If `oldKey === newKey`, no-op. - On `insert`: only `newKey` side runs; `oldKey` is null. - On `delete`: only `oldKey` side runs. - `invalidateShape(shape_id, op)`: - Recompute `shapeIndex.get(shape_id)` from current `shapes.txt` rows for that id. (For `delete`: remove the entry.) Then for every feature with `geometry_key === 'shape:' + shape_id`, replace its `geometry.coordinates`. If the new coords are <2 points, drop the feature and reassign its trips to the stop-sequence fallback (this keeps semantics consistent with initial build). - `invalidateStopTimes(trip_id, op)`: - The trip's stop sequence may have changed → its `geometry_key` (in stops mode, or in shapes mode if it had no shape) may have moved. Recompute and call the same bucket-move logic as `invalidateTrip`. If shapes-mode and the trip has a valid shape_id, no-op (stop_times don't affect shape-mode geometry). - `invalidateStop(stop_id, op)`: - Recompute coords for every `stops:` key in `stopSeqIndex` that contains this `stop_id`. Replace `geometry.coordinates` on every affected feature. (We can keep a reverse index `stopId → Set<geometry_key>` to make this O(features-touching-stop) instead of O(all stop-mode features); add it if profiling shows the linear scan hurts.) - [ ] Coalesce: each `invalidate*` call sets a dirty flag and schedules a single `requestAnimationFrame` that calls `source.setData(asFeatureCollection(routeFeatures))` and clears the flag. A burst of patches in the same task collapses to one paint. - [ ] Remove the `updateMap()` call from the undo/redo/jump handler in `src/index.ts`. The `MapController` patch subscription replaces it. (`updateMap()` itself stays — initial load and basemap change still call it as a hard reset.) - [ ] Add `[MapController]`-prefixed logs at every dispatch decision: `[MapController] patch routes:abc op=update → invalidateRoute`, etc. - [ ] Edge case: a patch's `before` value for a trip's `shape_id` is not directly provided in `update` patches (only the changed fields). The patch handler must read the row's *current* state from the parser *before* the DB write would have applied — but `patchManager.recordUpdate` already calls `applyPatchForward` before emitting `'change'`, so by the time we receive the event the new value is live. Solution: compute `oldKey` from `patch.inverse.changes` (the recorded before-values) rather than re-reading from the parser. For `insert`, before is null. For `delete`, before is the full record in `patch.inverse.record`. **Gotchas.** - The `stop_times` primary key shape: confirm the `id` format in this codebase (probably `${trip_id}_${stop_sequence}`) before writing the extraction logic. If it differs, adjust `invalidateStopTimes`. - A single user action (e.g. "duplicate trip") may emit a batch patch covering trips + stop_times. The coalesced `setData` ensures both invalidations land in one paint, but the order of `invalidate*` calls matters: if `stop_times` is invalidated before its trip exists in `tripsByRoute`, the bucket-move runs on a non-existent trip. Process the batch's ops in order (insert before update before delete by default already, but follow the order the patch records) and tolerate "trip not yet known" by checking presence. - `tripsByGeomKey` is the source of truth for which features should exist. After every invalidation, the invariant `routeFeatures.has(key) ⇔ tripsByGeomKey.get(key)?.trip_ids.size > 0` must hold. Add an assertion-style check (gated behind a debug flag) that runs in dev to catch leaks. - Snapshot restore (loading from a snapshot during undo) bypasses individual patches and replaces the whole DB. After a snapshot restore the patch manager fires a `'jump'` event but no per-row patches. Treat `'jump'` as a hard reset: invalidate all caches and trigger a full rebuild (essentially what `updateMap()` does today). Same for the `change`-but-from-import-replay scenarios — but `change` is only emitted from `appendAndPush`, which is only user-initiated, so we should be safe. ## Phase 3 (Possible improvement — not implemented now) **Goal if needed.** Move feature building off the main thread when feed is very large. **When to do it.** After Phase 1+2 ship, profile on a feed with 100k+ trips. If `createRouteFeatures` (the cached-build path) blocks the main thread for >100 ms, do this. Patch-driven invalidation is *already* incremental, so the main-thread cost on edit should be tiny — Worker is only useful for the cold-start build. **Implementation sketch (not in this PR).** - [ ] Create `src/workers/route-feature-builder.worker.ts`. Input message: `{ routes, trips, shapes, stop_times, stops }` (raw row arrays, transferable). Output: `{ features: RouteFeature[], tripsByGeomKey: ...serialized..., shapeIndex: ..., stopSeqIndex: ... }`. - [ ] In `RouteRenderer`, replace the synchronous initial build with a Worker `postMessage`; await the response inside `renderRoutes()`. Fall back to the synchronous path if `Worker` is unavailable (SSR/test contexts). - [ ] Keep all `invalidate*` methods on the main thread — they operate on already-built indexes and are O(affected features), no benefit from workerization. - [ ] Use `structuredClone`-friendly types throughout the worker boundary; do not pass class instances. The existing row types are plain objects, so this is mostly free. - [ ] Memory cost: row arrays are duplicated across the boundary. For a 100k-trip feed this is on the order of 50 MB; acceptable but not free. Consider `Transferable` `ArrayBuffer`s if it becomes a concern. - [ ] Cancellation: if a user triggers another reload before the previous build completes, drop the stale message. Track the latest "build id" and ignore responses with mismatched ids. - [ ] No need to share the Worker with other modules — keep it dedicated to route building. One Worker, owned by `RouteRenderer`, lifecycle tied to `RouteRenderer.destroy()`. --- ## Original Issue Extension to #21. Now that we can load the large feeds, the map is very slow. I think it might be because we're drawing every trip/every shape? Maybe we can dedup them or simplify them.