# Event Analytics Read-only analytics over the booking events held in the **`event-tracking`** Firestore database. The events are produced by a system outside this monorepo; core only reads them, and no code here may ever write to that database. ## Where things live | Concern | File | | --- | --- | | Firestore client | `src/lib/firestore/event-tracking.ts` | | Source-data assumptions | `event-analytics.config.ts` | | Document → `EventRecord` | `event-analytics.mapper.ts` | | Queries + aggregation | `event-analytics.service.ts` | | HTTP handlers | `src/controllers/admin/event-analytics/` | | Routes | `src/routes/event-analytics/event-analytics.route.ts` | | Console UI | `apps/admin-console/src/routes/event-analytics/` | ## Credentials Defaults to the service account already configured for GCS — `GOOGLE_SERVICE_ACCOUNT_CREDENTIALS` and `GOOGLE_CLOUD_PROJECT_ID` (`karmagroup`). That works only if the event data lives in the same project, which it usually does not: Firestore event-tracking is typically a separate Firebase project. | Variable | Falls back to | When you need it | | --- | --- | --- | | `EVENT_TRACKING_PROJECT_ID` | `GOOGLE_CLOUD_PROJECT_ID` | The Firebase project is not `karmagroup`. | | `EVENT_TRACKING_SERVICE_ACCOUNT_CREDENTIALS` | `GOOGLE_SERVICE_ACCOUNT_CREDENTIALS` | That project needs its own service account. | | `EVENT_TRACKING_DATABASE_ID` | `event-tracking` | The database is named something else. | Whichever account is used needs `roles/datastore.viewer` on the target database. ## Member details The events carry only a member number, so a member-wise report reads as a list of bare digits on its own. Names, emails and account details come from the **Updot member DB** (`memberDatastore`), joined in `internal/repository/event-analytics/member-lookup.repo.ts`. - **SELECT only.** That database is production data. - The event field is named `memberNumber` but holds the **membership number**, so it is matched against `members.membership_number` and nothing else. Matching both columns with an OR was tried and is wrong: the two are distinct identifiers, so one member's `member_number` can equal a *different* member's `membership_number`, and the query returns both rows — printing somebody else's name against the events. Override with `EVENT_TRACKING_MEMBER_COLUMN` (`membership_number` | `member_number`) if the producer ever changes. - If one number somehow matches two members, **neither** profile is used and the number is shown instead. A wrong name is worse than no name. - Lookups are per **page**, not per aggregate: a wide filter can produce tens of thousands of members and looking all of them up to render 25 rows would be pure waste. - A lookup failure degrades to bare member numbers rather than failing the report. The Firestore figures are correct and useful without names. `GET /members/:memberId` returns the profile, that member's aggregates and their recent events in one round trip, for the member detail page at `/admin/event-analytics/members/:memberId`. ### Reading the gRPC errors The client reports three very different problems as terse gRPC statuses. `describeFirestoreError` in `lib/firestore/event-tracking.ts` translates each into the fix, but if you see one raw in the logs: | Code | Means | Fix | | --- | --- | --- | | `5 NOT_FOUND` | The database does not exist **in that project**. Not a permissions or credentials problem, and nothing to do with missing documents. | Set `EVENT_TRACKING_PROJECT_ID` to the Firebase project. | | `7 PERMISSION_DENIED` | Right project, no IAM. | Grant `roles/datastore.viewer`. | | `16 UNAUTHENTICATED` | The key itself was rejected. | Check the credentials JSON. | ## Database layout The database has **one top-level collection per event type**, not one collection with a type field: ``` event-tracking ├── add_to_cart ├── booking_hold_initiated └── purchase ``` Everything follows from this: - **The event type is the collection name.** `mapEventDoc` takes it as an argument; no document field carries it. - **Every read fans out.** Firestore cannot union differently-named top-level collections — a collection-group query only unifies collections that share an ID — so the service queries each selected collection and merges here. - **The event-type filter costs nothing.** Narrowing to `purchase` means querying one collection instead of three, rather than adding a `where` clause. - **Detail paging is a k-way merge.** Each collection contributes its next page, the merged head is taken, and each collection's cursor advances only past what was consumed. The cursor is therefore one position *per collection*, encoded into a single opaque string. - **Collections are discovered at runtime** via `listCollections()` (cached 5 minutes), so a new event type appears in the dashboard without a deploy. Pin the list with `EVENT_TRACKING_COLLECTIONS` to skip the round trip. The aggregation pass walks collections **sequentially**, not concurrently: the document ceiling is a budget shared across all of them, and a sequential scan can stop the moment it is spent. Detail paging and counting do run concurrently — there is no shared budget there. ## Document shape The defaults are matched to a real document from `add_to_cart`: ``` amount null number when the event carries revenue createdAt "2026-08-07T17:09:05.822772" naive ISO string, no timezone currency null experienceName null memberNumber "1278640" memberOffer null points 11 int64 propertyName "Karma Chakra" sourceOfEvent "properties" ``` Two consequences worth knowing before reading the dashboard: - **`sourceOfEvent` is not the event type.** It is where in the app the event came from ("properties"), and it cuts *across* event types — an `add_to_cart` and a `purchase` from the properties section share it. It is exposed as its own filter and column. - **Points, not revenue, are usually the quantity.** `amount` and `currency` are frequently null, so the revenue cards read "—" while points carry the real signal. Both are aggregated; neither is coerced to `0` when absent, which would drag averages down and imply a booking worth nothing. The producer writes no member name, email or booking reference, so those columns are empty. The keys remain in the field map in case the producer adds them. ## Configuration Every assumption about the document is isolated in `event-analytics.config.ts` and overridable by environment variable. All are optional. | Variable | Default | Purpose | | --- | --- | --- | | `EVENT_TRACKING_COLLECTIONS` | discovered | Comma-separated collection list. Unset means `listCollections()` finds them. | | `EVENT_TRACKING_FIELD_MAP` | see above | JSON object remapping any of `occurredAt`, `memberId`, `memberName`, `memberEmail`, `bookingRef`, `revenue`, `currency`, `points`, `propertyName`, `experienceName`, `memberOffer`, `source`. | | `EVENT_TRACKING_TIMESTAMP_KIND` | `string` | How `occurredAt` is stored: `timestamp`, `string` or `number`. | | `EVENT_TRACKING_MAX_AGGREGATION_DOCS` | `200000` | Ceiling on documents read per aggregation pass, across all collections. | Example: ``` EVENT_TRACKING_COLLECTIONS=add_to_cart,booking_hold_initiated,purchase EVENT_TRACKING_FIELD_MAP={"occurredAt":"created_at"} ``` There is no `eventType` key — the event type is the collection, not a field. `occurredAt`, `memberId`, `currency`, `propertyName` and `source` are used in Firestore `where` / `orderBy` clauses, so each must name exactly one real field. Every field additionally falls back to a list of common aliases (`FALLBACK_KEYS`) when reading an already-fetched document. ## Timestamps `createdAt` is a fixed-width ISO string with **no timezone**, so Firestore compares it lexicographically and that ordering matches chronological ordering. Date bounds are therefore built as text, padded to the producer's six fractional digits (`.000000` for a lower bound, `.999999` for an upper one) rather than round-tripped through `toISOString()` — a three-digit `...000Z` lower bound sorts *after* an event at exactly midnight, because `'Z' > '0'`, and would silently drop it. Filter dates are interpreted as UTC. If the producer writes local time rather than UTC, events near midnight will fall on the neighbouring day; that has not been confirmed either way. ## Things that fail silently Firestore returns empty results rather than errors for these, so check them first when the dashboard shows zeroes: - **Missing `databaseId`.** The client must be constructed with `databaseId: "event-tracking"`. Without it, it reads `(default)` and every query succeeds and returns nothing. - **Wrong `EVENT_TRACKING_TIMESTAMP_KIND`.** A `>=` against a `Timestamp` never matches a field stored as an ISO string. This producer stores strings, hence the `string` default. - **Wrong field names.** Nothing validates them. Collection names are the exception: an event-type filter is intersected against the discovered list, so an unknown one is dropped rather than turned into a query against a collection that does not exist. By contrast, a **missing composite index** does raise an error, and the message contains a click-to-create console URL. That message is deliberately surfaced verbatim through the API rather than replaced with "Server error". Indexes are per collection and are needed for the field combinations the filters produce — typically `createdAt` + `memberNumber`, `createdAt` + `propertyName` and `createdAt` + `sourceOfEvent`, in each of `add_to_cart`, `booking_hold_initiated` and `purchase`. ## Cost and limits - `GET /records` counts via Firestore's aggregation `count()`, which is billed in index-entry units rather than document reads. The count is **not** available while a text search is active — Firestore has no substring matching, so that predicate is applied in memory and the endpoint returns `total: null`. - Detail paging is cursor-based. `offset(N)` bills every skipped document, so deep paging on an offset query gets steadily more expensive; a cursor page costs the same at page 500 as at page 1. - Per-member, per-event-type and per-currency figures cannot be computed server-side by Firestore (there is no `GROUP BY`), so they come from a single streamed pass that produces all of them at once and is cached in Redis for 5 minutes. - **Bump `AGGREGATE_SCHEMA_VERSION` whenever `EventAnalyticsAggregate` gains a field.** A cached entry from an older build is still valid JSON and still deserialises — it is simply missing the new field, which reads as empty rather than as an error. Adding `byDay` without bumping it made the dashboard report "0 days with activity" against data that was present. There is a shape check on read as a second line of defence, but the version is the fix. - That pass stops at `EVENT_TRACKING_MAX_AGGREGATION_DOCS`. When it does, every response carries `truncated: true`, the dashboard shows a warning banner, and the export writes a "Partial data" row into its Filters sheet. A capped scan is never presented as a complete one.