diff --git a/.cursor/rules/mindspace-analytics-creator-username.mdc b/.cursor/rules/mindspace-analytics-creator-username.mdc index 4e15bc9..d2ad7e3 100644 --- a/.cursor/rules/mindspace-analytics-creator-username.mdc +++ b/.cursor/rules/mindspace-analytics-creator-username.mdc @@ -1,6 +1,6 @@ --- description: MindSpace 生成页埋点必须上报创建者 username，且不得用 owner_id identify 匿名访客 -globs: mindspace-analytics.mjs,mindspace-rybbit.mjs,mindspace-analytics.test.mjs,mindspace-rybbit.test.mjs,server/portal-session-routes.mjs,server/portal-integration-services-bootstrap.mjs +globs: mindspace-analytics.mjs,mindspace-analytics-discovery.mjs,mindspace-analytics.test.mjs,mindspace-analytics-discovery.test.mjs,server/portal-session-routes.mjs,server/portal-integration-services-bootstrap.mjs alwaysApply: false --- @@ -10,7 +10,7 @@ Umami「创建用户」依赖 `event_data.username` / session `username`。路 ## 必须 -- 服务端 `page_generated`（`sendMindSpaceAnalyticsEvent` / `sendMindSpaceRybbitEvent`）的事件属性 **必须**包含 `username`（创建者展示名，来自 `resolveAnalyticsOwnerLabel`） +- 服务端 `page_generated`（`sendMindSpaceAnalyticsEvent`）的事件属性 **必须**包含 `username`（创建者展示名，来自 `resolveAnalyticsOwnerLabel`） - 可同时保留 `owner_label` 作兼容，值与 `username` 相同 - Public HTML 内嵌脚本的 `metadata` **必须**继续带 `username`（给 scroll/click 等客户端事件） @@ -22,5 +22,5 @@ Umami「创建用户」依赖 `event_data.username` / session `username`。路 ## 改完必跑 ```bash -node --test mindspace-analytics.test.mjs mindspace-rybbit.test.mjs +node --test mindspace-analytics.test.mjs mindspace-analytics-discovery.test.mjs ``` diff --git a/.env.example b/.env.example index e1d5086..05ad1cb 100644 --- a/.env.example +++ b/.env.example @@ -18,8 +18,7 @@ H5_PORT=8081 # pnpm dev / dev-core 在未显式覆盖时会把 H5_PUBLIC_BASE_URL 设为 8081。 H5_PUBLIC_BASE_URL=http://127.0.0.1:5173 -# Local Umami analytics (disabled by default; points only to the local -# memind-analytics service when enabled). +# 访问统计平台说明（Umami 唯一；Rybbit 已退役 2026-08）见 docs/analytics-platform.md # MEMIND_ANALYTICS_ENABLED=true # MEMIND_ANALYTICS_URL=http://127.0.0.1:3100 # MEMIND_ANALYTICS_WEBSITE_ID= @@ -30,12 +29,7 @@ H5_PUBLIC_BASE_URL=http://127.0.0.1:5173 # MEMIND_PRODUCT_ANALYTICS_ENABLED=true # MEMIND_PRODUCT_ANALYTICS_WEBSITE_ID= -# Local Rybbit analytics (same-origin /rybbit proxy to rybbit.tkmind.cn). -# Create a Rybbit Site for localhost / 127.0.0.1, then set its numeric site_id. -# MEMIND_RYBBIT_ENABLED=true -# MEMIND_RYBBIT_URL=https://rybbit.tkmind.cn -# MEMIND_RYBBIT_SITE_ID= -# MEMIND_RYBBIT_ID_SECRET= +# 访问统计仅使用 Umami；Rybbit 已退役，见 docs/analytics-platform.md # 生产 H5 public base 当前临时切到 https://mm.tkmind.cn。 # 后续公网 H5 不再依赖 105 转发链路；m.tkmind.cn 仅作为 legacy/rollback 记录处理。 diff --git a/AGENTS.md b/AGENTS.md index ab737b4..34639ab 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -101,6 +101,7 @@ Cursor 额外加载：`.cursor/rules/mindspace-publish-chat-finish-guards.mdc` ## 仓库惯例 - 本地开发：`pnpm dev`（见 [README.md](README.md)） +- 访问统计：**仅 Umami**；Rybbit 已退役，见 [docs/analytics-platform.md](docs/analytics-platform.md) 与 [docs/local-analytics.md](docs/local-analytics.md) - Page Data API：见 [docs/page-data-api-usage.md](docs/page-data-api-usage.md)；改动相关路径后执行 `npm run verify:page-data` - 生产隔离：[docs/service-isolation-runbook.md](docs/service-isolation-runbook.md) - 发版须 Git commit，禁止本机直 `rsync` 到 `103/105` diff --git a/docs/analytics-platform.md b/docs/analytics-platform.md new file mode 100644 index 0000000..23886b7 --- /dev/null +++ b/docs/analytics-platform.md @@ -0,0 +1,37 @@ +# Memind 访问统计平台说明 + +## 当前唯一平台：Umami + +Memind 公开页与生成页的真实用户访问统计 **仅使用 Umami**（`memind-analytics` 独立部署，生产入口 `fxi.tkmind.cn`）。 + +- Portal 通过同源 `/analytics/*` 代理 Umami tracker +- 管理后台（memind_adm）通过 SSO 进入 Umami，并提供 **SEO / GEO 发现流量** 专用看板 +- 生成页事件维度见 `mindspace-analytics.mjs`（含 `discovery_channel` / `discovery_source`） + +本地配置见 [local-analytics.md](local-analytics.md)。 + +## 已退役：Rybbit + +**Rybbit 已于 2026-08 下线，不再部署、不再注入 tracker、不再提供 SSO。** + +历史背景：Rybbit 曾作为 Umami 并行的行为分析方案（`/rybbit/*` 代理、`MEMIND_RYBBIT_*` 环境变量、memind_adm Rybbit SSO）。该方案已废弃。 + +**禁止**在新代码、文档、环境变量示例或运维脚本中重新引入 Rybbit 相关配置。若看到遗留引用，应删除而非「可选保留」。 + +## SEO / GEO 流量口径 + +| 字段 | 含义 | +|------|------| +| `discovery_channel` | `seo`（搜索引擎）/ `geo`（生成式引擎）/ `direct` / `referral` | +| `discovery_source` | 具体来源，如 `google`、`baidu`、`chatgpt`、`perplexity` | +| `referrer_host` | 外部 referrer 域名（不含路径） | + +分类逻辑见 `mindspace-analytics-discovery.mjs`。Shell + iframe 场景会通过 `memind_discovery` 查询参数向内层传播来源。 + +## 相关仓库 + +| 仓库 | 职责 | +|------|------| +| **Memind** | tracker 注入、discovery 分类、Umami 代理 | +| **memind-analytics** | Umami 定制部署、`memind-pages` API（overlay 在 `deploy/umami/overlay/`） | +| **memind_adm** | Analytics 配置、SEO/GEO 看板、Umami SSO | diff --git a/docs/branch-disposition.md b/docs/branch-disposition.md index 8237f4b..0ef61eb 100644 --- a/docs/branch-disposition.md +++ b/docs/branch-disposition.md @@ -17,6 +17,8 @@ **状态：禁止再次引用。改动已提交并进入 `origin/main`，该分支保留仅用于只读追溯，不是待合并开发分支。** +> **2026-08 补充**：该分支引入的 Rybbit tracking 已于 2026-08 全量下线并自代码库移除；当前访问统计 **仅 Umami**，见 [analytics-platform.md](analytics-platform.md)。勿再从本分支恢复 Rybbit 相关实现。 + 审计日期：2026-07-27 分支 HEAD：`90ee4c3` `origin/main` 对应提交：`90ee4c3` @@ -40,7 +42,7 @@ 本分支闭环前已通过： -- `node --test mindspace-rybbit.test.mjs server/portal-domain-services-bootstrap.test.mjs server/portal-published-page-delivery.test.mjs server/portal-publication-shell.test.mjs` +- `node --test mindspace-analytics.test.mjs mindspace-analytics-discovery.test.mjs server/portal-domain-services-bootstrap.test.mjs server/portal-published-page-delivery.test.mjs server/portal-publication-shell.test.mjs`（历史验证含 `mindspace-rybbit.test.mjs`，Rybbit 已移除） - `npm run verify:chat-finish-sync` - `npm run verify:mindspace-publish-guards` - `npm run verify:mindspace-publish-guards:full` diff --git a/docs/local-analytics.md b/docs/local-analytics.md index 7cf1b33..31fdf33 100644 --- a/docs/local-analytics.md +++ b/docs/local-analytics.md @@ -1,79 +1,38 @@ -# Local Memind analytics (Umami + Rybbit) +# Local Memind analytics (Umami) -## Umami (optional) +> **Rybbit 已退役**，见 [analytics-platform.md](analytics-platform.md)。下文仅描述 Umami。 -Memind can still proxy `/analytics/*` to a local Umami service at -`http://127.0.0.1:3100`. +Memind 通过同源 `/analytics/*` 代理本地或远程 Umami 服务（默认 `http://127.0.0.1:3100`）。 ```dotenv MEMIND_ANALYTICS_ENABLED=true MEMIND_ANALYTICS_URL=http://127.0.0.1:3100 MEMIND_ANALYTICS_WEBSITE_ID= MEMIND_ANALYTICS_ID_SECRET= -MEMIND_ANALYTICS_IDENTITY_MODE=raw +MEMIND_ANALYTICS_IDENTITY_MODE=pseudonymous MEMIND_ANALYTICS_DOMAINS=127.0.0.1,localhost ``` -`MEMIND_ANALYTICS_IDENTITY_MODE=raw` sends the stable Memind user ID to an -explicitly approved first-party analytics service. Omit it elsewhere to keep the -pseudonymous default. +`MEMIND_ANALYTICS_IDENTITY_MODE=raw` 仅在明确批准的第一方分析环境中使用；其他环境保持 pseudonymous 默认。 -Create a second Umami Website for the Memind product shell. Keeping it separate -prevents chat, MindSpace, and feedback navigation from inflating generated-page -views. Do not create a Website per page or per user. +## Product shell（独立 Website） + +为 Memind 产品壳（H5 SPA）创建第二个 Umami Website，避免聊天与导航事件污染生成页 PV： ```dotenv MEMIND_PRODUCT_ANALYTICS_ENABLED=true MEMIND_PRODUCT_ANALYTICS_WEBSITE_ID= ``` -The product shell loads its tracker from the same-origin -`/analytics/script.js` endpoint. It records one standard page view per SPA route -transition, `product_click` for links/buttons, and 10/30-second route engagement -events. Query strings are restricted to a small allowlist and click labels never -copy arbitrary chat or generated-page text. Authenticated product events use the -same configured identity as generated pages while remaining in the separate -product Website. +产品壳 tracker 同样走 `/analytics/script.js`，记录 SPA 路由 PV、`product_click` 与 10/30 秒参与事件。 -## Rybbit (recommended for behavior analytics) +## SEO / GEO 发现流量 -Rybbit runs on 105 as `https://rybbit.tkmind.cn`. Local Memind does not talk to -103 for analytics. It proxies same-origin `/rybbit/*` to Rybbit `/api/*` so the -browser script stays first-party: +公开页 Umami 事件附带 `discovery_channel`（`seo` / `geo` / `direct` / `referral`）与 `discovery_source`。memind_adm **Analytics → SEO / GEO 流量** 看板按页展示 SEO 与 GEO 来源的浏览明细（口径与 Umami「生成页明细」一致）。 -```text -Generated Page - -> /rybbit/script.js -> https://rybbit.tkmind.cn/api/script.js - -> /rybbit/track -> https://rybbit.tkmind.cn/api/track -``` +## 管理后台 -1. In Rybbit, create one Site for the local host you use (`127.0.0.1` or - `localhost`). Do not create a Site per page or per user. -2. Put the numeric Site ID and a local pseudonymization secret in Memind `.env`: +- **memind_adm → Analytics 配置**：Umami Website ID、密钥、SSO 入口 +- **memind_adm → SEO / GEO 流量**：按页监控搜索引擎 / 生成式引擎来源访问 - ```dotenv - MEMIND_RYBBIT_ENABLED=true - MEMIND_RYBBIT_URL=https://rybbit.tkmind.cn - MEMIND_RYBBIT_SITE_ID=2 - MEMIND_RYBBIT_ID_SECRET= - ``` - - If `MEMIND_RYBBIT_ID_SECRET` is omitted, Portal bootstrap reuses the - MindSpace analytics secret stored in `mindspace_config` (the same value - configured below in memind_adm → Analytics 配置). - -3. Restart the local Memind server. Full generated HTML pages receive a - same-origin `/rybbit/script.js` tracker. Publication routes under - `/u/:owner/pages/:slug` and `/MindSpace/:user/public/...` both receive - the tracker. The tracker identifies the visitor - with a stable pseudonymous owner ID, then records engagement events such as - `page_click`, scroll depth, and dwell time. Initial pageviews come from - Rybbit's site setting `trackInitialPageView`. - -4. Open Rybbit from local `memind_adm` → Analytics 配置 →「打开 Rybbit 分析后台」 - (SSO). That path requires matching `MEMIND_RYBBIT_SSO_SECRET` / - `RYBBIT_SSO_EMAIL` with the 105 Rybbit deployment. - -Both integrations are fail-open: missing configuration or a down analytics -service leaves page generation and page delivery unchanged. Do not expose the -Rybbit or Umami dashboards directly to end users. +Analytics 服务 fail-open：未配置或 Umami 不可用时，页面生成与交付不受影响。勿向终端用户暴露 Umami 控制台。 diff --git a/mindspace-analytics-discovery.mjs b/mindspace-analytics-discovery.mjs new file mode 100644 index 0000000..632a9a3 --- /dev/null +++ b/mindspace-analytics-discovery.mjs @@ -0,0 +1,177 @@ +export const DISCOVERY_QUERY_PARAM = 'memind_discovery'; +export const DISCOVERY_SOURCE_QUERY_PARAM = 'memind_discovery_source'; + +const SEO_HOST_RULES = [ + { source: 'google', hosts: ['google.com', 'google.com.hk', 'google.co.jp', 'google.co.uk', 'google.ca', 'google.com.tw'] }, + { source: 'baidu', hosts: ['baidu.com'] }, + { source: 'bing', hosts: ['bing.com'] }, + { source: 'sogou', hosts: ['sogou.com'] }, + { source: 'so360', hosts: ['so.com'] }, + { source: 'sm', hosts: ['sm.cn'] }, + { source: 'yandex', hosts: ['yandex.ru', 'yandex.com', 'yandex.by'] }, + { source: 'duckduckgo', hosts: ['duckduckgo.com'] }, + { source: 'yahoo', hosts: ['search.yahoo.com', 'yahoo.com'] }, + { source: 'naver', hosts: ['search.naver.com', 'naver.com'] }, + { source: 'ecosia', hosts: ['ecosia.org'] }, + { source: 'brave', hosts: ['search.brave.com'] }, +]; + +const GEO_HOST_RULES = [ + { source: 'chatgpt', hosts: ['chatgpt.com', 'chat.openai.com'] }, + { source: 'openai', hosts: ['openai.com'] }, + { source: 'perplexity', hosts: ['perplexity.ai'] }, + { source: 'claude', hosts: ['claude.ai', 'anthropic.com'] }, + { source: 'gemini', hosts: ['gemini.google.com'] }, + { source: 'copilot', hosts: ['copilot.microsoft.com'] }, + { source: 'you', hosts: ['you.com'] }, + { source: 'phind', hosts: ['phind.com'] }, + { source: 'kagi', hosts: ['kagi.com'] }, + { source: 'poe', hosts: ['poe.com'] }, + { source: 'meta_ai', hosts: ['meta.ai'] }, + { source: 'doubao', hosts: ['doubao.com'] }, + { source: 'tongyi', hosts: ['tongyi.aliyun.com', 'qianwen.aliyun.com'] }, + { source: 'deepseek', hosts: ['chat.deepseek.com', 'deepseek.com'] }, +]; + +function normalizeHostname(value) { + return String(value ?? '').trim().toLowerCase().replace(/^www\./, '').slice(0, 255); +} + +function parseReferrerHost(referrer) { + const raw = String(referrer ?? '').trim(); + if (!raw) return ''; + try { + return normalizeHostname(new URL(raw).hostname); + } catch { + return ''; + } +} + +function matchHostRule(hostname, rules = []) { + const host = normalizeHostname(hostname); + if (!host) return ''; + for (const rule of rules) { + for (const candidate of rule.hosts) { + const normalized = normalizeHostname(candidate); + if (!normalized) continue; + if (host === normalized || host.endsWith(`.${normalized}`)) { + return rule.source; + } + } + } + return ''; +} + +function normalizeDiscoveryChannel(value) { + const channel = String(value ?? '').trim().toLowerCase(); + return channel === 'seo' || channel === 'geo' ? channel : ''; +} + +function classifyByUtm(utmSource = '', utmMedium = '') { + const source = String(utmSource ?? '').trim().toLowerCase(); + const medium = String(utmMedium ?? '').trim().toLowerCase(); + if (!source && !medium) return null; + if (medium === 'organic' || /seo|search|sem/.test(source)) { + return { discovery_channel: 'seo', discovery_source: source || 'utm' }; + } + if (/geo|ai|llm|gpt|copilot|perplexity|chatgpt/.test(source)) { + return { discovery_channel: 'geo', discovery_source: source || 'utm' }; + } + return null; +} + +export function classifyDiscoveryTrafficSource({ + referrer = '', + utmSource = '', + utmMedium = '', + discoveryChannel = '', + discoverySource = '', + pageHost = '', +} = {}) { + const normalizedChannel = normalizeDiscoveryChannel(discoveryChannel); + if (normalizedChannel) { + const source = String(discoverySource ?? '').trim().toLowerCase().slice(0, 32) || 'other'; + return { + discovery_channel: normalizedChannel, + discovery_source: source, + referrer_host: parseReferrerHost(referrer), + }; + } + + const utmMatch = classifyByUtm(utmSource, utmMedium); + const referrerHost = parseReferrerHost(referrer); + const currentHost = normalizeHostname(pageHost); + const externalHost = + referrerHost && (!currentHost || referrerHost !== currentHost) + ? referrerHost + : ''; + + if (utmMatch) { + return { + discovery_channel: utmMatch.discovery_channel, + discovery_source: utmMatch.discovery_source, + referrer_host: externalHost, + }; + } + + const geoSource = matchHostRule(externalHost, GEO_HOST_RULES); + if (geoSource) { + return { + discovery_channel: 'geo', + discovery_source: geoSource, + referrer_host: externalHost, + }; + } + + const seoSource = matchHostRule(externalHost, SEO_HOST_RULES); + if (seoSource) { + return { + discovery_channel: 'seo', + discovery_source: seoSource, + referrer_host: externalHost, + }; + } + + if (externalHost) { + return { + discovery_channel: 'referral', + discovery_source: 'other', + referrer_host: externalHost, + }; + } + + return { + discovery_channel: 'direct', + discovery_source: 'direct', + referrer_host: '', + }; +} + +export function appendDiscoveryPropagationParams(url, discovery = {}) { + const channel = normalizeDiscoveryChannel(discovery.discovery_channel); + if (!channel || !url) return String(url ?? ''); + let next = String(url); + const joiner = next.includes('?') ? '&' : '?'; + next += `${joiner}${encodeURIComponent(DISCOVERY_QUERY_PARAM)}=${encodeURIComponent(channel)}`; + const source = String(discovery.discovery_source ?? '').trim().toLowerCase(); + if (source && source !== 'direct' && source !== 'other') { + next += `&${encodeURIComponent(DISCOVERY_SOURCE_QUERY_PARAM)}=${encodeURIComponent(source.slice(0, 32))}`; + } + return next; +} + +export function buildInlineDiscoveryResolverScript(referrerHint = '') { + const rules = { + seo: SEO_HOST_RULES, + geo: GEO_HOST_RULES, + }; + const serializedRules = JSON.stringify(rules) + .replaceAll('<', '\\u003c') + .replaceAll('>', '\\u003e') + .replaceAll('&', '\\u0026'); + const serializedHint = JSON.stringify(String(referrerHint ?? '').slice(0, 512)) + .replaceAll('<', '\\u003c') + .replaceAll('>', '\\u003e') + .replaceAll('&', '\\u0026'); + return `(function(){var R=${serializedRules},H=${serializedHint};function norm(v){return String(v||"").trim().toLowerCase().replace(/^www\\./,"").slice(0,255)}function hostFrom(u){try{return norm(new URL(String(u||"")).hostname)}catch(e){return""}}function matchHost(h,rules){h=norm(h);if(!h)return"";for(var i=0;i { + assert.deepEqual( + classifyDiscoveryTrafficSource({ + referrer: 'https://www.google.com/search?q=demo', + }), + { + discovery_channel: 'seo', + discovery_source: 'google', + referrer_host: 'google.com', + }, + ); + assert.deepEqual( + classifyDiscoveryTrafficSource({ + referrer: 'https://www.baidu.com/s?wd=demo', + }), + { + discovery_channel: 'seo', + discovery_source: 'baidu', + referrer_host: 'baidu.com', + }, + ); +}); + +test('classifies common GEO referrers', () => { + assert.deepEqual( + classifyDiscoveryTrafficSource({ + referrer: 'https://chatgpt.com/c/abc', + }), + { + discovery_channel: 'geo', + discovery_source: 'chatgpt', + referrer_host: 'chatgpt.com', + }, + ); + assert.deepEqual( + classifyDiscoveryTrafficSource({ + referrer: 'https://www.perplexity.ai/search/demo', + }), + { + discovery_channel: 'geo', + discovery_source: 'perplexity', + referrer_host: 'perplexity.ai', + }, + ); +}); + +test('prefers explicit discovery query params for iframe propagation', () => { + assert.deepEqual( + classifyDiscoveryTrafficSource({ + discoveryChannel: 'seo', + discoverySource: 'google', + referrer: 'https://m.tkmind.cn/u/alice/pages/demo', + pageHost: 'm.tkmind.cn', + }), + { + discovery_channel: 'seo', + discovery_source: 'google', + referrer_host: 'm.tkmind.cn', + }, + ); +}); + +test('falls back to server referrer hint when browser referrer is same-origin', () => { + assert.deepEqual( + classifyDiscoveryTrafficSource({ + referrer: 'https://m.tkmind.cn/u/alice/pages/demo', + pageHost: 'm.tkmind.cn', + }), + { + discovery_channel: 'direct', + discovery_source: 'direct', + referrer_host: '', + }, + ); +}); + +test('appendDiscoveryPropagationParams only propagates seo or geo channels', () => { + const seoUrl = appendDiscoveryPropagationParams('/u/alice/pages/demo?view=raw', { + discovery_channel: 'seo', + discovery_source: 'google', + }); + assert.match(seoUrl, new RegExp(`${DISCOVERY_QUERY_PARAM}=seo`)); + assert.match(seoUrl, new RegExp(`${DISCOVERY_SOURCE_QUERY_PARAM}=google`)); + + const directUrl = appendDiscoveryPropagationParams('/u/alice/pages/demo?view=raw', { + discovery_channel: 'direct', + discovery_source: 'direct', + }); + assert.equal(directUrl, '/u/alice/pages/demo?view=raw'); +}); + +test('inline discovery resolver matches server-side classification from referrer hint', () => { + const result = vm.runInNewContext( + `(${buildInlineDiscoveryResolverScript('https://www.google.com/search?q=demo')})()`, + { + URL, + URLSearchParams, + location: { + hostname: 'm.tkmind.cn', + search: '', + }, + document: { referrer: '' }, + }, + ); + assert.deepEqual(JSON.parse(JSON.stringify(result)), { + discovery_channel: 'seo', + discovery_source: 'google', + referrer_host: 'google.com', + }); +}); + +test('inline discovery resolver reads propagated query params inside iframe', () => { + const result = vm.runInNewContext( + `(${buildInlineDiscoveryResolverScript('')})()`, + { + URL, + URLSearchParams, + location: { + hostname: 'm.tkmind.cn', + search: '?view=raw&memind_discovery=geo&memind_discovery_source=chatgpt', + }, + document: { referrer: 'https://m.tkmind.cn/u/alice/pages/demo' }, + }, + ); + assert.deepEqual(JSON.parse(JSON.stringify(result)), { + discovery_channel: 'geo', + discovery_source: 'chatgpt', + referrer_host: '', + }); +}); diff --git a/mindspace-analytics.mjs b/mindspace-analytics.mjs index 0ce9f23..64080df 100644 --- a/mindspace-analytics.mjs +++ b/mindspace-analytics.mjs @@ -1,5 +1,7 @@ import crypto from 'node:crypto'; +import { buildInlineDiscoveryResolverScript } from './mindspace-analytics-discovery.mjs'; + const ANALYTICS_MARKER = 'data-memind-analytics="1"'; export function resolveMindSpaceAnalyticsConfig(env = process.env) { @@ -171,6 +173,7 @@ export function injectMindSpaceAnalytics(html, { generatedAt = '', channel = 'h5', viewerIdentity = null, + referrerHint = '', config = resolveMindSpaceAnalyticsConfig(), } = {}) { const source = String(html ?? ''); @@ -187,7 +190,8 @@ export function injectMindSpaceAnalytics(html, { ]; if (config.domains) attrs.push(`data-domains="${config.domains.replaceAll('"', '"')}"`); const viewerJson = viewerIdentity ? jsonForInlineScript(viewerIdentity) : 'null'; - const block = ``; + const resolveDiscovery = buildInlineDiscoveryResolverScript(referrerHint); + const block = ``; if (/<\/head>/i.test(source)) return source.replace(/<\/head>/i, `${block}`); return source.replace(/ { assert.match(out, /data-auto-track="false"/); assert.doesNotMatch(out, /umami\.identify\(d\.owner_id/); assert.match(out, /function identifyViewer\(\)/); - assert.match(out, /identifyViewer\(\);pageview\(\)/); - assert.match(out, /function pageview\(\).*window\.umami\.track\(\)/); + assert.match(out, /Object\.assign\(d,resolveDiscovery\(\)\);pageview\(\)/); + assert.match(out, /function pageview\(\).*window\.umami\.track\(function\(p\)/); assert.doesNotMatch(out, /t\('page_view'\)/); assert.match(out, /page_id/); assert.match(out, /owner_segment/); @@ -140,17 +140,20 @@ test('injects one local same-origin tracker with page dimensions', () => { assert.match(out, /page_form_submit/); assert.match(out, /page_scroll_/); assert.match(out, /page_engaged_10s/); + assert.match(out, /discovery_channel/); + assert.match(out, /resolveDiscovery=/); assert.match(out, /"owner_id":"user-123"/); assert.match(out, /"generated_at":"2026-07-20T01:02:03.000Z"/); assert.equal(injectMindSpaceAnalytics(out, { ownerId: 'user-123', config: { enabled: true, websiteId: 'local-website', idSecret: 'secret' } }), out); }); -test('public visitors skip creator identify and only send a standard page view', () => { +test('public visitors skip creator identify and attach discovery metadata to page views', () => { const out = injectMindSpaceAnalytics('', { ownerId: 'user-123', ownerSegment: 'plan:pro', ownerLabel: '张三', channel: 'h5', + referrerHint: 'https://www.google.com/search?q=demo', config: { enabled: true, websiteId: 'local-website', @@ -164,6 +167,8 @@ test('public visitors skip creator identify and only send a standard page view', const calls = []; vm.runInNewContext(inlineScript, { + URL, + URLSearchParams, window: { umami: { identify: (...args) => calls.push(['identify', ...args]), @@ -178,12 +183,23 @@ test('public visitors skip creator identify and only send a standard page view', title: 'Demo', documentElement: { scrollHeight: 1600 }, addEventListener: () => {}, + referrer: 'https://www.google.com/search?q=demo', + }, + location: { + href: 'https://m.tkmind.cn/MindSpace/demo/public/page.html', + pathname: '/MindSpace/demo/public/page.html', + hostname: 'm.tkmind.cn', + search: '', }, - location: { href: 'https://m.tkmind.cn/MindSpace/demo/public/page.html', pathname: '/MindSpace/demo/public/page.html' }, setTimeout: () => {}, }); - assert.deepEqual(JSON.parse(JSON.stringify(calls)), [['track']]); + assert.equal(calls.length, 1); + assert.equal(calls[0][0], 'track'); + const payload = calls[0][1]({}); + assert.equal(payload.data.discovery_channel, 'seo'); + assert.equal(payload.data.discovery_source, 'google'); + assert.equal(payload.data.referrer_host, 'google.com'); }); test('logged-in public visitors identify themselves without using the page creator id', () => { @@ -208,6 +224,8 @@ test('logged-in public visitors identify themselves without using the page creat const calls = []; vm.runInNewContext(inlineScript, { + URL, + URLSearchParams, window: { umami: { identify: (...args) => calls.push(['identify', ...args]), @@ -223,22 +241,20 @@ test('logged-in public visitors identify themselves without using the page creat documentElement: { scrollHeight: 1600 }, addEventListener: () => {}, }, - location: { href: 'https://m.tkmind.cn/MindSpace/demo/public/page.html', pathname: '/MindSpace/demo/public/page.html' }, + location: { + href: 'https://m.tkmind.cn/MindSpace/demo/public/page.html', + pathname: '/MindSpace/demo/public/page.html', + hostname: 'm.tkmind.cn', + search: '', + }, setTimeout: () => {}, }); - assert.deepEqual(JSON.parse(JSON.stringify(calls)), [ - ['identify', viewerId, { - username: '李四', - memind_page_url: 'https://m.tkmind.cn/MindSpace/demo/public/page.html', - owner_segment: 'plan:free', - plan_type: 'free', - channel: 'public', - surface: 'generated_page', - identity_mode: 'pseudonymous', - }], - ['track'], - ]); + assert.equal(calls.length, 2); + assert.equal(calls[0][0], 'identify'); + assert.equal(calls[0][1], viewerId); + assert.equal(calls[1][0], 'track'); + assert.equal(typeof calls[1][1], 'function'); assert.notEqual(viewerId, 'user-123'); }); diff --git a/mindspace-rybbit.mjs b/mindspace-rybbit.mjs deleted file mode 100644 index a58c14c..0000000 --- a/mindspace-rybbit.mjs +++ /dev/null @@ -1,125 +0,0 @@ -import crypto from 'node:crypto'; - -import { - pseudonymizeAnalyticsId, - resolveAnalyticsOwnerLabel, - resolveAnalyticsOwnerSegment, -} from './mindspace-analytics.mjs'; - -const RYBBIT_MARKER = 'data-memind-rybbit="1"'; - -export function resolveMindSpaceRybbitConfig( - env = process.env, - { idSecret = '' } = {}, -) { - const enabled = String(env.MEMIND_RYBBIT_ENABLED ?? '').toLowerCase() === 'true'; - const siteId = String(env.MEMIND_RYBBIT_SITE_ID ?? '').trim(); - const secret = [ - env.MEMIND_RYBBIT_ID_SECRET, - env.MEMIND_ANALYTICS_ID_SECRET, - idSecret, - ] - .map((value) => String(value ?? '').trim()) - .find(Boolean) ?? ''; - const rybbitUrl = String(env.MEMIND_RYBBIT_URL ?? env.RYBBIT_URL ?? 'https://rybbit.tkmind.cn').trim() - || 'https://rybbit.tkmind.cn'; - return { - enabled: enabled && Boolean(siteId) && Boolean(secret), - siteId, - idSecret: secret, - rybbitUrl: rybbitUrl.replace(/\/$/, ''), - // Same-origin proxy path. Script auto-discovers host by splitting on /script.js, - // so /rybbit/script.js makes events go to /rybbit/track → proxied to /api/track. - scriptPath: String(env.MEMIND_RYBBIT_SCRIPT_PATH ?? '/rybbit/script.js').trim() || '/rybbit/script.js', - hostPath: String(env.MEMIND_RYBBIT_HOST_PATH ?? '/rybbit').trim() || '/rybbit', - }; -} - -export { resolveAnalyticsOwnerLabel, resolveAnalyticsOwnerSegment, pseudonymizeAnalyticsId }; - -function jsonForInlineScript(value) { - return JSON.stringify(value) - .replaceAll('<', '\\u003c') - .replaceAll('>', '\\u003e') - .replaceAll('&', '\\u0026'); -} - -export function sendMindSpaceRybbitEvent({ - config, - eventName, - ownerId, - pageId = '', - publicationId = '', - agentRunId = '', - channel = 'h5', - ownerSegment = 'unknown', - ownerLabel = '未命名用户', - url = '', -} = {}) { - if (!config?.enabled || !config.siteId || !config.idSecret || !eventName) return Promise.resolve(false); - const owner = pseudonymizeAnalyticsId(ownerId, config.idSecret); - if (!owner) return Promise.resolve(false); - const endpoint = `${String(config.rybbitUrl || 'https://rybbit.tkmind.cn').replace(/\/$/, '')}/api/track`; - const username = resolveAnalyticsOwnerLabel({ displayName: ownerLabel }); - const payload = { - site_id: String(config.siteId), - type: 'custom_event', - event_name: String(eventName).slice(0, 256), - pathname: url || '/', - page_title: '', - referrer: '', - hostname: '127.0.0.1', - user_id: owner, - properties: JSON.stringify({ - owner_id: owner, - page_id: String(pageId || ''), - publication_id: String(publicationId || ''), - agent_run_id: String(agentRunId || ''), - channel, - owner_segment: String(ownerSegment || 'unknown'), - username, - owner_label: username, - }), - }; - return fetch(endpoint, { - method: 'POST', - headers: { 'content-type': 'application/json', 'user-agent': 'Memind/local-rybbit' }, - body: JSON.stringify(payload), - signal: AbortSignal.timeout(1500), - }).then((response) => response.ok).catch(() => false); -} - -export function injectMindSpaceRybbit(html, { - ownerId, - pageId = '', - publicationId = '', - ownerSegment = 'unknown', - ownerLabel = '未命名用户', - channel = 'h5', - config = resolveMindSpaceRybbitConfig(), -} = {}) { - const source = String(html ?? ''); - if (!config?.enabled || !config.siteId || !/^\s*(`; - if (/<\/head>/i.test(source)) return source.replace(/<\/head>/i, `${block}`); - return source.replace(/ { - assert.equal(resolveMindSpaceRybbitConfig({ MEMIND_RYBBIT_ENABLED: 'true' }).enabled, false); - assert.equal(resolveMindSpaceRybbitConfig({ - MEMIND_RYBBIT_ENABLED: 'true', - MEMIND_RYBBIT_SITE_ID: '2', - }).enabled, false); - const config = resolveMindSpaceRybbitConfig({ - MEMIND_RYBBIT_ENABLED: 'true', - MEMIND_RYBBIT_SITE_ID: '2', - MEMIND_RYBBIT_ID_SECRET: 'local-secret', - MEMIND_RYBBIT_URL: 'https://rybbit.tkmind.cn', - }); - assert.equal(config.enabled, true); - assert.equal(config.siteId, '2'); - assert.equal(config.rybbitUrl, 'https://rybbit.tkmind.cn'); - assert.equal(config.hostPath, '/rybbit'); - assert.equal(config.scriptPath, '/rybbit/script.js'); -}); - -test('rybbit config can reuse stored analytics secret', () => { - const config = resolveMindSpaceRybbitConfig({ - MEMIND_RYBBIT_ENABLED: 'true', - MEMIND_RYBBIT_SITE_ID: '2', - MEMIND_RYBBIT_ID_SECRET: '', - MEMIND_ANALYTICS_ID_SECRET: '', - }, { - idSecret: 'stored-secret', - }); - assert.equal(config.enabled, true); - assert.equal(config.idSecret, 'stored-secret'); -}); - -test('injects one local same-origin rybbit tracker with page dimensions', () => { - const html = 'Demo

Demo

'; - const out = injectMindSpaceRybbit(html, { - ownerId: 'user-123', - ownerLabel: '张三', - pageId: 'page-1', - publicationId: 'pub-1', - config: { - enabled: true, - siteId: '2', - idSecret: 'secret', - scriptPath: '/rybbit/script.js', - hostPath: '/rybbit', - }, - }); - assert.match(out, /src="\/rybbit\/script\.js"/); - assert.match(out, /data-site-id="2"/); - assert.match(out, /data-memind-rybbit="1"/); - assert.match(out, /r\.identify\(d\.owner_id,/); - assert.match(out, /r\.event\(n,p\)/); - assert.match(out, /page_click/); - assert.match(out, /page_form_submit/); - assert.match(out, /page_scroll_/); - assert.match(out, /page_engaged_10s/); - assert.match(out, /"username":"张三"/); - assert.doesNotMatch(out, /user-123/); - assert.equal(injectMindSpaceRybbit(out, { - ownerId: 'user-123', - config: { enabled: true, siteId: '2', idSecret: 'secret' }, - }), out); -}); - -test('identifies the pseudonymous owner once rybbit is ready', () => { - const out = injectMindSpaceRybbit('', { - ownerId: 'user-123', - ownerSegment: 'plan:pro', - ownerLabel: '张三', - channel: 'h5', - config: { - enabled: true, - siteId: '2', - idSecret: 'secret', - scriptPath: '/rybbit/script.js', - hostPath: '/rybbit', - }, - }); - const inlineScript = out.match(/