memind/docs/deep-search-service.md

TKMind Deep Search Service

TKMind Deep Search is a local-first, independently deployable research orchestrator. MindSearch treats it as a research-http service, in the same way that SearXNG is registered as a search provider.

The standalone runtime uses the built-in node:sqlite module and therefore requires a Node.js release that provides DatabaseSync (Node 22.5 or newer; the local verification used Node 26).

Runtime flow

1. Validate and persist the task.
2. Build a depth-aware research plan.
3. Run bounded, multi-round searches through SearXNG.
4. Canonicalize, deduplicate, and rerank source URLs.
5. Resolve DNS and block private destinations before reading public pages.
6. Extract evidence and remove duplicate claims.
7. Generate a Markdown report with numbered citations.
8. Persist task state, sources, events, report, and research memory in SQLite.

The planner and report writer use an optional model selected from TKMind's Unified Model Center. MindSearch stores only the Provider Key ID and model name. Deep Search calls a secret-protected Portal gateway, and the provider API key never leaves the Portal process. If the selected model is unavailable, deterministic planning and citation-safe extractive reporting keep the service operational.

createDeepSearchEngine also accepts an explicit memorySink callback. It is disabled by default; an in-process TKMind integration can inject memoryV2.write without giving the standalone service an unrestricted outbound memory endpoint. Memory sink failures are fail-open and recorded as task events.

Local run

TKMIND_DEEP_SEARCH_SEARXNG_URL=http://127.0.0.1:8080/search \
TKMIND_DEEP_SEARCH_DB=.deep-search/research.sqlite \
TKMIND_DEEP_SEARCH_SECRET=local-shared-secret \
npm run dev:deep-search

The server listens on 127.0.0.1:20100 by default.

On production host 103, SearXNG is exposed at http://127.0.0.1:20080/search. Goosed containers reach host services through host.docker.internal; Portal rewrites loopback MindSearch service endpoints only when constructing the MCP extension environment.

API

Method	Path	Purpose
GET	/health	Runtime health and task counts
POST	/v1/search	Synchronous normalized search
POST	/v1/research	Start an asynchronous research task
GET	/v1/research	List recent tasks
GET	/v1/research/:id	Get progress, evidence metadata, and report
GET	/v1/research/:id/events	Stream progress as server-sent events
DELETE	/v1/research/:id	Cancel a queued or running task

Start request:

{
  "question": "分析 AI Agent 市场",
  "depth": "standard",
  "userId": "optional-user-id"
}

Start response:

{
  "task_id": "uuid",
  "status": "queued",
  "service": "tkmind-deep-search"
}

Configuration

Variable	Default	Meaning
TKMIND_DEEP_SEARCH_HOST	127.0.0.1	Bind host
TKMIND_DEEP_SEARCH_PORT	20100	HTTP port
TKMIND_DEEP_SEARCH_DB	.deep-search/research.sqlite	SQLite database
TKMIND_DEEP_SEARCH_SEARXNG_URL	http://127.0.0.1:8080/search	Upstream search endpoint
TKMIND_DEEP_SEARCH_SECRET	empty	Optional X-Secret-Key required by non-health routes
TKMIND_DEEP_SEARCH_LLM_GATEWAY_URL	http://127.0.0.1:8081/api/internal/deep-search/llm	Portal LLM gateway
TKMIND_DEEP_SEARCH_LLM_GATEWAY_SECRET	TKMIND_DEEP_SEARCH_SECRET	Portal gateway bearer secret
GITHUB_TOKEN	empty	GitHub code-search token, stored only in the standalone service environment

Isolation and safety

• The default bind address is loopback only.
• An optional service secret protects all non-health endpoints.
• The same secret can authenticate the local service to the Portal LLM gateway.
• MCP forwards the current TKMind user ID as X-User-Id.
• Task status, event, and cancellation routes enforce user ownership when a user ID is present.
• Source reading rejects credentials, loopback, link-local, private, multicast, and private-DNS destinations.
• Redirect targets are resolved and validated again.
• Request bodies, result counts, source content, timeouts, and task depths are bounded.
• GitHub Code Search is proxied by Deep Search so the provider token is never copied into goosed extension environments.

Production runtime on 103

Deep Search is released independently from Portal and memindadm:

npm run build:deep-search-runtime
CONFIRMED_CI_SHA="$(git rev-parse HEAD)" \
  bash scripts/release-deep-search-runtime-prod.sh

The production layout is:

/Users/john/Project/deep-search-runtime/
├── current -> releases/<release-id>/
├── releases/<release-id>/
├── shared/.env
├── data/research.sqlite
├── logs/deep-search.log
└── backups/<release-id>/

The release workflow requires a clean main that exactly matches origin/main, a matching confirmed CI SHA, artifact checksum verification, SQLite online backup, LaunchAgent rollback, host health/search probes, and a Goose-container reachability probe. It never releases or restarts Portal or memindadm.

The installer creates a random service secret on first install. Add a newly issued, least-privilege GitHub token to shared/.env only after revoking any token that has appeared in chat or logs.

MindSearch registration

The built-in disabled service is:

{
  "id": "deep-search",
  "adapter": "research-http",
  "endpoint": "http://127.0.0.1:20100",
  "healthPath": "/health",
  "enabled": false,
  "llmProviderKeyId": "",
  "llmModel": ""
}

In memindadm → MindSearch → TKMind Deep Search, select an optional active Provider configuration and one of its models. The list comes directly from Unified Model Center. The 测试 LLM action checks that exact Provider/model combination before saving the MindSearch configuration. Selecting “不使用 LLM（确定性降级）” leaves both fields empty.

After local verification, enable the service and MindSearch, then select deep-search for the research route. New Goose sessions receive:

• tkmind_research
• tkmind_research_status
• tkmind_research_cancel

Enabling or publishing this service to production is a separate release action and is not performed by local development or tests.