diff --git a/documentation/docs/guides/config-files.md b/documentation/docs/guides/config-files.md index 6681fbd2..9db94da2 100644 --- a/documentation/docs/guides/config-files.md +++ b/documentation/docs/guides/config-files.md @@ -47,9 +47,10 @@ The following settings can be configured at the root level of your config.yaml f | `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP endpoint URL for [observability](/docs/guides/environment-variables#opentelemetry-protocol-otlp) | URL (e.g., `http://localhost:4318`) | None | No | | `OTEL_EXPORTER_OTLP_TIMEOUT` | Export timeout in milliseconds for [observability](/docs/guides/environment-variables#opentelemetry-protocol-otlp) | Integer (ms) | 10000 | No | | `SECURITY_PROMPT_ENABLED` | Enable [prompt injection detection](/docs/guides/security/prompt-injection-detection) to identify potentially harmful commands | true/false | false | No | -| `SECURITY_PROMPT_THRESHOLD` | Sensitivity threshold for [prompt injection detection](/docs/guides/security/prompt-injection-detection) (higher = stricter) | Float between 0.01 and 1.0 | 0.7 | No | - - +| `SECURITY_PROMPT_THRESHOLD` | Sensitivity threshold for prompt injection detection (higher = stricter) | Float between 0.01 and 1.0 | 0.8 | No | +| `SECURITY_PROMPT_CLASSIFIER_ENABLED` | Enable ML-based prompt injection detection for advanced threat identification | true/false | false | No | +| `SECURITY_PROMPT_CLASSIFIER_ENDPOINT` | Classification endpoint URL for ML-based prompt injection detection | URL (e.g., "https://api.example.com/classify") | None | No | +| `SECURITY_PROMPT_CLASSIFIER_TOKEN` | Authentication token for `SECURITY_PROMPT_CLASSIFIER_ENDPOINT` | String | None | No | ## Experimental Features diff --git a/documentation/docs/guides/environment-variables.md b/documentation/docs/guides/environment-variables.md index 09e9da4f..53e1a85c 100644 --- a/documentation/docs/guides/environment-variables.md +++ b/documentation/docs/guides/environment-variables.md @@ -295,6 +295,28 @@ These variables control security related features. |----------|---------|---------|---------| | `GOOSE_ALLOWLIST` | Controls which extensions can be loaded | URL for [allowed extensions](/docs/guides/allowlist) list | Unset | | `GOOSE_DISABLE_KEYRING` | Disables the system keyring for secret storage | Set to any value (e.g., "1", "true", "yes") to disable. The actual value doesn't matter, only whether the variable is set. | Unset (keyring enabled) | +| `SECURITY_PROMPT_ENABLED` | Enable [prompt injection detection](/docs/guides/security/prompt-injection-detection) to identify potentially harmful commands | true/false | false | +| `SECURITY_PROMPT_THRESHOLD` | Sensitivity threshold for prompt injection detection (higher = stricter) | Float between 0.01 and 1.0 | 0.8 | +| `SECURITY_PROMPT_CLASSIFIER_ENABLED` | Enable ML-based prompt injection detection for advanced threat identification | true/false | false | +| `SECURITY_PROMPT_CLASSIFIER_ENDPOINT` | Classification endpoint URL for ML-based prompt injection detection | URL (e.g., "https://api.example.com/classify") | Unset | +| `SECURITY_PROMPT_CLASSIFIER_TOKEN` | Authentication token for `SECURITY_PROMPT_CLASSIFIER_ENDPOINT` | String | Unset | + +**Examples** + +```bash +# Enable prompt injection detection with default threshold +export SECURITY_PROMPT_ENABLED=true + +# Enable with custom threshold (stricter) +export SECURITY_PROMPT_ENABLED=true +export SECURITY_PROMPT_THRESHOLD=0.9 + +# Enable ML-based detection with external endpoint +export SECURITY_PROMPT_ENABLED=true +export SECURITY_PROMPT_CLASSIFIER_ENABLED=true +export SECURITY_PROMPT_CLASSIFIER_ENDPOINT="https://your-endpoint.com/classify" +export SECURITY_PROMPT_CLASSIFIER_TOKEN="your-auth-token" +``` :::tip When the keyring is disabled, secrets are stored here: diff --git a/documentation/docs/guides/security/classification-api-spec.md b/documentation/docs/guides/security/classification-api-spec.md index f902cc05..acf2dfa6 100644 --- a/documentation/docs/guides/security/classification-api-spec.md +++ b/documentation/docs/guides/security/classification-api-spec.md @@ -1,19 +1,24 @@ --- +sidebar_position: 2 title: Classification API Specification -unlisted: true +description: API specification for self-hosting ML-based prompt injection detection endpoints. --- -This document defines the API that Goose uses for ML-based prompt injection detection. +This API specification defines the API that goose uses for ML-based [prompt injection detection](/docs/guides/security/prompt-injection-detection). -## Overview +:::info For Self-Hosting Only +This API specification is intended as a reference for users who want to self-host their own model and classification endpoint. -Goose requires a classification endpoint that can analyze text and return a score indicating the likelihood of prompt injection. This API follows the **HuggingFace Inference API format** for text classification, making it compatible with [HuggingFace Inference Endpoints](https://huggingface.co/docs/inference-providers/providers/hf-inference). +If you're using an existing inference service like Hugging Face, you can just configure it in your [prompt injection detection](/docs/guides/security/prompt-injection-detection) settings. +::: + +goose requires a classification endpoint that can analyze text and return a score indicating the likelihood of prompt injection. This API follows the Hugging Face Inference API format for text classification, making it compatible with [Hugging Face Inference Endpoints](https://huggingface.co/docs/inference-providers/providers/hf-inference). ## Security & Privacy Considerations **Warning:** When using ML-based prompt injection detection, all tool call content and user messages sent for classification will be transmitted to the configured endpoint. This may include sensitive or confidential information. -- If you use an external or third-party endpoint (e.g., HuggingFace Inference API, cloud-hosted models), your data will be sent over the network and processed by that service. +- If you use an external or third-party endpoint (e.g., Hugging Face Inference API, cloud-hosted models), your data will be sent over the network and processed by that service. - Consider the sensitivity of your data before enabling ML-based detection or selecting an endpoint. -- For highly sensitive or regulated data, use a self-hosted endpoint, run BERT models locally (see reference implementation) or ensure your chosen provider meets your security and compliance requirements. +- For highly sensitive or regulated data, use a self-hosted endpoint, run BERT models locally or ensure your chosen provider meets your security and compliance requirements. - Review the endpoint's privacy policy and data handling practices. ## Endpoint @@ -22,7 +27,7 @@ Goose requires a classification endpoint that can analyze text and return a scor Analyzes text for prompt injection and returns classification results. -**Note:** The endpoint path can be configured. For HuggingFace, it's typically `/models/{model-id}`. For custom implementations, it can be any path (e.g., `/classify`, `/v1/classify`). +**Note:** The endpoint path can be configured. For Hugging Face, it's typically `/models/{model-id}`. For custom implementations, it can be any path (e.g., `/classify`, `/v1/classify`). #### Request @@ -68,17 +73,17 @@ Analyzes text for prompt injection and returns classification results. - `"SAFE"` or `"LABEL_0"`: Indicates safe/benign text - Implementations SHOULD return results sorted by score (highest first) -**Goose's Usage:** -- Goose looks for the label with the highest score -- If the top label is "INJECTION" (or "LABEL_1"), the score is used as the injection confidence -- If the top label is "SAFE" (or "LABEL_0"), Goose uses `1.0 - score` as the injection confidence +**goose's Usage:** +- goose looks for the label with the highest score +- If the top label is `"INJECTION"` (or `"LABEL_1"`), the score is used as the injection confidence +- If the top label is `"SAFE"` (or `"LABEL_0"`), goose uses `1.0 - score` as the injection confidence #### Status Codes - `200 OK`: Successful classification - `400 Bad Request`: Invalid request format - `500 Internal Server Error`: Classification failed -- `503 Service Unavailable`: Model is loading (HuggingFace specific) +- `503 Service Unavailable`: Model is loading (Hugging Face specific) #### Example diff --git a/documentation/docs/guides/security/index.mdx b/documentation/docs/guides/security/index.mdx index 7b026861..1905516d 100644 --- a/documentation/docs/guides/security/index.mdx +++ b/documentation/docs/guides/security/index.mdx @@ -17,9 +17,14 @@ import styles from '@site/src/components/Card/styles.module.css';