feat: AI-driven threat detection — behavioral analysis, external threat intel, prompt injection signatures #68

New issue

Open

opened 2026-06-17 11:37:38 -07:00 by pyr0ball · 1 comment

pyr0ball commented

2026-06-17 11:37:38 -07:00

Owner

Context: Attackers now use LLMs to craft SQL injection payloads with natural language variation, run credential stuffing with human-like timing jitter, and send prompt injection strings to LLM-backed APIs. Static regex signatures miss these because payloads mutate. The existing cybersec zero-shot scorer and 5-stage diagnose pipeline are the right foundation — this issue extends them.

Plan: circuitforge-plans/turnstone/superpowers/plans/2026-06-17-honeypot-ai-threat-integration.md — Track B

Scope:

patterns/ai-threats.yaml — LLM-variant SQL injection, credential stuffing behavioral, prompt injection, vulnerability enumeration; all tagged ai_threat: true
threat_intel_cache table (ip, source, fetched_at, expires_at, payload, summary_tags)
app/services/threat_intel.py — AbuseIPDB, GreyNoise, Shodan InternetDB clients; async, cached, graceful degradation on missing keys
behavioral_analysis and rate_analysis tables
app/services/behavioral.py — IRI variance analysis, browser fingerprint absence detection, rate-of-fire computation; post-ingest aggregation job
Settings UI: AbuseIPDB key, GreyNoise key fields
Diagnose pipeline Stage 3 (Hypothesizer) + Stage 5 (Synthesizer) extensions: threat intel and behavioral context injected into prompts and output
Incident detail UI: behavioral flags, rate-of-fire data, threat intel feed results
Unit tests for each threat intel client (mocked HTTP) and behavioral analysis functions

Out of scope: Paid threat intel tiers, ML fine-tuning on AI-generated payloads (separate roadmap item), router firewall API adapters.

Acceptance criteria:

Synthetic nginx log with LLM-variant SQL injection matches new signature
Synthetic POST body with "Ignore previous instructions" matches prompt_injection_attempt
After a CRITICAL cybersec event, threat_intel_cache has rows from all three feeds for the source IP
Missing API keys degrade gracefully: feed skipped, settings warning shown, no crash
Simulated AI timing pattern (50-200ms IRI, no gaps >30s, 10+ requests) flagged as ai_timing_pattern
Diagnose Stage 5 output includes "Threat Context" section when behavioral data exists
All existing tests pass

Related: Plan doc — circuitforge-plans/turnstone/superpowers/plans/2026-06-17-honeypot-ai-threat-integration.md | Honeypot ingestion — #67 | Router API backlog (project_turnstone_router_api_backlog)

**Context:** Attackers now use LLMs to craft SQL injection payloads with natural language variation, run credential stuffing with human-like timing jitter, and send prompt injection strings to LLM-backed APIs. Static regex signatures miss these because payloads mutate. The existing cybersec zero-shot scorer and 5-stage diagnose pipeline are the right foundation — this issue extends them. **Plan:** `circuitforge-plans/turnstone/superpowers/plans/2026-06-17-honeypot-ai-threat-integration.md` — Track B **Scope:** - [ ] `patterns/ai-threats.yaml` — LLM-variant SQL injection, credential stuffing behavioral, prompt injection, vulnerability enumeration; all tagged `ai_threat: true` - [ ] `threat_intel_cache` table (ip, source, fetched_at, expires_at, payload, summary_tags) - [ ] `app/services/threat_intel.py` — AbuseIPDB, GreyNoise, Shodan InternetDB clients; async, cached, graceful degradation on missing keys - [ ] `behavioral_analysis` and `rate_analysis` tables - [ ] `app/services/behavioral.py` — IRI variance analysis, browser fingerprint absence detection, rate-of-fire computation; post-ingest aggregation job - [ ] Settings UI: AbuseIPDB key, GreyNoise key fields - [ ] Diagnose pipeline Stage 3 (Hypothesizer) + Stage 5 (Synthesizer) extensions: threat intel and behavioral context injected into prompts and output - [ ] Incident detail UI: behavioral flags, rate-of-fire data, threat intel feed results - [ ] Unit tests for each threat intel client (mocked HTTP) and behavioral analysis functions **Out of scope:** Paid threat intel tiers, ML fine-tuning on AI-generated payloads (separate roadmap item), router firewall API adapters. **Acceptance criteria:** - [ ] Synthetic nginx log with LLM-variant SQL injection matches new signature - [ ] Synthetic POST body with "Ignore previous instructions" matches `prompt_injection_attempt` - [ ] After a CRITICAL cybersec event, `threat_intel_cache` has rows from all three feeds for the source IP - [ ] Missing API keys degrade gracefully: feed skipped, settings warning shown, no crash - [ ] Simulated AI timing pattern (50-200ms IRI, no gaps >30s, 10+ requests) flagged as `ai_timing_pattern` - [ ] Diagnose Stage 5 output includes "Threat Context" section when behavioral data exists - [ ] All existing tests pass **Related:** Plan doc — `circuitforge-plans/turnstone/superpowers/plans/2026-06-17-honeypot-ai-threat-integration.md` | Honeypot ingestion — #67 | Router API backlog (project_turnstone_router_api_backlog)

pyr0ball added the

enhancement

label 2026-06-17 11:37:38 -07:00

pyr0ball commented

2026-06-20 14:04:36 -07:00

Author

Owner

`threat_intel_mode` gating — corporate/enterprise safety

After evaluating Prism-platform integration, external OSINT calls require explicit gating. Every call to AbuseIPDB, VirusTotal, Shodan, or GreyNoise sends the queried IP to a third-party service. For corporate deployments (SOC 2, HIPAA-adjacent, medical device, finance), that is a data governance violation without a signed DPA — even if those services have strong privacy policies.

Adding a threat_intel_mode setting to control this:

Mode	Behavior	Default for
`off`	No external lookups; behavioral analysis and local patterns only	Corporate/enterprise (system default)
`local_only`	Prism sidecar in local-only mode; no external API calls from Prism	Privacy-first homelab
`prism`	Prism sidecar with external feeds enabled (AbuseIPDB, VirusTotal, Shodan)	Prosumer/homelab
`external_feeds`	Direct calls to AbuseIPDB, GreyNoise, Shodan InternetDB with configured API keys	Self-hosted with API keys

System default: off

Prism scope: Prism is appropriate for prosumer/homelab. NOT recommended for corporate deployments — Prism's modules still call the same external APIs, so data still leaves the network. Gated behind threat_intel_mode: prism, off unless explicitly configured.

cf-orch dependency: Threat intel LLM workloads will use the threat_analysis task name when cf-orch #17 lands (security-tuned model routing). Tracked as a dependency.

Plan doc updated: circuitforge-plans/turnstone/superpowers/plans/2026-06-17-honeypot-ai-threat-integration.md — Phase 3 checklist and new threat_intel_mode Configuration subsection.

## `threat_intel_mode` gating — corporate/enterprise safety After evaluating Prism-platform integration, external OSINT calls require explicit gating. Every call to AbuseIPDB, VirusTotal, Shodan, or GreyNoise sends the queried IP to a third-party service. For corporate deployments (SOC 2, HIPAA-adjacent, medical device, finance), that is a data governance violation without a signed DPA — even if those services have strong privacy policies. Adding a `threat_intel_mode` setting to control this: | Mode | Behavior | Default for | |------|----------|-------------| | `off` | No external lookups; behavioral analysis and local patterns only | Corporate/enterprise (**system default**) | | `local_only` | Prism sidecar in local-only mode; no external API calls from Prism | Privacy-first homelab | | `prism` | Prism sidecar with external feeds enabled (AbuseIPDB, VirusTotal, Shodan) | Prosumer/homelab | | `external_feeds` | Direct calls to AbuseIPDB, GreyNoise, Shodan InternetDB with configured API keys | Self-hosted with API keys | **System default:** `off` **Prism scope:** Prism is appropriate for prosumer/homelab. NOT recommended for corporate deployments — Prism's modules still call the same external APIs, so data still leaves the network. Gated behind `threat_intel_mode: prism`, off unless explicitly configured. **cf-orch dependency:** Threat intel LLM workloads will use the `threat_analysis` task name when cf-orch #17 lands (security-tuned model routing). Tracked as a dependency. Plan doc updated: `circuitforge-plans/turnstone/superpowers/plans/2026-06-17-honeypot-ai-threat-integration.md` — Phase 3 checklist and new `threat_intel_mode Configuration` subsection.