Read and forward telemetry

Telltale emits compact JSONL events for local review, forwarding, dashboards, and alerting. The default scan path writes one canonical JSON object per line to a local file; optional delivery paths can wrap the same event body for a specific sink without changing the event schema.

Event 3.0 remains the current supported and frozen compatibility/output contract. Production Event4 output and CanonicalPayload are inactive. A non-production Event4 4.0 foundation exists in telltale-schema; sinks do not emit it.

Default JSONL Sink

By default, telltale scan uses the user path profile and appends native Event 3.0 telemetry to an operating-system-standard per-user JSONL path:

OS	Default user telemetry path
Linux	$XDG_STATE_HOME/telltale/logs/telltale-events.jsonl or ~/.local/state/telltale/logs/telltale-events.jsonl
macOS	~/Library/Logs/Telltale/telltale-events.jsonl
Windows	%LOCALAPPDATA%\Telltale\Logs\telltale-events.jsonl

cargo run --bin telltale -- scan --once --emit-activity

Use --path-profile system for managed service deployments, or --path-profile project when you intentionally want repo-relative development paths such as logs/telltale-events.jsonl and state/telltale-state.json. Explicit --log-path and --state-path flags still override profile defaults. Service managers can set TELLTALE_LOG_PATH and TELLTALE_STATE_PATH instead of repeating flags. Unknown inherited variables are ignored.

Use --dry-run when validating fixtures or command behavior without writing events:

cargo run --bin telltale -- scan --once --dry-run --no-local-config --root tests/fixtures/session_stores

The JSONL sink is the stable interchange point. Each line is a complete event that follows schemas/event.schema.json.

Event 3.0 consumer boundary

The lower-level telltale-schema crate exposes Event3Record for external consumers that already have one terminal Event 3.0 JSON object. It parses the exact JSON object, requires schema_version: "3.0", validates the frozen strict schema, applies the existing identity/family/risk semantics, and returns typed common and family projections. It does not deserialize native Event, read session stores, add provenance, reorder events, deduplicate, or execute the response guidance. Errors are stable classified codes and never include raw JSON or input-derived values. The current and historical Event 3.0 schema artifacts remain byte-identical at SHA-256 9014a15c010bc613b4deb7e0195ec56f702e9e950fb13a12c6937a733e38d754.

Native Event is the trusted producer model. Event3Record is the consumer view of already terminal-safe bytes; it is not a producer, provenance record, session-store reader, or action executor.

Runtime-neutral local journal polling

Embedding hosts that need a typed local journal view can opt into telltale_core::LocalEventFeed. It is below the CLI and uses the same public path-profile resolution semantics through LocalEventFeedConfig::for_profile. The host calls poll on its own timer or event loop; no background runtime, watcher, busy-loop, sidecar lock, persistent cursor, database, or outbox access is involved.

use telltale_core::{LocalEventFeed, LocalEventFeedConfig, StartupMode};
use telltale_sources::paths::PathProfile;

let config = LocalEventFeedConfig::for_profile(
    PathProfile::User,
    None,
    StartupMode::Beginning,
);
let mut feed = LocalEventFeed::new(config)?;
// The host chooses when to poll; this is not a watcher or a scheduler.
let batch = feed.poll()?;
assert!(batch.bytes_read <= 1024 * 1024);
# Ok::<(), Box<dyn std::error::Error>>(())

Beginning reads discoverable generations oldest-to-newest. End skips historical complete lines while retaining the active partial frame, and Recent is bounded by both event count and bytes; it never searches unbounded history for a requested count. Only LF-complete records are parsed through the existing Event3Record consumer. Malformed, oversized, partial, replaced, truncated, unsafe, and uncertain generations appear as bounded static-code notices with no paths, OS text, raw JSON, or evidence. Deduplication retains only a bounded FIFO map of public event IDs and exact-line SHA-256 values; identical replay is suppressed, conflicting bytes fail closed, and eviction or restart ends the suppression guarantee. Physical order is the journal order, not timestamp order. The feed recognizes only Telltale’s built-in rotation namespace; external rotation is surfaced as bounded uncertainty. The private durable outbox remains a separate delivery mechanism and is not a query API.

For each result, FeedBatch.bytes_read is the actual journal payload bytes physically read/processed during that poll. It is not a cursor substitute; a drain-only poll that emits pending records may report zero. caught_up means the currently discoverable eligible bytes and records were drained consistently with the startup floor and a stable observation. It is false when a budget, bound, partial frame, unresolved condition, or generation gap remains. It does not mean no future events, complete sessions, current event time, or that excluded Recent history does not exist.

Producer / Detector Provenance Manifest

The separate telltale config provenance command prints one compact ProducerProvenanceManifestV1 JSON object. It identifies the effective compiled Rule v1 output semantics, its active rule count and separately represented opaque policy label, the exact Event 3.0 contract, risk and operational-alert thresholds, allowlist suppression semantics, and the audited activity, session-summary, baseline, process-chain, and install-inventory switches. The CLI uses the same resolve_scan_config and rule resolver as scan. The public Rust assembler is assemble_producer_provenance_manifest (also available through Pipeline::producer_provenance_manifest) in telltale-core; it takes an already effective CompiledRuleSet and resolved values and does not resolve filesystem paths, managed tiers, policy files, or allowlist paths.

PipelineBuilder is an in-memory convenience for bundled/explicit rule and policy documents, not a replacement for the CLI path resolver. LocalEventFeed uses the public path-profile resolver for journal selection and remains separate from provenance assembly.

The manifest has three distinct SHA-256 identities. Rule and suppression fingerprints identify their own effective content; the full producer_manifest_id hashes the canonical manifest payload without that ID. All are rendered as sha256: plus 64 lowercase hexadecimal characters.

Three independent version labels apply to Rule v1 provenance:

• the producer manifest schema remains ProducerProvenanceManifestV1, version 1;
• rules.canonicalization is currently rule-v1-compiled-compatibility-v2;
• the rule fingerprint preimage uses domain telltale:producer-rule-v1-fingerprint:v2.

The current Rule v1 identity is derived from compiled output semantics, including target exclusions. Policy names, source YAML formatting, line endings, paths, and descriptive rule title/description or rule/modifier false-positive guidance do not affect it. Rule and modifier order remain identity-bearing because evaluation emits ordered rule IDs and evidence. Suppression provenance declares canonicalization suppression-v1-effective-v1; suppression criteria are included only in the domain-separated digest preimage. The manifest exposes state, count, and fingerprint, never suppression names or criteria.

Historical well-formed manifest-schema-v1 documents using rule canonicalization rule-v1-compiled-compatibility-v1 remain accepted; newly produced manifests use the current canonicalization and fingerprint domain above. The other fixed digest domains are telltale:producer-suppression-v1-fingerprint:v1 and telltale:producer-provenance-manifest-v1-id:v1, each followed by a NUL separator before its canonical JSON payload.

This is a content-integrity comparison aid, not authentication, attestation, encryption, telemetry, a scan/run/host/tenant identity, or a claim that any historical Event 3.0 record was produced under the manifest. Digests can have guessable preimages for low-entropy configuration. The command performs no event, state, sidecar, journal, outbox, or manifest-file writes. Event 3.0 remains unchanged and has no manifest linkage.

The embedded process-chain rules, source-adapter definitions, MCP detector definitions, timeline/correlation/baseline algorithms, and Event 3.0 schema are immutable binary or frozen-contract assets for this v1 manifest. They are identified by the normal Telltale package/build identity rather than an open-ended asset graph; the process-chain enablement switch is represented. Scan-local roots, source files, MCP/session observations, state, and delivery destinations are deliberately excluded.

Semver alone does not distinguish same-version custom binaries, so package or artifact identity is still required alongside the manifest when immutable embedded detector assets matter. Client-scoped scans may suppress install-inventory observations; client and source selection remain intentionally outside the manifest, so it is not per-scan or per-event proof.

The identity fields have deliberately limited meanings:

• event_id is the Event 3.0 record identity. It is not a session, source, delivery-attempt, host, or device identity.
• session_id is semantic agent-session correlation when present. It is not globally unique across machines.
• source_path_hash is privacy-safe source correlation when present. It is not the path, is not reversible, and is not host identity.
• timeline_anchors.entry_index locates the corresponding normalized session timeline entry. It is not a canonical observation ID and does not guarantee immutable source-history storage. If the source later disappears, Event 3.0 remains authoritative while rich session context may be unavailable.

Event 3.0 has no public scan or run identity; health counters are operational observations, not scan IDs. Host, device, tenant, and collector metadata are outside Event 3.0.

The timing fields retain their current coarse meanings: timestamp is valid source time normalized to UTC milliseconds when available, otherwise local observation time with time_source, time_confidence, and an optional time_override_reason; event_time retains source/derived event time; and observed_at and ingested_at are local scan/ingestion timestamps. Identity is carried by event_id, session_id, source-derived hash, and optional timeline anchors. Taxonomy, risk contributions, redacted/hashed evidence, and response fields retain their Event 3.0 meanings; response is guidance, not an executed action. Health counters are operational observations and are not scan identity.

JSONL remains the complete-line durable first-write boundary, with at-least-once delivery. Receivers deduplicate by event_id: identical canonical-byte replay is valid, while conflicting canonical content for the same ID is an integrity failure. File order is not universal event ordering. The private outbox is not the public consumer query API, and future observation lifecycle semantics belong to Event4 rather than this contract.

Scan Diagnostics

Every scan also prints one JSON summary to stdout. This local diagnostic is not a native Event 3.0 payload and is not appended to JSONL or wrapped for HEC. In addition to the existing delivery and event totals, diagnostic sections explain an otherwise ambiguous zero-detection result without exposing session content:

• source_processing reports selected sources, successful parses, empty and failed parses, the number of normalized records, and fixed counts for user, assistant, tool-call, tool-result, session-metadata, and other records. evaluated_window_count counts evaluated windows; it exceeds the parsed source count when a backlog was drained in several windows. incomplete_window_source_count counts sources whose last evaluated window was still cut when the scan stopped draining; the next scan continues the backlog. A --history all scan adds "history": "all". quarantined_unit_count counts native units excluded for an oversized or invalid value or for exceeding evaluation capacity, once each even when an overlapping window re-reads them, and quarantined_units lists up to 1,000 of them as {source, session_id, native_id, source_sequence, code, bound_category, bound_dimension}, where code is unbounded_value, invalid_native_contribution, invalid_body (a value the canonical body cannot hold, such as a requested tool name with a line break), oversized_native_cell or evaluation_capacity_exceeded. These are coordinates only, never content. Quarantine makes selected_source_coverage partial. Development builds after RC1 additionally report evaluation_complete_source_count, visibility_limited_source_count, accounting_complete_source_count, and accounting_partial_source_count. Evaluation completion and whole-source accounting coverage are independent: successful acquisition may have limited evaluation visibility, and OpenCode accounting remains partial even when evaluation completes normally. selected_source_coverage is no_sources, partial (any selected source rejected, visibility-limited, or partial-accounting), or complete otherwise. This describes only selected acquired inputs under the active rules, not exhaustive discovery, immutable files, every source capability, or absence of threats. Empty successful inputs do not prove useful detection coverage. failures groups rejected sources by typed processing stage, acquisition code, and optional canonical bound category/dimension, with source_count. These aggregates contain no paths, identities, content, measured sizes, or raw error text. processing_code preserves a closed, content-free processing error reason when available; acquisition failures have a null processing code, while later-stage failures without acquisition details have null acquisition and bound fields. Event 3.0 scanner error codes are unchanged. Existing parse counters retain their meaning: parse_error_source_count includes any source-processing rejection.
• detection_flow reports effective detection candidates before state deduplication, matched rule-ID references, allowlist-marked candidates, state-deduplicated candidates, and emitted detections. Effective candidates equal emitted plus state-deduplicated detections.
• runtime reports the package version, embedded build hash, and best-effort executable observation. The executable path is represented by a path hash and its bytes by a streaming SHA-256 digest; unavailable executable observations degrade to a bounded status without aborting the scan.
• effective_configuration reports hashed path provenance for local config, log/state resolution, rules and overrides, policy/allowlist selection, project config, and startup output selection. Rule sources and replacement winners are identity-hashed. rules.community is enabled, installed_only, or disabled, and rules.packages lists each active rule package’s ID, version, content SHA-256, origin, and any replaced embedded version. Output projections expose only sink name/type, selection/origin, JSONL destination hashes, secret/TLS posture, and delivery posture; endpoints, credentials, credential references, hosts, indices, sourcetypes, and CA paths are excluded. settings lists each operational setting with its effective value, default, origin (default, settings_file, environment, or cli), and source: the winning file’s path hash, the variable name, or the flag.
• source_discovery reports privacy-safe discovery accounting. A full scan uses basis=current_full_scan and performs checked discovery for that scan. A targeted watch scan uses the retained basis=watch_source_index_snapshot and sets performed_for_current_scan=false; a full watch reconciliation refreshes the current discovery result. returned_source_count is before selection, while operational_source_count is after client filtering and max_sources for a full scan (or is the canonical path-keyed watch index size). Selection order is unchanged.
• Checked discovery reports only the first error category (invalid_root, traversal, or other). On that error, the CLI uses the best-effort partial result and marks best_effort_fallback_used; this is first-error status, not a total failure count. Project configuration reports the mode (default_roots, configured_documents, or none) and attempted, successful, failed-document, and loaded-project counts. A failed configured document contributes no projects.
• diagnostic_warnings contains constant-only {code, classification, basis} observations. Failure codes are project_config_load_failed, source_discovery_degraded, and source_parse_error_observed. Suspicious zero codes are no_sources_selected, selected_sources_produced_no_records, all_selected_sources_parse_failed_or_empty, no_tool_records_observed, and no_effective_detection_candidates. Ok([]) is a parse success; empty sources do not imply parse failure when another source is productive, and repeated state-deduplicated positives do not imply zero candidates. These are observations only: they are not health verdicts, security alerts, Event 3.0 fields, sink payloads, persisted state, or exit-status changes. Discovery and project-load diagnostics never include roots, paths, source IDs, loader errors, or operating-system error text. Post-RC1 development builds also emit source_coverage_partial with classification=coverage_limitation and basis=source_processing when the selected-source coverage is partial. Exit zero still means the scan command completed, not complete detection coverage: inspect this warning, processing counts, failure groups, and discovery degradation before accepting a canary.

The existing top-level log_path field remains raw for compatibility and is a local diagnostic caveat. Newly added path fields use hashes. These diagnostic sections are stdout-only: they are not Event 3.0 fields and are not written to JSONL or HEC.

Allowlisting marks a detection informational and does not necessarily prevent emission. Scanner-error events remain visible through the existing scan totals but do not inflate the detection-only flow. With an active rule policy, the stdout-only summary reports policy_match_accounting with status=available, the source/session-scoped pre-policy candidate count, fully filtered candidate count, and filtered matched rule-ID reference count. These counts are computed from the in-memory post-override rule set and the existing session projection; they are not Event 3.0 fields and are not written to JSONL or HEC. If diagnostic compilation, evaluation, snapshot integrity, monotonicity, or aggregation fails, status is unavailable and all three counters are null. Without a configured policy, status remains not_applicable and no diagnostic matching work is performed.

Event Families

Common event types include:

• activity: redacted per-session activity summaries.
• activity with check_name=install_inventory: metadata-only installed-agent inventory collected on a cadence. Evidence records agent, confidence, and kind:name signals. Path material stays inside the hashed agent JSON, not as readable evidence text, and the event does not include raw transcript or session contents.
• detection: rule matches, risk scores, categories, and response guidance.
• session_risk_summary: optional per-session rollups from already-redacted activity and detection events.
• health: source-discovery and scanner health status.
• scanner_error: source-acquisition or scan errors that should be visible to operators.
• operational_alert: operator-facing threshold and delivery alerts, including alert_type=sink_delivery_failure when a configured remote sink (Splunk HEC, Elastic bulk) could not be delivered to after retries, and alert_type=source_parse_outage (check name operational_alert) once per client when a full scan selected sources for that client and every one failed processing (acquisition, parsing, evaluation or a later stage). That client is unmonitored, so the alert is a warning; each per-source scanner_error stays informational, as Event 3.0 fixes that family’s severity. Targeted (watch) scans see only changed sources and do not raise it. State records the outage, so an unchanged outage alerts once. A full scan in which one of the client’s sources processes successfully clears it, so a recurrence alerts again. --backfill re-emits all events, so each full backfill scan repeats the alert while the outage lasts.
• correlation: cross-session patterns built from emitted telemetry.
• process_chain: process-relationship detections with risk attribution and a terminally sanitized process context.

The terminal Event boundary preserves established lowercase-hex source_path_hash values and canonical ATT&CK IDs (T1234 or T1234.001). Native output always carries the trusted compile-time Telltale package version; public credential-bearing SemVer mutations are replaced with that version. Known source_counts keys use the canonical <client>.<source-kind> form. Non-canonical public values are deterministically emitted as a SHA-256 source hash, a mitre:<sha256> identifier, or a source_count:<sha256> map key; collision suffixes retain each source-count value without merging. These corrections intentionally change emitted bytes for noncanonical public inputs, including mutated native version metadata, but not canonical constructor output, event identity, or delivery mechanisms. Historical JSONL/Elastic export keeps safe historical version metadata.

Event 3.0 timing

Native Event 3.0 resolves timestamp from a valid source timestamp when one is available, normalizing it to UTC with millisecond precision. A missing or unparseable source timestamp, or a source timestamp more than five minutes in the future, falls back to the local observation time and records the reason in time_override_reason; time_source and time_confidence identify the resulting posture. event_time retains the source/derived event time when available. observed_at and ingested_at are local scan/ingestion timestamps, not source event time. These are the frozen coarse Event 3.0 semantics; a future observation lifecycle belongs to a separately reviewed Event 4.0 architecture.

Event 3.0 freeze policy

Event 3.0 is the stable external compatibility contract for the scanner and deterministic detection layer, established and frozen in v0.6 and still current for v0.7. After this freeze, only security, privacy, correctness, documentation, and compatibility fixes may change its implementation or evidence. New runtime observation lifecycle, gateway telemetry, decisions, actions, approvals, or runtime/browser/OS context belong to Event 4.0 or future architecture and must not be added implicitly to Event 3.0.

Sinks and Delivery Failures

Development hardening (not release qualification): non-loopback HTTP endpoints now require tls.allow_insecure_http: true in outputs configuration. This lab-only choice emits a warning: credentials and telemetry are plaintext. Use HTTPS normally; certificate verification remains enabled by default. Only literal loopback IPs may use HTTP without this opt-in (localhost is not an IP exemption). Credential-bearing CLI HEC endpoints use the same validation and require HTTPS outside loopback; use outputs configuration for a lab opt-in. Endpoint user information and fragments are rejected. HTTP redirects are never followed, including same-origin redirects; configure the final collector URL. Redirect responses block durable rows for operator correction and explicit release rather than declaring them poison. Plaintext HTTP requests bypass ambient proxies so loopback-only HTTP cannot escape through a proxy. HTTPS retains environment proxy support and end-to-end certificate verification.

Beyond the default local JSONL sink, scans can deliver the same events to Splunk HEC and the Elasticsearch Bulk API, configured centrally through outputs.d YAML files under the standard config roots (/etc/telltale, the user config dir, or --config-dir); an annotated example ships in config/examples/telltale-outputs.yaml. Remote delivery uses bounded in-memory retries where applicable; on failure the scan continues and emits an operational_alert event with alert_type=sink_delivery_failure (check name sink_delivery) to the remaining healthy sinks. The failed sink never receives its own failure alert, so delivery problems cannot cascade. These alerts bypass duplicate suppression and are not counted in the health event’s emitted_count; the scan’s stdout summary also lists failures under sink_failures. A failure writing the local JSONL sink itself still aborts the scan, because that file is the durable record. Local JSONL serializes the complete batch before opening the file, holds a permanent sidecar lock across rotation and append, flushes and synchronizes the file before returning, and therefore before scanner state commit. Empty batches return before lock, rotation, or file creation. A trailing partial record is refused rather than silently concatenated or discarded; repair or replace that file before retrying. On Unix, file creation and rotation also sync the parent directory. On Windows, writable file handles are flushed and renames use write-through semantics; parent-directory durability is not asserted.

Delivery Guarantees

Local JSONL is the durable first write and bounded local handoff record. Newly emitted noncanonical source-hash, MITRE, and source-count-key values, plus mutated native version metadata, are corrected before that first write. In durable mode, SQLite reconciles and replays the exact terminal-sanitized canonical bytes written to JSONL; it is not a second canonical payload representation and does not reserialize persisted historical bytes during replay. JSONL is the ingress/recovery source until reconciliation, not an indefinite system of record or a file retained until downstream ACK. Direct Splunk HEC and Elastic HTTP delivery are best-effort sinks with bounded in-memory retries, not queues. A process exit or restart discards pending direct delivery attempts, while the JSONL record remains available to an external shipper when JSONL is enabled. Uncertain responses and retries can produce duplicate delivery.

Splunk HEC delivery requires an HTTP 2xx response containing a JSON object with integer code: 0; HTTP success alone is not acceptance. Known transient HEC codes (8–9, 17–20, 23–27) share the bounded retry budget with transport failures, 429, and 5xx. Token/authentication codes (1–4, 21–22) block durable rows; known payload rejections (5–7, 10–16) dead-letter them. Unknown codes and missing or malformed responses block durable rows for operator investigation, rather than acknowledging or declaring the event poison. Response text is never included in delivery diagnostics. This checks HEC request acceptance, not indexer acknowledgment.

Durable rollback boundary (Issue 69): HEC response validation adds persisted sink_application_retryable and sink_response_blocked error classes. This build reads existing outboxes, but builds predating this change reject rows containing either new class when decoding delivery state, even though the SQLite schema version is unchanged. A binary-only downgrade is not supported once these classes have been written. Best-effort sinks have no outbox migration.

Before upgrading a durable deployment, stop all scanners, dispatchers, and schedules and take a verified, coordinated snapshot of the old binary, configuration, scanner state, canonical JSONL with rotations, and outbox with its SQLite/coordination sidecars. See the inventory and quiescing guidance in controlled deployment. For rollback, stop writers again, quarantine the complete post-upgrade state set, and restore the matching pre-upgrade snapshot together with the old binary; never mix a newer outbox or scanner cursor with the restored set. Retain the quarantined data for explicit reconciliation of post-snapshot events; restoring the snapshot alone does not deliver those events, and replay can duplicate already accepted events. Without a verified pre-upgrade snapshot, keep the compatible binary and repair forward. There is no outbox downgrade conversion: do not rename error classes, delete rows, or silently reset the queue.

Remote-only output is valid but has no built-in persistent replay. After retry exhaustion, or process exit/restart while the endpoint is unavailable, events may be lost. Elastic uses _id = event_id, so a redelivery overwrites the same document rather than duplicating it, but this is not an exactly-once guarantee. Best-effort Elastic item-level Bulk API errors are observable failures and are not retried by the sink. Durable single-event sends instead require exactly one index item with the matching _id, a consistent errors flag, and status/result 200/updated or 201/created before acknowledgment. Item statuses 408, 429, and 5xx use the persisted retry budget; 401/403 block for operator action; confirmed 400/413/422 item rejections are permanent. Malformed, mismatched, inconsistent, or otherwise unrecognized outcomes block rather than acknowledge or declare the event poison. Diagnostics exclude response content. These are at-least-once-oriented handoff semantics, not an exactly-once guarantee: crashes around delivery, retries, and external rotation can still require downstream deduplication. Durable delivery is at-least-once; receivers should deduplicate by event_id within their deployment context. The same ID with different canonical bytes is an integrity failure, not a new event.

These local durability statements cover flushed file contents and the platform-specific parent-directory behavior above; they are not a claim of crash-proof delivery across every filesystem or operating-system failure.

Durable storage boundaries

Durable admission is serialized for cooperating Telltale writers by the private outbox sidecar lock. A ready-work drain runs before the capacity gate, so an acknowledged pending row can release capacity without a restart; blocked and retry-delayed rows remain capacity-consuming until their normal state transition or an explicit blocked-delivery release. The admission lock covers recovery, ready-work selection, transport attempts and result commits, capacity inspection, JSONL append, and reconciliation. Standalone dispatch uses the same owner across selection, transport, and result commit, even with distinct scanner-state paths; admission predrain reuses its held owner without recursive locking. Contention fails immediately with structured DurableStorage: the losing invocation does not send, update delivery state, or prune generations. This is not a distributed lock service and does not prevent at-least-once crash replay duplicates.

Each dispatch invocation now selects at most 16 rows per sink and stops selecting for that sink after 100 ms. An in-flight request may finish up to its configured transport timeout; this is not a 100 ms cancellation deadline. Each configured sink receives its own share, so a large backlog in one cannot exhaust another’s share. Unselected rows remain durable for later invocations. A full queue can still reject admission if the bounded drain does not free enough capacity. Total network time scales with configured sink count and timeouts, not backlog length. Reconciliation and storage costs remain separate limitations below.

Persistent durable delivery currently has one explicit platform boundary: Windows durable private storage is rejected fail-closed before outbox or sidecar creation/opening/inspection/mutation, before a prospective canonical JSONL append, and before scanner-state progress. The bounded diagnostic is persistent durable-delivery private storage is not supported on Windows yet. This does not make Telltale or existing Windows best-effort operation unsupported, and a requested durable policy is never silently downgraded to best-effort. Cross-platform API/cfg compilation and deterministic policy tests do not establish native Windows durable-runtime support. Network filesystems are unsupported for durable storage.

The durable guarantee assumes private local storage controlled by a cooperating Telltale process/user boundary. Hostile same-OS-principal actors and privileged/root/admin actors are outside the durable-storage threat model. Path, advisory-lock, and stable-identity checks are useful integrity defenses for cooperating writers, but they are not an atomic pathname-to-opened-SQLite object binding against those excluded actors. A stronger opened-object-aware SQLite/VFS design would be needed for that defense and is not implemented.

Three storage limits remain explicit:

1. Reconciliation memory: the bounded unread-capacity inspection is not a bound on replay ingest. Reconciliation currently snapshots complete discovered JSONL generations and builds their payload plan in memory. Keep generations within available process resources; arbitrary-size or streaming reconciliation is not claimed, and an oversized generation is not permission to advance an unverified cursor.
2. Terminal payload retention: SQLite retains canonical payload bytes for Acked and Dead history. Terminal rows are excluded from pending queue limits, and there is no automatic purge or compaction guarantee, so private outbox storage may grow beyond configured pending limits.
3. External writers: verified rotation deletion coordinates only with writers that honor Telltale’s JSONL sidecar lock. A non-cooperating local writer can race the final path check and unlink; external rotation/writers in the managed generation namespace and network filesystems are unsupported durable configurations. The verification and fail-safe retention checks reduce this risk but cannot provide atomic deletion against an actor that ignores the lock.

Future local collector boundary

The public out-of-process extension boundary is exactly terminal/emittable Event 3.0 -> durable JSONL -> future generic vendor-neutral local collector transport. This transport is not implemented here. Event 3.0 is independent of transport. Sink identity, transport, delivery policy, and persistence role remain separate concerns: a future local IPC transport may be BestEffort, and a durable transport need not be HTTP. Future transport work should reuse generic structured delivery classification and outbox dispatch rather than require a foundational sink refactor.

The future local collector implementation and protocol are deferred. There is no local collector protocol or public plugin ABI, and durable delivery does not change Event 3.0. Adopter-specific integrations remain outside the core contract; JSONL-only is not the final adoption architecture.

Durable queue health

When durable downstream delivery is configured, scan summaries and the effective_configuration.outputs.delivery projection include durable_queue_health. telltale status --outbox-path PATH exposes the same health without reading event payloads. The health object is metadata-only and has this shape:

{
  "mode": "durable",
  "sinks": {
    "sink-id": {
      "pending_depth": 0,
      "pending_bytes": 0,
      "oldest_pending_age_seconds": null,
      "dead_count": 0,
      "last_success_at": null,
      "last_error": null
    }
  }
}

Values are independent for each stable sink identity. pending_depth and pending_bytes include pending rows, including retry-delayed rows, and blocked rows awaiting operator action. oldest_pending_age_seconds is based on the oldest event creation timestamp among those rows. dead_count counts permanent/dead rows and is not part of pending depth; acked rows are excluded from both. last_success_at and last_error.at are persisted Unix milliseconds from sink state transitions. last_error contains at, the structured error class, and, when applicable, an HTTP status; it never contains response bodies or event bytes.

mode is not_configured when no durable outbox is selected and unavailable when the bounded read-only lookup cannot validate or open the outbox. Health lookup uses SQLite’s bounded busy timeout and fails closed for locked, corrupt, or inaccessible state. It does not repair state or authorize delivery. Queue health is not a cross-sink total: independent sinks can have different depth, age, dead, success, and error values. These metrics describe local durable replay state only and do not prove receiver acceptance or exactly-once delivery.

If the outbox lookup succeeds but the optional status journal is missing, empty, lacks native health, or cannot be strictly read, status --outbox-path still prints the valid queue health and exits successfully. The top-level status is unavailable, scanner-health fields and detection_count are null, and an additive journal_status object reports availability: unavailable with the bounded reason no_native_health or journal_read_failed. No raw reader diagnostic is included; queue availability does not imply scanner health. Native-health and historical-only output shapes are unchanged. Without a valid outbox lookup, journal failures retain their existing error/exit behavior. Status inspection does not create storage or parents, reconcile, prune, or change delivery state.

Releasing blocked deliveries

After repairing a sink’s credentials or endpoint, an operator can explicitly release all of that sink’s blocked rows for retry (including ambiguous HEC responses, after investigating the endpoint):

telltale delivery retry-blocked \
  --outbox-path /var/lib/telltale/outbox/outbox.sqlite \
  --sink corp-splunk

The command updates only blocked rows for the exact sink identity, preserves attempt and error history, and schedules them immediately. It prints only released_blocked=<count>; a valid sink with no blocked rows reports zero.

telltale config validate reports the outputs.d delivery posture, while each scan summary reports its effective posture and delivery status, including whether Telltale’s built_in_persistent_replay field is true only when durable policy activates Telltale’s built-in replay/outbox; otherwise it is false. Scan CLI HEC overlays can change the runtime posture; they are not part of the config validate report. These reports do not create remote history or imply external-shipper replay capability.

Enable optional activity and session summary events when dashboards need more than detection-only output:

cargo run --bin telltale -- scan --once --emit-activity --emit-session-risk-summary

Installed-agent inventory is separate from session-store discovery. It checks metadata such as executable names, package roots, VS Code-style extension IDs, and globalStorage presence to identify installed tooling even when no sessions exist. By default, scans collect and emit this inventory at most once every 24 hours according to the state file. Tune the cadence with --install-inventory-interval-seconds or TELLTALE_INSTALL_INVENTORY_INTERVAL_SECONDS; use 0 to collect every scan, or --install-inventory-disabled to suppress inventory for a run.

Privacy Boundary

Telemetry should be useful without becoming a transcript dump. Telltale emits redacted excerpts, evidence hashes, rule IDs, risk scores, source metadata, and bounded context by default. It should not emit raw secrets, full auth files, raw private keys, complete .env values, or full session bodies.

See privacy-model.md for the evidence classes and redaction rules that govern emitted event content.

Public Examples And Release Evidence

Public documentation, release notes, and support evidence should follow the same privacy boundary as emitted telemetry: use synthetic fixtures or already-redacted event output, not live session stores, raw transcripts, local telemetry logs, scanner state, workstation paths, SIEM endpoint details, or credential-like values. Keep host-specific validation observations in local-only notes and recreate public examples with fixture-backed commands when evidence is needed.

Forwarding To SIEMs

Forward the active JSONL file for the path profile your deployment actually uses. A safe starter pattern is:

1. Write events locally with the default path profile, TELLTALE_LOG_PATH, or an explicit --log-path.
2. Validate the event shape against the schema.
3. Configure the shipper to read only that active JSONL event path.
4. Keep human-readable diagnostics, scanner state, credentials, and raw agent session stores outside the forwarded telemetry path.

For default workstation installs, that active file is the user profile path, such as ~/.local/state/telltale/logs/telltale-events.jsonl on Linux. For managed service deployments, run scans with --path-profile system or explicit TELLTALE_LOG_PATH/TELLTALE_STATE_PATH values and point shippers at the system path, such as /var/log/telltale/telltale-events.jsonl on Linux. The project profile uses logs/telltale-events.jsonl; use an explicit path when an operator intentionally selects another destination.

Use explicit file paths instead of broad log directory monitors so diagnostic logs or source logs do not get indexed as Telltale events. For Splunk Universal Forwarder deployments, install timestamp and JSON parsing props on the tier that performs parsing (the indexer or a heavy forwarder; a UF with indexed JSON extractions may also apply them before forwarding). The telltale:json timestamp parser expects Telltale’s canonical timestamp field to remain the first JSON field.

For managed deployments, prefer OS-native rotation first. Keep the active file name stable, such as /var/log/telltale/telltale-events.jsonl on Linux, and configure logrotate, newsyslog, a Windows scheduled task, or your endpoint collector to rotate completed files without changing the active shipper target. The Linux starter example is config/examples/telltale-logrotate.

Built-In Rotation

For user-profile installs and cross-platform consistency, Telltale includes built-in size-based rotation. No OS-specific tooling (logrotate, newsyslog, Scheduled Tasks) is required.

When the active JSONL file exceeds the configured max size, Telltale renames it to a date-stamped rotated file and starts a fresh active file. The active file name is always stable (telltale-events.jsonl) so shippers can monitor a single path. Rotated files are named telltale-events-YYYY-MM-DD.jsonl, with a counter suffix (.1, .2) for same-day rotations. Files beyond the keep count are deleted oldest-first.

Defaults:

Setting	Default	Env var
Max size	100 MB (104_857_600 bytes)	TELLTALE_LOG_ROTATE_MAX_SIZE
Keep count	5	TELLTALE_LOG_ROTATE_KEEP

CLI flags override env vars:

telltale scan --once --emit-activity --log-rotate-max-size 52428800 --log-rotate-keep 10

To disable built-in rotation (for system-profile deployments that use OS-native logrotate instead):

telltale scan --once --emit-activity --log-rotate-disabled

The local JSONL file is a bounded handoff record, not the canonical store. Once events are shipped to a SIEM or central log platform, that platform is canonical. Rotation keeps local retention bounded without external tooling.

Optional Export And Sink Paths

Each delivery path uses the same terminal canonical event payload:

• telltale export --format jsonl reads existing JSONL telemetry.
• telltale export --format elastic-bulk writes Elasticsearch Bulk API pairs.

Development export admission accepts regular files only, with inclusive limits of 8 MiB per physical frame (including terminators), 1 GiB per journal, 1,000,000 distinct event IDs, and 64 MiB of ID text. These resource limits apply equally to historical schemas 1.0/2.0 and native 3.0; they do not change schema validation. Duplicate IDs still require byte-identical objects, excluding line terminators. Exceeding a limit fails explicitly rather than truncating output. Summary and ordinary filtered exports stream records; non-summary output is sanitized into a temporary spool before publication, so an invalid tail produces no output prefix. The spool is capped at 1 GiB and removed on normal completion or errors. Status also streams rather than retaining the journal. Timeline/correlation and other retaining readers additionally cap retention at 128 MiB of admitted frames and 100,000 records. Split larger evidence journals before using these retaining modes. These are physical input limits, not promises about allocator overhead or immutable acquisition snapshots.

• telltale scan --splunk-hec-endpoint ... --splunk-hec-token ... can post the same events through a Splunk HEC envelope when a deployment explicitly opts in.

Sink-specific metadata belongs at the sink or export layer. Do not add deployment-specific SIEM fields to the canonical event body.