Repair Score Tool

Status: WAITING DATA
Current Repair Score
0.00
Trigger Threshold (P90): 0.00
System State
IDLE
Policy: RS > P90(Base)
Distribution Shift (KS)
0.00
vs. Baseline CDF
Dataset Split
0 / 0
Baseline / Monitoring Obs
Signal Analysis & Phases
Green Zone: Baseline (Training)
Repair Score Evolution
Eq: RS = α₁·Perf + α₂·Dist + α₃·KS
Idx Phase Signal Perf (Δ%) Dist (μ) KS (D) RS Status

System Methodology

Technical Documentation & Methodological Foundation

Based on: Silva et al., "A Repair-Time Trigger for Cyberattack Classifiers", MILCOM.

1. System Architecture

The Repair-Time Trigger is an observability mechanism designed to detect Concept Drift and regime degradation in non-stationary time series, without relying on external ground truth labels in real-time.

Unlike traditional monitoring systems that measure error (which requires knowing the correct answer), this system measures the statistical plausibility of current behavior relative to a historical reference period.

Input Stream
Time Series Data
→
Metric Extraction
Perf, Dist, KS
→
Normalization
vs. Baseline Stats
→
Repair Score
Weighted Sum
→
Decision Policy
P90 Threshold

2. Baseline & Data Leakage Prevention

The scientific integrity of the system depends on the strict definition of a Reference Regime (Baseline). All normalization statistics (μ, σ, min, max) and the reference distribution for the KS test are calculated exclusively on the first N observations.

Isolation Principle: Future data (monitoring phase) must never influence normalization parameters. If new data exceeds the historical maximum, the system reports a normalized value > 1.0 (Out-of-Distribution), rather than rescaling the past.

Baseline Simulation

Adjust the Baseline size. The green area represents the system's "frozen" knowledge.

40%

3. Metric Decomposition

The Repair Score is composed of three orthogonal indicators of degradation.

Performance Degradation Proxy

In supervised systems, we use F1-Score. In unlabeled time series, we use Magnitude of Variation (Derivative) or Short-term Prediction Error as a proxy.

  • Measures: Abrupt instability and directional volatility.
  • Normalization: Relative to the maximum Δ observed in the baseline.
  • Interpretation: High values indicate erratic behavior not seen during training.

Euclidean Distance Proxy

Measures how far the current system state is from the "center of gravity" (centroid) of the normal regime.

  • Measures: Central Tendency Deviation (Mean Shift).
  • Methodology: D(t) = |xt - μbaseline|.
  • Diff vs Volatility: A system can be stable (low volatility) but operating at a totally anomalous price/value level (high distance).

Kolmogorov-Smirnov (KS) Test

A non-parametric test comparing the cumulative distribution function (CDF) of the current sliding window against the full baseline CDF.

The red line (D-stat) represents the maximum distance between the curves.

4. Repair Score Calculation

The final score is a weighted linear aggregation of the normalized metrics.

RS(t) = α1·M1(t) + α2·M2(t) + α3·M3(t)

Where Mn(t) are metrics normalized by baseline stats.
Note: Since the future can be more extreme than the past, it is mathematically expected for Mn(t) > 1.0 in severe anomalies.

Sensitivity Simulator

0.5
0.2
0.1
Resulting Repair Score: 0.29

5. Decision Policy (Trigger)

Absolute thresholds (e.g., "Alert if RS > 0.5") are arbitrary and fail in different contexts. This system uses a data-driven approach based on historical percentiles.

  • P90 (90th Percentile of Baseline): Upper limit of normal operation. If exceeded, indicates the system is in a state observed in only 10% (or less) of the calibration period.
  • Trigger: IF RS(t) > P90(baseline) THEN REPAIR

6. Functional Scope

✔ The System DOES ✖ The System DOES NOT
Detects statistical regime change (Concept Drift) in real-time. Does not forecast future price or market direction.
Quantifies the magnitude of anomaly relative to history. Does not classify the root cause (e.g., sensor error vs. market event).
Works in an unsupervised manner (no labels). Does not replace human validation for corrective action.
Adapts thresholds to baseline volatility. Does not automatically fix the model (only triggers the repair signal).