# Turbofan RUL Prediction: repository analysis

Audit date: September 2026. Baseline commit: `0873ef9`. Repository: <https://github.com/MEK-0/Turbofan-RUL-Prediction-LSTM>.

## Scope and evidence hierarchy

Inspected every tracked source file, both notebooks (code and recorded textual output), all twelve raw data files, HDF5 model configuration and weights, training CSV and both saved plots before implementing the report. There were no existing automated test suites or GitHub workflows. `api/test-api.py` was a manually invoked HTTP client. The baseline working tree was clean. No model was retrained.

Implementation and numerical artifacts take precedence over README claims. Source-level observations, historical annotations and newly reproduced results are distinguished below. The report shares the local NACA-0012-Airfoil-Noise-Prediction report's typography, layout and lightweight translation pattern, with original turbofan content and identity.

## Repository overview

- `src/data_loader.py`: space-separated C-MAPSS ingestion, standard column names, capped RUL construction.
- `src/preprocessing.py`: fixed sensor list, MinMaxScaler, per-engine windows and labels.
- `src/model_def.py`: Keras stacked LSTM regression network.
- `train.py`: fitting, validation, callbacks, checkpoint, scaler and training plot/log outputs.
- `evaluate.py`: last-window evaluation, explicit engine-ID ground-truth join.
- `api/main.py`: FastAPI lifespan artifact loading and `/predict` endpoint.
- `api/test-api.py`: local HTTP client using the first eligible test engine (ID 3).
- `notebooks/1_Exploratory_Data_Analysis.ipynb`: standard deviation analysis, raw trajectories and correlations.
- `notebooks/2_Signal_Processing_Windowing.ipynb`: illustrative rolling means, not executable training preprocessing.
- `models/`: saved HDF5 network, training log, training plot and evaluation plot. **No saved scaler is tracked or present.** `.gitignore` excludes `*.pkl`.
- `docs/`: added static bilingual report, evidence, figure attribution and deployment instructions.

## Dataset and complete raw-data inventory

All raw files are whitespace-delimited numeric data without headers. Trajectory files contain 26 columns; RUL files contain one value per engine. Counts below are computed directly from repository files, not copied from the README.

| Subset | Train engines | Train rows | Test engines | Test rows | RUL rows |
|---|---:|---:|---:|---:|---:|
| FD001 | 100 | 20,631 | 100 | 13,096 | 100 |
| FD002 | 260 | 53,759 | 259 | 33,991 | 259 |
| FD003 | 100 | 24,720 | 100 | 16,596 | 100 |
| FD004 | 249 | 61,249 | 248 | 41,214 | 248 |

Only **FD001** is selected by the training/evaluation code: `load_data()` defaults to FD001, and callers do not override it. The presence of the other subsets is not evidence of model training or evaluation on them. NASA Open Data confirms FD001 has one sea-level operating condition and one fault mode, high-pressure compressor (HPC) degradation. Small numerical variations in settings are not additional condition classes.

The NASA Open Data page lists FD004 train/test counts as 248/249; the checked-in files have 249/248. This discrepancy does not affect FD001. The same catalog text's final column says “sensor measurement 26”; the actual 26-column format consists of 5 index/setting columns plus **21 sensor channels**, confirmed from files and the historical paper. Raw files were not byte-compared to a freshly downloaded NASA archive; their NASA origin is attributed through repository context and schema agreement.

Input columns, in order:

```
unit_nr, time_cycles, setting_1, setting_2, setting_3,
s_1, s_2, s_3, s_4, s_5, s_6, s_7, s_8, s_9, s_10,
s_11, s_12, s_13, s_14, s_15, s_16, s_17, s_18, s_19, s_20, s_21
```

## Actual data pipeline

1. Construct per-row training RUL as each engine's final cycle minus the current cycle, capped above at 125. Test targets are the separately supplied RUL vector and remain uncapped.
2. Retain sensors in this exact order: `s_2, s_3, s_4, s_7, s_8, s_9, s_11, s_12, s_13, s_14, s_15, s_17, s_20, s_21`.
3. Remove `s_1, s_5, s_6, s_10, s_16, s_18, s_19` and all three operational settings from model features. ID and cycle remain available for grouping but are not model inputs.
4. Fit a scikit-learn `MinMaxScaler` to **all FD001 training sensor rows**; transform test rows with the same fitted object. Default output range is [0, 1]; test/API outliers can exceed it because clipping is not enabled.
5. **No moving average or other smoothing is applied** in training, evaluation or serving. Notebook 2 only visualizes 5- and 50-cycle rolling means, calls 50 “selected”, and conflates smoothing length with sequence length in its conclusion.
6. Generate windows within each engine, preserving file order. For N rows, `gen_sequence` emits **N − 50** windows. Total FD001 training samples: **15,631**, shape `(15631, 50, 14)`; labels `(15631, 1)`.
7. `gen_labels` uses `RUL[50:N]`. Therefore input rows 1–50 are paired with RUL at cycle 51. The final observed cycle is never an input in a training window; it is the final label. Evaluation/API use the last 50 observed rows and interpret output as endpoint RUL. This **one-cycle temporal mismatch** is preserved for historical-model evaluation rather than silently changing results.

The exploratory notebook calls its threshold “variance”, but computes **standard deviation < 0.01**. Source preprocessing hardcodes the resulting sensor list rather than fitting a VarianceThreshold object. Notebook claims that gate mechanisms “handle” multicollinearity or that all trends are monotonic are interpretations, not validated results.

## Model architecture and artifacts

The saved model configuration agrees with `src/model_def.py`:

| Layer | Configuration | Output per sample | Parameters |
|---|---|---|---:|
| Input | 50 time steps, 14 channels | 50 × 14 | 0 |
| LSTM | 128 units, return sequences | 50 × 128 | 73,216 |
| Dropout | 0.2 | 50 × 128 | 0 |
| LSTM | 64 units, final state | 64 | 49,408 |
| Dropout | 0.2 | 64 | 0 |
| Dense | 25, ReLU | 25 | 1,625 |
| Dense | 1, linear | 1 | 26 |
| Total | | | **124,275** |

LSTMs use tanh activation and sigmoid recurrent gates. Their internal dropout and recurrent dropout are zero; external Dropout layers implement regularization. No L1/L2 penalties, weight decay or constraints are configured. Linear output can in principle be negative or exceed 125; neither evaluation nor serving clamps it.

`models/saved_models/best_model_v1.h5` is a full serialized model with weights and optimizer state, not merely weights. Its recorded Keras version is 3.13.0. Stored optimizer is Adam, MSE loss, MAE metric and learning rate approximately 0.0005. Evaluation loads with `compile=False` to avoid historical loss-serialization issues.

Original artifact SHA-256 hashes (unchanged by this work):

```
best_model_v1.h5  7c62fd8ad495425a327479bbb84c2c87fe93a715d03f57c26678b0e8b24588db
training_log.csv b151eb32a2402abacefbdb5d0a3184c992a7e3ae0a329f127528740b22f907ef
training_plot.png 05d1fcf42db5876cc5fa8bb561a3601c2f18567635665e3456006a0b1d18c0f4
test_result_plot.png 083b025c6d3b080431cd6c3cca59181056d18a144a5dd959e136053b2fb25b33
```

## Training and validation protocol

Source configuration: Adam initial LR 0.001, MSE loss, batch size 64, up to 100 epochs, `validation_split=0.2`, default training shuffle, no fixed seeds. Callbacks select minimum validation loss, use early stopping patience 10 with best-weight restoration, halve LR after 3 unimproved epochs, and write CSV history.

The saved log has **25 epochs (0–24)**. Minimum validation MSE is **148.4639129638672 at index 14**; corresponding validation MAE is **9.415528297424316**. Square root of the MSE is approximately 12.1846 cycles. These are validation values, not test metrics. The epoch history is consistent with stopping ten epochs after the best validation epoch; the log is not an independently authenticated training record.

Keras takes the last 20% of ordered windows as validation before shuffling training samples. The split is 12,504 training and 3,127 validation windows. It falls **12 windows into engine 83**: engines 1–82 and the first 12 windows of engine 83 are training, the rest of 83 and engines 84–100 are validation. The adjacent windows across the split share **49 of 50 rows**.

## Evaluation protocol and verified metrics

Select the final 50 observations of each test engine with at least 50 cycles. Exactly **93 engines** qualify. Excluded IDs: **1, 2, 14, 22, 25, 39, 85**. True RUL is joined through its one-based engine ID (`RUL_FD001.iloc[id-1]`), not through a shortened positional list. Targets and predictions remain unclipped. Metrics pool one endpoint prediction per included engine.

The archived evaluation image explicitly displays RMSE **13.87**, but contains no recoverable numerical prediction table or MAE annotation. The old README claims RMSE 13.87 and MAE 10.24 for 100 test engines. Re-evaluation now independently produces:

| Quantity | Verified value |
|---|---:|
| RMSE | **13.8725686550 cycles** |
| MAE | **10.2374445187 cycles** |
| Mean signed error (prediction − truth) | **+1.3382252929 cycles** |
| Included engines | **93 / 100** |

**Essential qualification:** the original scaler is missing. This verification fits a new MinMaxScaler on the committed FD001 training rows, following the exact current preprocessing recipe. It does not fit network weights, use test data for scaling, or prove the identity of the absent original scaler. Rounded metrics agree with historical claims, while historical full-run provenance remains unverified.

Reproduction:

```sh
python evaluate.py --reconstruct-scaler --output-dir docs/evidence
MPLCONFIGDIR=/tmp/turbofan-mpl python scripts/generate_report_figures.py
```

`docs/evidence/metrics.json` records versions and hashes; `predictions.csv` records engine IDs, truth, predictions and signed errors. The environment used TensorFlow 2.21.0, scikit-learn 1.9.1, pandas 3.0.6 and NumPy 2.5.3 on Python 3.12.13. Tiny platform-dependent numerical differences are possible. No PHM scoring function, confidence interval, repeated-run distribution or external test set is implemented.

## FastAPI inference architecture

The lifespan manager loads model and scaler once into `ml_models`. `POST /predict` accepts `{ "data": [ ... ] }`, requiring all 14 named numeric fields on each row. At least 50 rows are required; extra rows are discarded from the beginning. Sensor columns are reordered to the training order, scaled and reshaped to `(1,50,14)` before inference. Response fields are `status: "success"`, `predicted_rul` rounded to 2 decimals and `unit: "cycles"`.

`GET /` returns service information. `/docs` is FastAPI's generated Swagger UI, unrelated to this repository's static `docs/` directory. Startup catches missing artifacts and leaves the service running but unavailable for prediction. The baseline exception handler converted intended 400/503 errors into 500; it now preserves HTTPException status codes. Pydantic request failures return 422. The API has no identity/time-order/range validation, finite-value constraints, rate limits, authentication, uncertainty output or deployment evidence. Only a separately hosted Python service can perform inference; GitHub Pages cannot.

## Mismatches and corrections

| Baseline claim/problem | Verified state and action |
|---|---|
| “High Accuracy”, benchmark superiority, production-ready/deployed API | Unsupported wording removed; regression errors and research-prototype scope stated. |
| Dropout 0.3 | Source and saved model both use 0.2 twice; Dense(25, ReLU) was omitted from README. |
| Moving average window 5 | Absent in executable pipeline; notebook compares 5 and 50, selecting 50 only illustratively. |
| 100 evaluated engines | 93 under existing no-padding protocol. Metric values reproduce with reconstructed scaler. |
| Scaler listed as present | Missing and globally ignored by `*.pkl`; serving prerequisite made explicit. |
| Root train/evaluate imports and `../` paths | Broken module imports and working-directory-dependent paths repaired using repository-root paths and `src.*` imports. |
| `python src/evaluate.py` | File is at repository root; README corrected. |
| Placeholder clone URL and wrong tree | Real URL, actual filenames and notebook count used. No third evaluation notebook exists. |
| API errors swallowed | HTTPException is re-raised unchanged; modern Pydantic serialization used. |
| Client imports requests but dependency absent | requests declared; Pydantic v2 floor reflects `model_dump`. |
| Notebook 2 starts by uninstalling TensorFlow | Hazardous environment-mutating cell removed; analytical cells/outputs preserved. |
| MIT badge with no LICENSE file | Unverified license claim removed; no license grant invented. |
| Broad runtime error in train exits successfully | Failure now re-raised for nonzero process status. |

The training objective, windows, sensor order and saved artifacts have not been changed. Evaluation now offers explicit scaler reconstruction and optional evidence export instead of overwriting the historical plot by default. No silent reconstruction happens in the API.

## Strengths

Clear module separation; sensor-based sequence model; per-engine windows avoid mixing engines inside a sample; training-only test normalization; ID-preserving test-label joins; actual model and training history available; repeated inference reproduces rounded historical metrics; lightweight API model caching. The added report links machine-readable evidence and identifies its limitations.

## Weaknesses, leakage risks and technical debt

- **Validation preprocessing leakage:** all training-file rows, including future validation engines, determine scaler extrema. Fit preprocessing only after an engine-wise split in a future experiment.
- **Validation temporal overlap:** split crosses engine 83, with nearly identical adjacent windows. This is not an engine-independent validation protocol.
- **No direct test leakage detected in current code:** scaler reconstruction uses training rows only and evaluation does not train. Historical notebook tuning or test reuse cannot be ruled out from artifacts.
- **Target timing and clipping mismatch:** one-cycle-ahead capped training labels versus uncapped endpoint evaluation. Fixing this properly requires a separately documented training run; no such run was performed.
- **Selection bias:** short test histories are omitted. Report coverage explicitly; future padding/masking policy needs separate evaluation.
- **Limited domain:** FD001 alone cannot support claims about multiple conditions, multiple faults, real fleets or general safety.
- **Reproducibility gaps:** missing scaler, no dependency lock or seeds, incomplete original provenance, notebook/source divergence, no external validation.
- **Model uncertainty absent:** no calibration, confidence intervals, uncertainty quantification or domain adaptation.
- **Serving debt:** synchronous inference, weak sequence semantics validation, permissive coercion, no production operational controls. The scaler pickle must come from a trusted source.

## Deployment constraints

Static site uses relative local paths, complete English fallback HTML, CSS, inline SVG file assets and local vanilla JavaScript. German switching updates text, figure captions, alt text, document language/title and generated plot assets. Source titles and NASA diagram labels retain their original language. It contains no inference endpoint calls or remote runtime dependencies. GitHub Pages must publish the `main` branch `/docs` folder; activation/push remain user actions. See `GITHUB_PAGES_DEPLOYMENT.md`.

Python serving separately requires compatible TensorFlow/Keras dependencies plus the model and original or explicitly reconstructed scaler. Default evaluation fails clearly when the scaler is absent. Deliberate reconstruction can optionally save a new scaler file using `--scaler-output`; existing files are protected from overwrite. Training is not needed for report reproduction and would replace historical model outputs if intentionally run.

## Sources and unverified claims

- [NASA PCoE dataset repository](https://www.nasa.gov/intelligent-systems-division/discovery-and-systems-health/pcoe/pcoe-data-set-repository/): simulation provenance and dataset citation.
- [NASA Open Data C-MAPSS catalog](https://data.nasa.gov/dataset/cmapss-jet-engine-simulated-data): FD001 conditions, faults, schema and endpoint-RUL context.
- [Saxena et al., PHM 2008, official NASA PDF](https://c3.ndc.nasa.gov/dashlink/static/media/publication/2008_IEEEPHM_CMAPPSDamagePropagation.pdf): historical simulation context and Figures 1–2.
- [NASA NTRS record 20090029214](https://ntrs.nasa.gov/citations/20090029214): public distribution/use metadata and alternate catalog title.

Unverified: exact original scaler bytes, original execution environment and seeds, complete training provenance, claimed superiority to “standard benchmarks”, production deployment, generalization to physical engines, and an MIT license grant. No university affiliation or NASA endorsement is claimed.