Skip to content
DSML.KZ
Publications

Code · Zenodo · October 28, 2026

A Hybrid Machine Learning System for Low-Cost, Interpreta-ble Bend, Connector, and Break Detection in OTDR Traces

Ilyas Kazambayev, Magzhan Amangeldi

Ilyas Kazambayev
Open via DOI

Abstract

OTDR Hybrid Detector — reproduction package Reproduction package for the article "A Hybrid Machine Learning System for Low-Cost, Interpretable Bend, Connector, and Break Detection in OTDR Traces" (Applied Sciences, MDPI, manuscript applsci-4616678). The package contains the data, code, trained models and reference results needed to reproduce the event-level evaluation reported in the article. Data OTDR reflectograms exported as CSV files (semicolon-separated, decimal comma; header lines followed by distance in km and signal level in dB). Recorded with a Gamma Lite OTDR on a laboratory fiber link of about 132 m (five fiber sections, four connectors at about 36, 66, 97 and 128 m). Acquisition settings: wavelength 1550 nm, pulse width 6 ns, group index 1.4681, 627 samples over 200 m, sampling interval 0.319 m. Three acquisition series (Class 1–3). These labels denote acquisition configurations, not event classes. Each annotated trace has a JSON mask with the positions (m) and types of the manually annotated events: bend, connector, break. Of 581 collected traces, 414 have annotation masks; one has no numeric data, leaving 413 usable traces. For eight byte-identical traces (SHA-256 aea30f11…), the masks were reconciled to a single reference annotation (connector 127.63 m, break 132.41 m). The original masks and the reconciliation log are included. Split The 413 usable traces were assigned to 151 heuristic measurement groups (byte-identical files, near-identical signals, consecutive measurements with matching annotations) and split at the group level into training / validation / test subsets of 290 / 42 / 81 traces (103 / 16 / 32 groups). split_manifest.csv lists the subset, group and SHA-256 of every file. The groups are heuristic and do not prove physical independence of repeated measurements. Method Hybrid detector: Main branch — candidate generation + ExtraTrees classifier for connector and break events. Bend branch — dense 1 m sliding-window scan with a separate ExtraTrees classifier. Both classifiers use 19 engineered local features, 500 trees, maximum depth 16 and inverse-group-size sample weights. The thresholds (main 0.50, bend 0.75) and class-aware 5 m non-maximum suppression were selected on the validation subset only. Reference results Test subset, 81 traces, 227 reference events, ±5 m, event level: TP / FP / FN Precision Recall F1 (pooled) F1 (group-weighted) 178 / 16 / 49 0.9175 0.7841 0.8456 0.8622 The package also contains the NMS ablation and the matching-tolerance sensitivity analysis. Contents Dataset_clean/ — 413 annotated CSV traces with reconciled JSON masks Dataset_group_split_clean/ — split_manifest.csv; train/val (CSV + mask), test (CSV only) reference_results/ — reference outputs of run_group_file.py and the trained models (main_model.joblib, bend_model.joblib) reference_results/clean_validation_analysis/ — validation P/R/F1 curves, feature distributions, permutation importance run_group_file.py, train_and_test_bend_detector_full.py, otdr_common.py, analyze_clean_validation.py, reproduce.py provenance/ — data-cleaning scripts, duplicate reconciliation (original masks, log), grouping audits, logs, build metadata, environment freeze README.md, requirements.txt, checksums.json How to reproduce pip install -r requirements.txt python reproduce.py verify # SHA-256 check of all 1630 files python reproduce.py run # retrain + evaluate; all 9 result files must equal reference_results/ within 1e-8 python reproduce.py analyze # validation-only analysis with the reference models; 6 files compared Verified on Windows (Python 3.11) and Linux (Python 3.13) with identical results. Limitations Laboratory data from a single set-up; no field validation. Most connector and break events are located at nearly fixed positions (≈127.6 m and ≈131–132 m). Measurement groups are heuristic. In the present system the Raspberry Pi acquires and transmits the traces; event detection runs on a host computer. Citation If you use this package, please cite the article (Applied Sciences, applsci-4616678; full reference will be added after publication).

Optical Network TechnologiesAdvanced Fiber Optic SensorsElectrical Fault Detection and Protectionfiber break detectioninterpretable classificationsliding window feature extraction

More by Ilyas Kazambayev