SD Instances Open-Supply Challenge of the Week: Lumos


This week’s featured open-source venture is Lumos, a Python library constructed to check metrics between two datasets, accounting for inhabitants variations and invariant options.

Lumos was open sourced this month by Microsoft. In a technical paper that reveals the outcomes from a real-world deployment of Lumos in Microsoft RTC functions , the Microsoft group wrote: “Regressions in metric values should be detected and identified as early as potential to cut back the disruption to customers and product homeowners.”

“It has enabled engineering groups to detect 100s of actual adjustments in metrics and reject 1000s of false alarms detected by anomaly detectors,” the group added.

Based on Microsoft, regressions generally floor because of real product regressions, adjustments in person inhabitants, and bias because of telemetry loss or processing. 

The applying of Lumos has resulted in liberating up as a lot as 95% of the time allotted to metric-based investigations. Now the venture has been open sourced, it may be coupled with any manufacturing system to handle the amount of alerting effectively, Microsoft defined. 

The answer makes use of a brand new methodology that features present, domain-specific anomaly detectors, however was discovered to cut back the false-positive alert fee by over 90%. It additionally offers perception into finding the foundation reason for a Key Efficiency Indicator (KPI) incident. 

Lumos makes use of the rules of A/B testing and compares the dataset earlier than the metric anomaly and the dataset through the metric anomaly. The configuration file comprises hyper-parameters for working the workflow and particulars which columns within the dataset correspond to the metric, invariant, and speculation columns. 

This venture has adopted the Microsoft Open Supply Code of Conduct and is welcoming contributions and recommendations.