Today’s post is about Model Explainability, techniques to try to understand why models give certain results – often blackbox/neural networks, and what features are more important.
There are two reasons I post this. One, that we have first class/leading approaches in use that were either created or are owned by people currently at Microsoft, and second, because not many people actually know this.
As one example, check out this link from June this year, looking at 7 packages for Explainability in Python: https://towardsdatascience.com/explainable-ai-xai-a-guide-to-7-packages-in-python-to-explain-your-models-932967f0634b . The page features at #1 SHAP, probably the most popular of all, but also at #2 LIME and also EBM/InterpretML, this one described as “an open source package from Microsoft”. What it doesn’t say however, is that the creators of both SHAP (Scott Lundberg) and LIME (Marco Tulio Ribeiro) are researchers at MSR. So out of a list of 7 packages, 3 were either created at Microsoft, or are owned by current employees. And if we add Fairlearn in a related area, and the recent developments on top of SHAP for Explainability on NLP and Vision, we are (in my view) unbeatable in this area.
Because EBM’s is probably the less well known of these, I’m also leaving a link to an article just focusing on it: https://towardsdatascience.com/interpretml-another-way-to-explain-your-model-b7faf0a384f8 , and what I’d suggest is this: next time you are training a model with XGBoost or LightGBM, try also EBMs. You may be surprised with how good it is.