profile

Train in Data

by Soledad Galli, PhD | Data scientist | Python Developer | Best-selling instructor | Book author | 👉 www.trainindata.com

TunedThresholdClassifierCV: the good, the bad, the ugly
Featured Post

TunedThresholdClassifierCV: the good, the bad, the ugly

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! TunedThresholdClassifierCV: the good, the bad, the ugly For years, scikit-learn classifiers made one decision for us: predict() converted probabilities into class labels using a de facto cut-off of 0.5. That number seems obvious from stats classes. But in practice, it’s rarely the right one. A threshold is not a property...

Undersampling does not improve model performance

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! My Hyperparameter Optimization Course Just Got a Major Update! I’ve released a completely updated version of my course, now called Master Hyperparameter Optimization for Tabular Learning. It focuses on optimizing today’s leading tabular models using the tools and techniques the machine-learning community relies on,...

Class imbalance makes metrics volatile

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! Have you added it to your reading list yet? Imbalanced Data, Myths, Mistakes and Modern Solutions Imbalanced data isn't a problem. How you handle it is. My book, Imbalanced Data: Myths, Mistakes and Modern Solutions, cuts through the common misconceptions and brings together practical, evidence-based approaches you can...

Class Weights Do Not Improve Model Performance

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! My New Book is Out! Imbalanced Data, Myths, Mistakes and Modern Solutions Most advice about imbalanced data is incomplete, or simply wrong. This book challenges common assumptions and provides practical, evidence-based guidance for tackling class imbalance and building better machine learning models. Get the book Class...

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! My New Book is Out! Imbalanced Data, Myths, Mistakes and Modern Solutions Most advice about imbalanced data is incomplete, or simply wrong. This book challenges common assumptions and provides practical, evidence-based guidance for tackling class imbalance and building better machine learning models. Find out more...

Models are not sensitive to class imbalance

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! My latest eBook Imbalanced Data, Myths, Mistakes and Modern Solutions is here! Most advice about imbalanced data is incomplete, or simply wrong. This book challenges common assumptions and provides practical, evidence-based guidance for tackling class imbalance and building better machine learning models. Get your copy...

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! The ROC Curve Myth for Imbalanced Datasets One myth I hear far too often is that the ROC curve is not a good metric for evaluating models trained on imbalanced datasets because it tends to produce overly optimistic ROC-AUC values. This is not true. Last Monday we saw that ROC curves are insensitive to class prevalence:...

Welcome! Every Monday, I’ll drop a no-fluff, straight-to-the-point tip on a data science skill, tool, ormethod to help you stay sharp in the field. I hope you find it useful! Just a friendly reminder to complete our survey 💙 Your feedback helps us create courses and learning resources that are more useful for our community. Thanks so much for your support! Take the survey Myth: ROC-AUC is sensitive to class imbalance Far too often, we hear that the ROC curve and ROC-AUC are unsuitable for...