LightGBM vs XGBoost vs Catboost

Quick summary

Hello 👋 In this article, I will compare LightGBM, XGBoost and CatBoost in the following areas:

• Boosting algorithm
• Node splitting
• Missing data handling
• Feature handling
• Data sampling
• LightGBM-specific features
• XGBoost-specific features
• CatBoost-specific features
• Tips for choosing between LightGBM, XGBoost and CatBoost
• Resources

🚀 Subscribe to us @ newsletter.verticalsolution.io

Boosting algorithm

Conventional boosting (LightGBM, XGBoost) vs Order boosting (CatBoost)

One of the major differences in tree building between LightGBM/XGBoost and CatBoost is the usage of ‘Order boosting’ in CatBoost.

In conventional boosting algorithms (used by LightGBM and XGBoost), at each boosting iteration, the tree is built using the same data points. It is argued that this repeated use of a single set of data points and can…