Understand Catboost in 10 minutes

What exactly is Catboost? How does it differ from LightGBM and XGBoost?

Table of content

1. Quick summary
2. Concepts for the algorithm
3. Parameters
4. Practical considerations
5. Reference materials

Quick summary

Catboost is another powerful library for gradient tree boosting, similar to LightGBM and XGBoost.

Compared to other boosting implementations (LightGBM and XGBoost), its special features are:

• Order boosting (a slightly modified gradient boosting algorithm)
• Advanced categorical variable handling
• Feature combination
• Symmetric trees

We will elaborate on these concepts in the following sessions.

Concepts for the algorithm

The following assumes that you are already familiar with how gradient boosting works. Otherwise for a brief review, please visit Concepts of boosting algorithms in machine learning.

Order boosting

Repeated use of the same dataset for each boosting iteration can lead to overfitting and hence prediction…