Topic 10. Gradient Boosting
Topic 10. Gradient Boosting¶
Gradient boosting is one of the most prominent Machine Learning algorithms, it finds a lot of industrial applications. For instance, the Yandex search engine is a big and complex system with gradient boosting (MatrixNet) somewhere deep inside. Many recommender systems are also built on boosting. It is a very versatile approach applicable to classification, regression, and ranking. Therefore, here we cover both theoretical basics of gradient boosting and specifics of most widespread implementations – Xgboost, LightGBM, and Catboost.
Steps in this block¶
2. Watch a video lecture on logistic regression coming in 2 parts:
the theoretical part covers fundamental ideas behind gradient boosting;
the practical part, reviews key ideas behind major implementations: Xgboost, LightGBM, and CatBoost;