All Projects → jia-zhuang → xgboost-lightgbm-hyperparameter-tuning

jia-zhuang / xgboost-lightgbm-hyperparameter-tuning

Licence: other
Bayesian Optimization and Grid Search for xgboost/lightgbm

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to xgboost-lightgbm-hyperparameter-tuning

Hyperparameter hunter
Easy hyperparameter optimization and automatic result saving across machine learning algorithms and libraries
Stars: ✭ 648 (+1520%)
Mutual labels:  xgboost, lightgbm, hyperparameter-tuning
neptune-client
📒 Experiment tracking tool and model registry
Stars: ✭ 348 (+770%)
Mutual labels:  xgboost, lightgbm
Apartment-Interest-Prediction
Predict people interest in renting specific NYC apartments. The challenge combines structured data, geolocalization, time data, free text and images.
Stars: ✭ 17 (-57.5%)
Mutual labels:  xgboost, lightgbm
syne-tune
Large scale and asynchronous Hyperparameter Optimization at your fingertip.
Stars: ✭ 105 (+162.5%)
Mutual labels:  bayesian-optimization, hyperparameter-tuning
aws-machine-learning-university-dte
Machine Learning University: Decision Trees and Ensemble Methods
Stars: ✭ 119 (+197.5%)
Mutual labels:  xgboost, lightgbm
stackgbm
🌳 Stacked Gradient Boosting Machines
Stars: ✭ 24 (-40%)
Mutual labels:  xgboost, lightgbm
Kaggle
Kaggle Kernels (Python, R, Jupyter Notebooks)
Stars: ✭ 26 (-35%)
Mutual labels:  xgboost, lightgbm
fast retraining
Show how to perform fast retraining with LightGBM in different business cases
Stars: ✭ 56 (+40%)
Mutual labels:  xgboost, lightgbm
mlforecast
Scalable machine 🤖 learning for time series forecasting.
Stars: ✭ 96 (+140%)
Mutual labels:  xgboost, lightgbm
differential-privacy-bayesian-optimization
This repo contains the underlying code for all the experiments from the paper: "Automatic Discovery of Privacy-Utility Pareto Fronts"
Stars: ✭ 22 (-45%)
Mutual labels:  bayesian-optimization, hyperparameter-tuning
datascienv
datascienv is package that helps you to setup your environment in single line of code with all dependency and it is also include pyforest that provide single line of import all required ml libraries
Stars: ✭ 53 (+32.5%)
Mutual labels:  xgboost, lightgbm
mango
Parallel Hyperparameter Tuning in Python
Stars: ✭ 241 (+502.5%)
Mutual labels:  bayesian-optimization, hyperparameter-tuning
open-box
Generalized and Efficient Blackbox Optimization System.
Stars: ✭ 64 (+60%)
Mutual labels:  bayesian-optimization, hyperparameter-tuning
Kaggle-Competition-Sberbank
Top 1% rankings (22/3270) code sharing for Kaggle competition Sberbank Russian Housing Market: https://www.kaggle.com/c/sberbank-russian-housing-market
Stars: ✭ 31 (-22.5%)
Mutual labels:  xgboost, lightgbm
JLBoost.jl
A 100%-Julia implementation of Gradient-Boosting Regression Tree algorithms
Stars: ✭ 65 (+62.5%)
Mutual labels:  xgboost, lightgbm
MSDS696-Masters-Final-Project
Earthquake Prediction Challenge with LightGBM and XGBoost
Stars: ✭ 58 (+45%)
Mutual labels:  xgboost, lightgbm
Tencent2017 Final Rank28 code
2017第一届腾讯社交广告高校算法大赛Rank28_code
Stars: ✭ 85 (+112.5%)
Mutual labels:  xgboost, lightgbm
docker-kaggle-ko
머신러닝/딥러닝(PyTorch, TensorFlow) 전용 도커입니다. 한글 폰트, 한글 자연어처리 패키지(konlpy), 형태소 분석기, Timezone 등의 설정 등을 추가 하였습니다.
Stars: ✭ 46 (+15%)
Mutual labels:  xgboost, lightgbm
decision-trees-for-ml
Building Decision Trees From Scratch In Python
Stars: ✭ 61 (+52.5%)
Mutual labels:  xgboost, lightgbm
AutoTabular
Automatic machine learning for tabular data. ⚡🔥⚡
Stars: ✭ 51 (+27.5%)
Mutual labels:  xgboost, lightgbm

XGBoost/LightGBM 调参:基于贝叶斯优化和网格搜索

XGBoost 需要调节的参数

参数 取值 解释
learning_rate 0 ~ 1 学习速率越小,学习得越精细,但所需的学习次数(num_boost_round)越多。默认值 0.1,一般设置 0.05,0.01,0.005
max_depth > 0 树的深度。越大,模型越复杂,表达能力越强,但会过拟合。默认值是 6,常用探索范围 4 ~ 10,根据需要也可以设置更高的树深
min_child_weight > 0 当每个样本权重相等时,min_child_weight 就等于落入每个叶子结点的样本数。可以限制叶子结点继续分割,设置一个大的值可以防止过拟合。默认值是 1,常用探索范围 20 ~ 200,当样本规模很大时,可以设置更大的值
subsample 0 ~ 1 每一轮 boosting 时随机使用一部分样本。可以用来对抗过拟合
colsample_bytree 0 ~ 1 随机使用一部分 feature,可用来对抗过拟合

LightGBM 需要调节的参数

参数 取值 解释
learning_rate 0 ~ 1 与 XGBoost 类似
num_leaves > 0 与 XGBoost 中 max_depth 转换关系大致为 num_leaves = 2**max_depth,但树不是完全二叉树,一般设置时会小于 2**max_depth。比如,max_depth 为 7 时效果最好,那么 num_leaves 设为 70 或 80 就够了。默认值为 31
min_data_in_leaf > 0 与 XGBoost 中 min_child_weight 类似。默认值为 20
bagging_fraction 0 ~ 1 与 XGBoost 中 subsample 类似
feature_fraction 0 ~ 1 与 XGBoost 中 colsample_bytree 类似
lambda_l1/l2 > 0 L1/L2 正则化参数

贝叶斯优化

探索/利用平衡

参见代码:bayesHyperTuning.py

网格搜索

参见代码:gridHyperTuning.py

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].