All Projects → nishiwen1214 → GLUE-bert4keras

nishiwen1214 / GLUE-bert4keras

Licence: MIT license
基于bert4keras的GLUE基准代码

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to GLUE-bert4keras

DiscEval
Discourse Based Evaluation of Language Understanding
Stars: ✭ 18 (-69.49%)
Mutual labels:  glue, bert, natural-language-understanding
classy
classy is a simple-to-use library for building high-performance Machine Learning models in NLP.
Stars: ✭ 61 (+3.39%)
Mutual labels:  bert, natural-language-understanding
Clue
中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard
Stars: ✭ 2,425 (+4010.17%)
Mutual labels:  glue, bert
Chineseglue
Language Understanding Evaluation benchmark for Chinese: datasets, baselines, pre-trained models,corpus and leaderboard
Stars: ✭ 1,548 (+2523.73%)
Mutual labels:  glue, bert
tfbert
基于tensorflow1.x的预训练模型调用,支持单机多卡、梯度累积,XLA加速,混合精度。可灵活训练、验证、预测。
Stars: ✭ 54 (-8.47%)
Mutual labels:  bert, electra
WSDM-Cup-2019
[ACM-WSDM] 3rd place solution at WSDM Cup 2019, Fake News Classification on Kaggle.
Stars: ✭ 62 (+5.08%)
Mutual labels:  bert, natural-language-understanding
Tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Stars: ✭ 5,077 (+8505.08%)
Mutual labels:  bert, natural-language-understanding
AiSpace
AiSpace: Better practices for deep learning model development and deployment For Tensorflow 2.0
Stars: ✭ 28 (-52.54%)
Mutual labels:  bert, electra
Pycorrector
pycorrector is a toolkit for text error correction. 文本纠错,Kenlm,Seq2Seq_Attention,BERT,MacBERT,ELECTRA,ERNIE,Transformer等模型实现,开箱即用。
Stars: ✭ 2,857 (+4742.37%)
Mutual labels:  bert, electra
Mt Dnn
Multi-Task Deep Neural Networks for Natural Language Understanding
Stars: ✭ 1,871 (+3071.19%)
Mutual labels:  bert, natural-language-understanding
label-studio-transformers
Label data using HuggingFace's transformers and automatically get a prediction service
Stars: ✭ 117 (+98.31%)
Mutual labels:  bert, natural-language-understanding
Kevinpro-NLP-demo
All NLP you Need Here. 个人实现了一些好玩的NLP demo,目前包含13个NLP应用的pytorch实现
Stars: ✭ 117 (+98.31%)
Mutual labels:  baseline, bert
bert extension tf
BERT Extension in TensorFlow
Stars: ✭ 29 (-50.85%)
Mutual labels:  bert, natural-language-understanding
text2class
Multi-class text categorization using state-of-the-art pre-trained contextualized language models, e.g. BERT
Stars: ✭ 15 (-74.58%)
Mutual labels:  bert, natural-language-understanding
Fill-the-GAP
[ACL-WS] 4th place solution to gendered pronoun resolution challenge on Kaggle
Stars: ✭ 13 (-77.97%)
Mutual labels:  bert, natural-language-understanding
Filipino-Text-Benchmarks
Open-source benchmark datasets and pretrained transformer models in the Filipino language.
Stars: ✭ 22 (-62.71%)
Mutual labels:  bert, electra
Bert As Service
Mapping a variable-length sentence to a fixed-length vector using BERT model
Stars: ✭ 9,779 (+16474.58%)
Mutual labels:  bert, natural-language-understanding
Transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
Stars: ✭ 55,742 (+94377.97%)
Mutual labels:  bert, natural-language-understanding
CLUE pytorch
CLUE baseline pytorch CLUE的pytorch版本基线
Stars: ✭ 72 (+22.03%)
Mutual labels:  glue, bert
Person reid baseline pytorch
Pytorch ReID: A tiny, friendly, strong pytorch implement of object re-identification baseline. Tutorial 👉https://github.com/layumi/Person_reID_baseline_pytorch/tree/master/tutorial
Stars: ✭ 2,963 (+4922.03%)
Mutual labels:  baseline

基于bert4keras的GLUE基准代码

引用苏神博客中的话:“事实上,不管是tensorflow还是pytorch,不管是CLUE还是GLUE,笔者认为能找到的baseline代码,都很难称得上人性化,试图去理解它们是一件相当痛苦的事情。” (附上苏神的中文CLUE基准代码)

本人也是感同身受,既然有了中文CLUE基准代码,那么英文GLUE基准代码也得搞一个,所以决定基于bert4keras实现一套GLUE的baseline。经过测试,基本上复现了Huggingface的基准成绩,并且大多数任务还更优。最重要的是,所有代码尽量保持了清晰易读的特点(bert4keras的最大特点)。

GLUE benchmark: General Language Understanding Evaluation

⭐️欢迎star和提问~

实验结果:

  • val set:
Task Metric BERT-base* BERT-base# ELECTRA-base# ELECTRA-large#
CoLA Matthews corr 56.53 61.53 65.63 68.99
SST-2 Accuracy 92.32 92.66 94.04 94.95
MRPC F1/Accuracy 88.85/84.07 88.53/84.17 90.83/87.71 91.78/89.16
STS-B Pearson/Spearman corr 88.64/88.48 89.20/88.77 90.97/90.75 91.13/91.25
QQP Accuracy/F1 90.71/87.49 90.81/87.66 91.23/88.33 92.20/89.70
MNLI M acc/MisM acc 83.91/84.10 84.02/84.24 88.32/88.16 91.10/91.15
QNLI Accuracy 90.66 91.42 92.11 93.74
RTE Accuracy 65.70 69.68 81.23 87.73
WNLI Accuracy 56.34 56.34 64.79 84.51

* Huggingface # Our

  • test set:

image

image

使用

  • 下载GLUE数据集和bert预训练的权重(这里使用的是Google原版bert)到指定文件夹;
  • 例如:训练CoLA,直接运行 python CoLA.py

环境

  • 软件:bert4keras>=0.10.8, tensorflow = 1.15.0, keras = 2.3.1;
  • 硬件:bert-base和electra-base结果是用RTX 2080(12G)跑出来的,electra-large是用RTX 3090 (24G),注意3090显卡不支持Google的tf 1.x系列,需使用Nvidia的tf.15。安装教程可参考:https://www.bilibili.com/read/cv9162965/

更新

  • 2021.11.28,增加code的test set的预测功能,并且上传到GLUE网站进行评估,结果已公开。
  • 2021.12.07,增加ELECTRA-base和ELECTRA-large的dev set结果(代码暂时未公开)。
  • 增加SuperGLUE基准代码 https://github.com/nishiwen1214/SuperGLUE-bert4keras
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].