源代码在我的github:
/danan0755/Bert_Classifier
头条新闻数据下载链接：
链接：/s/1P9G8pl4B78aPL-wtFTKf0g
提取码：4hm9
预训练模型使用chinese_roberta_wwm_ext_L-12_H-768_A-12
下载链接：
链接：/s/11iqTeuja63leVkTDO5dy8g
提取码：es0h
# bert模型设置bert_model = load_trained_model_from_checkpoint(config_path, checkpoint_path, seq_len=None) # 加载预训练模型for l in bert_model.layers:l.trainable = Truex1_in = Input(shape=(None,))x2_in = Input(shape=(None,))x = bert_model([x1_in, x2_in])x = Lambda(lambda x: x[:, 0])(x) # 取出[CLS]对应的向量用来做分类p = Dense(15, activation='softmax')(x)model = Model([x1_in, x2_in], p)pile(loss='sparse_categorical_crossentropy',optimizer=Adam(1e-5),metrics=['accuracy'])model.summary()train_D = data_generator(train_data)valid_D = data_generator(valid_data)
5个epoch后
loss: 0.1223 - accuracy: 0.9654 - val_loss: 0.9093 - val_accuracy: 0.8500
