Machine learningDeep learning / NLP / CV

基于领域自适应BERT的分类

基于领域自适应BERT的分类通过以下方式扩展了标准微调流程：首先，在大量领域内无标签文本语料库上继续BERT的掩码语言模型预训练；然后，在目标分类任务的标签样本上微调自适应模型。这种两阶段方法弥合了BERT通用预训练语料库与生物医学、法律、金融或社交媒体文本等专业领域之间的词汇和分布差距。

在 MethodMind 中打开即将推出视频即将推出Download slides

阅读完整方法

仅限会员

使用免费账户登录即可阅读本节。

Method map

The neighbourhood of related methods — select a node to explore.

基于领域自适应BERT的分类

[需翻译标题：BERT-based Classi…域自适应 Transformer 微调 BERT 分类基于RoBERTa的分类句子嵌入 BERT 기반 전이 학습 域自适应Doc2Vec 领域自适应命名实体识别领域自适应问答域自适应循环神经网络

+4 more

来源

Gururangan, S., Marasovic, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., & Smith, N. A. (2020). Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), 8342–8360. DOI: 10.18653/v1/2020.acl-main.740 ↗
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020). BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4), 1234–1240. DOI: 10.1093/bioinformatics/btz682 ↗

如何引用本页

ScholarGate. (2026, June 3). Domain-Adaptive Pre-training with BERT for Text Classification. ScholarGate. https://scholargate.app/zh/deep-learning/domain-adaptive-bert-based-classification

Which method?

Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.

[需翻译标题：BERT-based Classification...]深度学习↔ compare
域自适应 Transformer深度学习↔ compare
微调 BERT 分类深度学习↔ compare
基于RoBERTa的分类深度学习↔ compare
句子嵌入深度学习↔ compare
BERT 기반 전이 학습深度学习↔ compare

Compare side by side →

被引用于

域自适应Doc2Vec 领域自适应命名实体识别领域自适应问答域自适应循环神经网络基于域自适应的 RoBERTa 分类领域自适应文本摘要领域自适应视觉 Transformer 弱监督BERT分类

发现本页有问题？报告或提出修改建议 →