Machine learningDeep learning / NLP / CV
Semi-supervised RoBERTa-based Classification
Semi-supervised RoBERTa-based classification combines a large pretrained RoBERTa language model with both a small labeled dataset and a larger pool of unlabeled text. By generating pseudo-labels or enforcing consistency on unlabeled examples, the method extracts supervisory signal from unannotated data, yielding stronger classifiers when ground-truth annotations are scarce.
Open in MethodMindSoonVideoSoon
Read the full method
Members only
Sign inSign in with a free account to read this section.
Sources
- Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692. link ↗
- Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., & Le, Q. V. (2020). Unsupervised Data Augmentation for Consistency Training. Advances in Neural Information Processing Systems (NeurIPS), 33, 11904–11915. link ↗