| 朱徽* **,胡斌*,宋怡宁***,赵晓芳* ****.基于样本动态权重的课程式半监督学习方法[J].高技术通讯(中文),2024,34(4):342~355 |
| 基于样本动态权重的课程式半监督学习方法 |
| Curriculum paradigm based on the dynamic weights of samples for semi-supervised learning |
| |
| DOI：10. 3772 / j. issn. 1002-0470. 2024. 04. 002 |
| 中文关键词: 半监督学习； 特征表示向量； 课程学习； 特征原型； 语义相关度 |
| 英文关键词: semi-supervised learning, feature representation vector, curriculum learning, prototype of features, semantic relevancy |
| 基金项目: |
| 作者 | 单位 | | 朱徽* ** | （*中国科学院计算技术研究所北京 100190）
（**中国科学院大学北京 100049）
（***中央军委国防动员部信息中心北京 100034）
（****中科苏州智能计算技术研究院苏州 215028） | | 胡斌* | | | 宋怡宁*** | | | 赵晓芳* **** | |
|
| 摘要点击次数: 646 |
| 全文下载次数: 416 |
| 中文摘要: |
| 本文针对半监督场景中极度匮乏的监督信号导致的标签传播困难、模型训练严重受噪声干扰等问题展开研究。伪标签化带来的噪声和低数据利用率导致的确认偏差，会随着自训练过程造成错误累积，进而形成不可逆偏差，损害性能。本文提出基于样本动态权重的课程式半监督学习方法，旨在通过非离散的课程设计，鼓励模型由简单至困难地利用样本，逐步构建分类面，进而缓解伪标签化过程中的噪声产生，增强模型泛化能力。从类内角度，提供弱监督信号的高置信度伪标签被混合用于构建特征原型，估计样本的学习难度。从类间角度，标签嵌入被用于评估类间语义相关度，课程式地减弱训练前期对语义相关类别间的辨别。在通用的半监督学习基准数据集上进行了广泛的实验和分析，证明了方法的有效性。 |
| 英文摘要: |
| This work studies the difficulty of label propagation and serious noise interference in model training, which are due to the extreme lack of supervision signals in semi-supervised learning scenarios. Noise from pseudo-labeling and confirmation bias caused by low data utilization will lead to error accumulation along with the self-training process, thus forming irreversible deviation and damaging the performance. In this paper, a curriculum paradigm based on the dynamic weights of samples for semi-supervised learning is proposed, aiming at encouraging the model to utilize samples from easy to hard and gradually construct hyperplanes based on the non-discrete curriculum, so as to alleviate the generation of noise in the pseudo-labeling process and enhance the generalization ability of the model. Specifically, from the intra-class perspective, prototypes of features are constructed by mixing pseudo-labels with high confidence, which can provide weak supervision signals. Then, the learning difficulties of samples are estimated. From the inter-class perspective, label embedding is used to evaluate the semantic relevancy between categories, and the discrimination between semantically related categories are weaken in the early stage of training. Comprehensive experiments and analyses are conducted on commonly-used semi-supervised learning benchmark datasets to demonstrate the effectiveness of this method. |
|
查看全文
查看/发表评论 下载PDF阅读器 |
| 关闭 |
|
|
|
