WANG Xue-rui, LIU Yuan. Chinese live-streaming danmaku new word detection based on boundary boosting[J]. 2018, (7): 142-146+150. DOI: 10.13873/j.1000-9787(2018)07-0142-05.
基于边界增强的中文直播弹幕新词发现
摘要
针对中文直播语料具有长度简短、用语不规范、字母数字混杂等特点
使得通常的新词发现方法效果不佳的问题
在现有基于词内部结合度与边界自由度的无监督新词发现算法基础上
引入了边界增强上下文熵的概念
使其适用于通常出现于句子片段边界位置的词语。在某知名直播平台的弹幕语料上进行了实验
结果表明:方法模型简洁
可有效提取新词
并能够通过调整参数适应不同规模的语料输入
且时空复杂度与执行性能良好。
Abstract
Aiming at problem that danmaku are short
irregular in nature and mixed with alphanumerical characters
which cause it is hard to handle with traditional new word detection methods. On the basis of previous methods based on internal cohesion and boundary freedom of words
the concept of boundary-boosted context entropy is introduced
making context entropy compatible with words mostly appearing at sentence fragment boundaries. Experimental results on a danmaku corpus collected from a well-known live-streaming service show effectiveness despite model simplicity
adaptability to differently sized input by means of parameter tuning