An optimal feature selection method for approximately duplicate records detecting

LI Xin; LI Jun; FENG Ji-lin; GAO Fang-ping; LI Zhong

doi:10.13873/j.1000-97872011.02.024

Chinese

您当前的位置：

首页 >

文章列表页 >

An optimal feature selection method for approximately duplicate records detecting

更新时间：2025-09-11

- An optimal feature selection method for approximately duplicate records detecting
- Vol. 30, Issue 2, Pages: 37-40(2011)
- 作者机构：
  
  防灾科技学院灾害信息工程系
- 作者简介：
- 基金信息：
- DOI：10.13873/j.1000-97872011.02.024
  CLC： TP311.13
- Published：2011
- 稿件说明：
移动端阅览
[1]李鑫,李军,丰继林,高方平,李忠.面向相似重复记录检测的特征优选方法[J].传感器与微系统,2011,30(02):37-40.

LI Xin, LI Jun, FENG Ji-lin, et al. An optimal feature selection method for approximately duplicate records detecting[J]. 2011, 30(2): 37-40.
[1]李鑫,李军,丰继林,高方平,李忠.面向相似重复记录检测的特征优选方法[J].传感器与微系统,2011,30(02):37-40. DOI： 10.13873/j.1000-97872011.02.024.

LI Xin, LI Jun, FENG Ji-lin, et al. An optimal feature selection method for approximately duplicate records detecting[J]. 2011, 30(2): 37-40. DOI： 10.13873/j.1000-97872011.02.024.

摘要

大数据集相似重复记录检测和识别中

数据源组成复杂、表征数据记录的特征属性过多

因而检测精度不高、执行检测的代价过大。针对这些问题

提出了一种分组模糊聚类的特征优选方法。首先进行分组记录的属性处理

以有效降低记录属性的维数

并获得分组中的代表性记录

然后采用一种相似度比较计算方法进行组内相似重复记录的检测。理论分析和实验表明:该方法有较高的识别精度和检测效率

能较好地解决大数据集中相似重复记录的识别问题。

Abstract

During duplicate records detection and recognition in large number of data sets

detection precision is low and cost of detecting is high because source of data are complicated and there are too many feature attributes.To solve these problems

an optimal feature selection method based on fuzzy clustering in groups is proposed.It deals with attributes of records in groups so as to reduce dimensions of attributes recorded effectively and obtain representative records in groups.It detects approximately duplicate records in groups by a computing method which compares with similarity.By theory analysis and experiments

it shows that identification precision and detection efficiency of this method are higher and it can solve recognition problem of approximately duplicate records in large number of data sets better.

关键词

Keywords

references

数据仓库中的相似重复记录检测方法 [J]. 李星毅,包从剑,施化吉. 电子科技大学学报 . 2007(06)

位置编码在数据仓库ETL中的应用 [J]. 张永,迟忠先. 计算机工程 . 2007(01)

对基于MPN的相似重复记录识别算法的改进 [J]. 刘伟,曹先彬. 微计算机信息 . 2005(14)

一种使用双阈值的数据仓库环境下重复记录消除算法 [J]. 洪圆,孙未未,施伯乐. 计算机工程与应用 . 2005(01)

一种高效的检测相似重复记录的方法 [J]. 邱越峰,田增平,季文贇,周傲英. 计算机学报 . 2001(01)

模糊聚类分析及其应用[M]. 西安电子科技大学出版社 , 高新波著, 2004

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Algorithm of collaborative filtering recommendation based on non-negative matrix factorization and item heat

D2D resource allocation algorithm in heterogeneous networks

Path planning method of autonomous mobile robot based on vision sensing system

Optimized design of TS fuzzy model based on improved genetic algorithm

Fuzzy C-spherical shell cluster algorithm based on genetic technique and FCSS

Related Author

YANG Haiqing

WU Hao

ZENG Junfei

XUE Jianbin

GUO Yujing

ZHAO Jing

CAO Jiang-tao

LI Pan-pan

Related Institution

College of Information Engineering,Zhejiang University of Technology

College of Computer and Communication,Lanzhou University of Technology

National Mobile Communications Research Laboratory,Southeast University

School of Information and Control Engineering,Liaoning Shihua University

School of Electrical and Information Engineering,Jiangsu University

⁰