Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection

Yun LIU; Wenfeng ZHENG; Yi ZHANG

doi:10.14188/j.1671-8836.2021.0111

Chinese Version 武汉大学期刊中心

Your Location：

Home >

Browse articles >

Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection

Updated：2022-02-23

- Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection
- J Wuhan Univ （Nat Sci Ed）
- Affiliations：
  
  1.Faculty of Information Engineering and Automation， Kunming University of Science and Technology， Kunming 650500， Yunnan， China
- Author bio：
- Funds：
- DOI：10.14188/j.1671-8836.2021.0111 CLC： TP311
- Published：24 February 2022，
  
  Received：06 May 2021，
Scan for full text
Cite this article
刘云,郑文凤,张轶.卡方校正算法对入侵检测特征选择的优化[J].武汉大学学报（理学版）,2022,68(1):65-72.

LIU Yun，ZHENG Wenfeng，ZHANG Yi.Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection ［J］.J Wuhan Univ （Nat Sci Ed），2022，68（1）：65-72. DOI：10.14188/j.1671-8836.2021.0111（Ch）.
刘云,郑文凤,张轶.卡方校正算法对入侵检测特征选择的优化[J].武汉大学学报（理学版）,2022,68(1):65-72. DOI：10.14188/j.1671-8836.2021.0111

LIU Yun，ZHENG Wenfeng，ZHANG Yi.Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection ［J］.J Wuhan Univ （Nat Sci Ed），2022，68（1）：65-72. DOI：10.14188/j.1671-8836.2021.0111（Ch）. DOI：

卡方校正算法对入侵检测特征选择的优化 Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection first-author 刘云，男，副教授，现从事数据挖掘、深度学习等方向研究。 E-mail：liuyun@kmust.edu.cn 刘云，男，副教授，现从事数据挖掘、深度学习等方向研究。 E-mail：liuyun@kmust.edu.cn liuyun@kmust.edu.cn corresp E-mail： 2867328528@qq.com E-mail： 2867328528@qq.com 2867328528@qq.com 昆明理工大学信息工程与自动化学院，云南昆明 650500 昆明理工大学信息工程与自动化学院，云南昆明 650500 Faculty of Information Engineering and Automation， Kunming University of Science and Technology， Kunming 650500， Yunnan， China Faculty of Information Engineering and Automation， Kunming University of Science and Technology， Kunming 650500， Yunnan， China 在入侵检测系统中互信息特征选择标准可快速选择重要特征，通常信息熵的计算偏差会降低系统的分类性能。为了减少特征选择偏差的影响，提出卡方校正算法（chi-square correction algorithm，CSCA）。首先，对所有候选特征进行离散化处理，计算互信息特征选择相关标准的偏差；然后，将偏差项添加在特征选择目标函数中，通过CSCA优化离散化水平和特征偏差；最后，在更新后的特征集中选择当前最重要的特征子集，在分类模型中检测攻击。仿真结果表明，与MIGM（mutual information gain maximize）算法和M-DFIFS（M-dynamic feature importance based feature selection）算法相比，卡方校正算法提高了入侵检测系统的精度，同时降低了系统的误报率。 Mutual information feature selection criteria can be used in intrusion detection systems to quickly select important features. Usually, the calculation deviation of information entropy will reduce the classification performance of the system. In order to reduce the influence of feature selection deviation, a chi-square correction algorithm（CSCA） is proposed. First, we discretize all candidate features and calculate the deviation of the relevant criteria for mutual information feature selection; secondly, we add the deviation term to the feature selection objective function, and optimize the discretization level and feature deviation through the CSCA; finally, in the updated feature set, the most important feature subset is selected to detect attacks in the classification model. The simulation results show that, compared with the MIGM（mutual information gain maximize） algorithm and the M-DFIFS（M-dynamic feature importance based feature selection） algorithm, the chi-square correction algorithm improves the accuracy of the intrusion detection system and reduces the false alarm rate of the system. 入侵检测特征选择互信息离散化卡方检验 intrusion detection feature selection mutual information discretization chi-square test 0　引言 1 数据特征选择在大数据挖掘和分析及机器学习等领域都有广泛应用，尤其在入侵检测系统（intrusion detection system，IDS）中，选择重要数据特征是识别网络攻击准确性的关键因素［ 1 1 ］。为了给分类模型构建高效的特征选择（feature selection，FS）算法，通常用基于互信息（mutual information，MI）的方法学习和估算高维数据特征的重要性，该方法可以快速找到数据特征中的相关信息，选择出最大相关最小冗余（maximum relevance minimum redundancy， mRMR）的特征子集［ 2 2 ， 3 3 ］。 Wang等［ 4 4 ］结合数据特征的关联性，构建了互信息增益最大化（mutual information gain maximize， MIGM）算法，通过等效分区概率对特征数据进行划分，并用最大联合互信息准则对候选特征进行评估。该方法可以快速识别有效的特征子集，比传统方法具有更好的适用性。Wei等［ 5 5 ］根据动态特征重要性（dynamic feature importance，DFI）指标，提出基于MI的动态特征重要性特征选择（M-dynamic feature importance based feature selection，M-DFIFS）算法，使用最大信息系数有效排除冗余特征，再通过随机森林的基尼系数选出重要特征，所选的特征数据与类要素之间有更强的相关性。但是这些方法都没有考虑信息熵的偏差校正。为了在入侵检测系统中减少信息熵偏差对分类性能的影响，构建实时可靠的IDS，本文提出了卡方校正算法（chi-square correction algorithm，CSCA）。首先对所有候选特征进行离散化处理，根据特征的潜在概率分布估计特征选择相关标准的偏差；然后，通过考虑偏差的目标函数得到特征选择的临界值；最后，通过 <math id="M1"><msup><mrow><mi mathvariant="bold-italic">χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544028&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544023&type= 3.13266683 3.89466691 检验优化离散化水平和特征偏差，更新特征子集。 1　入侵检测特征选择 1 1.1　入侵检测特征选择模型 2 选择重要相关的特征并及时做出数据更新对IDS防御攻击非常重要。特征选择的目的就是在所有候选特征集 <math id="M2"><mi mathvariant="bold-italic">F</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544037&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544033&type= 2.62466669 2.79399991 中选出最优特征子集 <math id="M3"><mi mathvariant="bold-italic">f</mi><mo>=</mo><mo stretchy="false">{</mo><msub><mrow><mi>f</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544046&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544042&type= 26.16200066 4.91066647 ，为分类模型提供在类之间具有最大区分力的特征数据，并降低数据维度［ 6 6 ］。针对高维数据流中存在的特征变化，基于MI的特征选择方法可减少无关特征和冗余特征对模型的影响。为了简化特征数据，通常在特征选择之前把连续值的特征转换为离散特征。在图1 图1 入侵检测模型中，将数据流中提取的特征和训练后的标准特征动态离散化为 n 个区间，则离散化间隔为 <math id="M4"><mo stretchy="false">{</mo><mo stretchy="false">[</mo><msub><mrow><mi>d</mi></mrow><mrow><mn mathvariant="normal">0</mn></mrow></msub><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">]</mo><mo>,</mo><mo>⋯</mo><mo>,</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544056&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544050&type= 22.26733208 4.48733330 <math id="M5"><mo stretchy="false">[</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>n</mi><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>n</mi></mrow></msub><mo stretchy="false">]</mo><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544067&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544062&type= 17.44133186 4.48733330 ，特征 <math id="M6"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544077&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544070&type= 1.86266661 4.57200003 的最小值为 <math id="M7"><msub><mrow><mi>d</mi></mrow><mrow><mn mathvariant="normal">0</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544085&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544079&type= 3.30200005 3.72533321 ，最大值为 <math id="M8"><msub><mrow><mi>d</mi></mrow><mrow><mi>n</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544093&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544091&type= 3.30200005 3.72533321 ， <math id="M9"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub><mi>s</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544103&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544101&type= 4.57200003 3.72533321 是间隔点。特征离散化可能存在信息丢失，但可以减少数据噪声，增加模型的稳定性和分类准确性［ 7 7 ］。数据特征离散化后，通过特征选择的标准为分类器选择最优特征子集，并将该特征子集更新为下一次检测的候选特征。最后使用支持向量机（support vector machines，SVM）分类模型检测攻击，SVM有效减少了数据维度的影响，有较高的分类精度［ 8 8 ］。为了独立于分类算法并降低计算成本，一般构建基于MI过滤的特征选择方法评估特征。MI方法能够测量随机变量之间的非线性依赖关系，评估复杂分类任务中数据特征的信息内容［ 9 9 ］。两个变量的MI通常用熵表示，熵为信息不确定性的度量，计算形式如下 <math display="block" id="M10"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">)</mo><mo>=</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>+</mo><mi>H</mi><mo stretchy="false">(</mo><mi>Y</mi><mo stretchy="false">)</mo><mo>-</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544125&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544122&type= 60.62133789 3.55599999 其中， <math id="M11"><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544133&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544129&type= 9.99066734 3.55599999 和 <math id="M12"><mi>H</mi><mo stretchy="false">(</mo><mi>Y</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544142&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544137&type= 9.65200043 3.55599999 为边际熵； <math id="M13"><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544150&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544145&type= 14.64733315 3.55599999 为联合熵，为变量 <math id="M14"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 和 <math id="M15"><mi>Y</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545096&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545093&type= 2.87866688 2.79399991 平均所需要的信息量。若 <math id="M16"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 是连续型随机变量，则 <math id="M17"><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>=</mo><mo>-</mo><msubsup><mo>∫</mo><mrow><mo>-</mo><mi>∞</mi></mrow><mrow><mo>+</mo><mi>∞</mi></mrow></msubsup><mrow><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mi>l</mi><mi>o</mi><mi>g</mi></mrow><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mi mathvariant="normal">d</mi><mi>x</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544183&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544179&type= 51.30799866 7.87400007 ， f （ x ）表示 <math id="M18"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 的概率分布函数。若用 <math id="M19"><mi mathvariant="italic">𝒩</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544915&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544912&type= 3.04800010 3.13266683 个等长 <math id="M20"><mi mathvariant="normal">Δ</mi><mi>x</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544200&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544196&type= 4.48733330 2.79399991 分隔 <math id="M21"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 ，则第 <math id="M22"><mi>i</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544322&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544321&type= 1.10066664 2.79399991 个间隔中观测样本的概率值 <math id="M23"><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mo>∫</mo><mi>f</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mi mathvariant="normal">d</mi><mi>x</mi><mo>≈</mo><mi>f</mi><mo stretchy="false">(</mo><msub><mrow><mi>x</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mi mathvariant="normal">Δ</mi><mi>x</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544216&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544213&type= 39.87799835 6.60400009 ，假设 f （ x ）在一个区间内近似恒定，熵的近似值如下 <math display="block" id="M24"><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>=</mo><mo>-</mo><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mi>l</mi><mi>o</mi><mi>g</mi><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo stretchy="false">)</mo></mrow><mo>+</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544225&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544220&type= 61.55266953 8.72066593 <math display="block" id="M25"><mo>-</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mi>l</mi><mi>n</mi><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo stretchy="false">)</mo></mrow><mo>+</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mi>x</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544232&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544228&type= 48.09066391 8.72066593 其中， <math id="M26"><mi>N</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544238&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544235&type= 2.87866688 2.87866688 表示总的样本数， <math id="M27"><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544446&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544442&type= 2.62466669 3.72533321 为第 <math id="M28"><mi>i</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544322&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544321&type= 1.10066664 2.79399991 个间隔的观测样本， <math id="M29"><mi>i</mi><mo>∈</mo><mo stretchy="false">(</mo><mn mathvariant="normal">0</mn><mo>,</mo><mi mathvariant="italic">𝒩</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544258&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544254&type= 15.32466698 3.55599999 且 <math id="M30"><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo>=</mo><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544266&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544261&type= 11.00666618 8.04333401 。 1.2　特征选择标准 2 假设一组数据集中有 <math id="M31"><mi>K</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544273&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544270&type= 2.79399991 2.79399991 个特征数据和 <math id="M32"><mi>M</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544281&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544277&type= 3.38666677 2.87866688 个数据类别，用互信息 <math id="M33"><mi>I</mi><mo stretchy="false">(</mo><mi mathvariant="bold-italic">f</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544287&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544283&type= 11.51466751 3.97933316 表示特征数据中包含的关于目标类别 <math id="M34"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 的信息量，从最大化下式 <math display="block" id="M35"><mi>I</mi><mo stretchy="false">(</mo><mi mathvariant="bold-italic">f</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>=</mo><munder><mo movablelimits="false">∑</mo><mrow><msub><mrow><mi>f</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>C</mi></mrow></munder><mrow><mi>P</mi></mrow></mrow><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mfrac><mrow><mi>P</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></mrow><mrow><mi>P</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>s</mi></mrow></msub><mo stretchy="false">)</mo><mi>P</mi><mo stretchy="false">(</mo><mi>C</mi><mo stretchy="false">)</mo></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544301&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544297&type= 85.34400177 9.82133293 可以得到特征子集 <math id="M36"><mi mathvariant="bold-italic">f</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544309&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544305&type= 1.18533325 3.64066648 的最小集合。为了减少 <math id="M37"><mi>I</mi><mo stretchy="false">(</mo><mi mathvariant="bold-italic">f</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544317&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544314&type= 11.51466751 3.97933316 的计算成本，通过联合互信息（joint mutual information，JMI）方法［ 10 10 ］选择第 <math id="M38"><mi>i</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544322&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544321&type= 1.10066664 2.79399991 个特征 <math id="M39"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 的模型如下式 <math display="block" id="M40"><mi>J</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mi>R</mi><mo>+</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mfenced open="|" close="|" separators="|"><mrow><mi mathvariant="bold-italic">f</mi></mrow></mfenced></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo>∈</mo><mi mathvariant="bold-italic">f</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mi>C</mi><mi>I</mi><mo>-</mo></mrow><mi>r</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544335&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544331&type= 44.78866577 9.90600014 其中， <math id="M41"><mfenced open="|" close="|" separators="|"><mrow><mi mathvariant="bold-italic">f</mi></mrow></mfenced></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544339&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544338&type= 5.67266655 4.82600021 为所选特征子集 <math id="M42"><mi mathvariant="bold-italic">f</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544345&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544343&type= 1.18533325 3.64066648 的特征数， <math id="M43"><mi>R</mi><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544354&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544349&type= 20.40466690 4.91066647 表示 <math id="M44"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M45"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 之间的相关性； <math id="M46"><mi>r</mi><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544375&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544370&type= 18.71133423 4.91066647 表示数据特征 <math id="M47"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M48"><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544415&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544410&type= 2.11666679 4.57200003 之间的冗余； <math id="M49"><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544391&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544388&type= 24.46866608 4.91066647 表示在给定的类别 <math id="M50"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 下 <math id="M51"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M52"><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544415&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544410&type= 2.11666679 4.57200003 之间的互补信息［ 11 11 ］。通过最大化（4）式得到特征选择标准的最优值，即最大特征的相关性 <math id="M53"><mi>R</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545878&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545874&type= 2.70933342 2.79399991 、最小的特征冗余 <math id="M54"><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545869&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545866&type= 1.52400005 2.79399991 和最大特征的互补信息 <math id="M55"><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545861&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545857&type= 3.89466691 2.79399991 ，根据这三个标准可以选择当前最重要的特征。用潜在的概率分布估计特征熵时会引入偏差，假设样本 <math id="M56"><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544446&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544442&type= 2.62466669 3.72533321 是多项式独立分布的，根据概率函数在 <math id="M57"><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544461&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544453&type= 3.64066648 8.04333401 处的泰勒级数，样本期望为 <math display="block" id="M58"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mo>=</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544472&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544466&type= 21.33600044 4.23333359 <math display="block" id="M59"><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mo>-</mo><mfrac><mrow><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mfrac><mrow><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo>-</mo><mo stretchy="false">(</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi>N</mi></mrow></mfrac><mo>+</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi>N</mi></mrow></mfrac><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mfrac><mrow><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo stretchy="false">)</mo><mo>⋅</mo></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544483&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544476&type= 56.64199829 8.72066593 <math display="block" id="M60"><mi>E</mi><mo stretchy="false">{</mo><mo stretchy="false">(</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>-</mo><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mo>-</mo><mfrac><mrow><mi>E</mi><mo stretchy="false">{</mo><mo stretchy="false">(</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>-</mo><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub><msup><mrow><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup><mo stretchy="false">}</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow></mfrac><mo>+</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544492&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544488&type= 50.46133423 9.14400005 <math display="block" id="M61"><mi>E</mi><mo stretchy="false">{</mo><msubsup><mrow><mi>R</mi></mrow><mrow><mi>i</mi></mrow><mrow><mn mathvariant="normal">3</mn></mrow></msubsup><mo stretchy="false">(</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mo stretchy="false">)</mo><mo>+</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mi>x</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544502&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544498&type= 37.08399963 5.07999992 取 <math id="M62"><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo>=</mo><mfrac><mrow><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544518&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544509&type= 12.02266598 8.04333401 ，（5）式第二项为0。第三项简化为 <math display="block" id="M63"><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi></mrow></munder><mrow><mfrac><mrow><mi>E</mi><mo stretchy="false">{</mo><mo stretchy="false">(</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>-</mo><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub><msup><mrow><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup><mo stretchy="false">}</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow></mfrac></mrow><mo>=</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544526&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544523&type= 63.41533279 9.14400005 <math id="M64"><mi mathvariant="italic">𝒩</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544915&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544912&type= 3.04800010 3.13266683 是变量 <math id="M65"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 中的离散间隔数，观测样本的期望和方差如（7）式 <math display="block" id="M66"><mi>E</mi><mo stretchy="false">{</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">}</mo><mo>=</mo><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub><mo>=</mo><mi>N</mi><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>D</mi><mo stretchy="false">{</mo><msub><mrow><mi>n</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">}</mo><mo>=</mo><mi>N</mi><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">(</mo><mn mathvariant="normal">1</mn><mo>-</mo><msub><mrow><mi>p</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544557&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544548&type= 61.97600174 4.91066647 （8）式表示忽略高阶项的 <math id="M67"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544568&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544563&type= 17.52599907 4.23333359 。 <math display="block" id="M68"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mo>≈</mo><munder><mo movablelimits="false">∑</mo><mrow><mi>i</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mo>-</mo><mfrac><mrow><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mi>l</mi><mi>o</mi><mi>g</mi><mfrac><mrow><msub><mrow><mover accent="true"><mi>n</mi><mo>¯</mo></mover></mrow><mrow><mi>i</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac><mo stretchy="false">)</mo></mrow><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>+</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mo stretchy="false">(</mo><mi mathvariant="normal">Δ</mi><mi>x</mi><mo stretchy="false">)</mo><mo>≈</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544582&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544576&type= 115.31600189 8.89000034 将变量 <math id="M69"><mi>Y</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545096&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545093&type= 2.87866688 2.79399991 分为 <math id="M70"><mi mathvariant="italic">𝒥</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544922&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544918&type= 2.53999996 3.04800010 个间隔， <math id="M71"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>Y</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544611&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544607&type= 17.27199936 4.23333359 可用下式表示 <math display="block" id="M72"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>Y</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mo>≈</mo><mi>H</mi><mo stretchy="false">(</mo><mi>Y</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544620&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544617&type= 48.76800156 8.29733276 若 x - y 平面被分为 <math id="M73"><mi mathvariant="italic">𝒩</mi><mo>×</mo><mi mathvariant="italic">𝒥</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544629&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544626&type= 10.41399956 3.38666677 个大小相等（ <math id="M74"><mi mathvariant="normal">Δ</mi><mi>x</mi><mo>×</mo><mi mathvariant="normal">Δ</mi><mi>y</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544641&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544635&type= 13.63133430 3.72533321 ）的单元，则变量 <math id="M75"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 和 <math id="M76"><mi>Y</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545096&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545093&type= 2.87866688 2.79399991 联合熵的期望为 <math display="block" id="M77"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mtext> </mtext><mo>=</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544668&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544661&type= 62.65333176 8.29733276 结合式（1）式（1）可以得到 <math display="block" id="M78"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>I</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544679&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544674&type= 68.41066742 8.29733276 根据两个连续随机变量熵的期望，可以看到互信息 <math id="M79"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544871&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544868&type= 13.63133430 3.55599999 的偏差是 <math id="M80"><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544699&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544694&type= 26.07733345 8.29733276 。 2　卡方校正算法(CSCA) 1 2.1　偏差校正 2 根据MI计算信息熵中存在的偏差会对特征选择产生影响，因此需要进行偏差校正。设存在有 <math id="M81"><mi mathvariant="italic">𝒩</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544915&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544912&type= 3.04800010 3.13266683 个间隔的连续随机变量 <math id="M82"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 和离散的类变量 <math id="M83"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 ， <math id="M84"><mi mathvariant="italic">𝒦</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544733&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544727&type= 2.87866688 2.70933342 表示类别 <math id="M85"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 的个数， <math id="M86"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 和 <math id="M87"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 的MI用信息熵表示为 <math display="block" id="M88"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>=</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>-</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo><mo>=</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544765&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544761&type= 48.09066391 3.72533321 <math display="block" id="M89"><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>-</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="italic">𝒦</mi></mrow></munderover></mstyle><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mi>H</mi></mrow><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">|</mo><mi>C</mi><mo>=</mo><mi>c</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544773&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544769&type= 46.56666946 8.55133343 结合（8）式，假设给定 <math id="M90"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 的信息熵是一个固定值，用 <math id="M91"><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544791&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544788&type= 9.99066734 4.14866638 代替 <math id="M92"><mi>E</mi><mo stretchy="false">{</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo stretchy="false">}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544800&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544795&type= 17.52599907 4.23333359 ，用 <math id="M93"><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>c</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544810&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544806&type= 3.64066648 8.04333401 估计 <math id="M94"><mi>p</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544823&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544814&type= 6.85799980 3.97933316 ，则（12）式可写为 <math display="block" id="M95"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>=</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">)</mo><mo>+</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>-</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544833&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544829&type= 48.76800156 8.29733276 <math display="block" id="M96"><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="italic">𝒦</mi></mrow></munderover></mstyle><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>c</mi><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mover accent="true"><mi>H</mi><mo>̂</mo></mover></mrow><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">|</mo><mi>C</mi><mo>=</mo><mi>c</mi><mo stretchy="false">)</mo><mo>+</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><msub><mrow><mi>n</mi></mrow><mrow><mi>c</mi></mrow></msub><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo stretchy="false">)</mo><mo>=</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544841&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544836&type= 57.48867035 9.05933285 <math display="block" id="M97"><mover accent="true"><mi>I</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>+</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>-</mo><mstyle displaystyle="true"><munderover><mo largeop="true">∑</mo><mrow><mi>c</mi><mo>=</mo><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="italic">𝒦</mi></mrow></munderover></mstyle><mrow><mfrac><mrow><msub><mrow><mi>n</mi></mrow><mrow><mi>c</mi></mrow></msub></mrow><mrow><mi>N</mi></mrow></mfrac></mrow><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><msub><mrow><mi>n</mi></mrow><mrow><mi>c</mi></mrow></msub><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>=</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544850&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544846&type= 62.06066895 9.05933285 <math display="block" id="M98"><mover accent="true"><mi>I</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒦</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544856&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544854&type= 43.26466751 8.29733276 由此可以看到 <math id="M99"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544864&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544860&type= 13.12333298 3.55599999 和 <math id="M100"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544871&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544868&type= 13.63133430 3.55599999 产生的偏差是相似的，同理可得 <math id="M101"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544879&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544875&type= 12.86933327 4.91066647 ， <math id="M102"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544885&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544882&type= 12.10733318 4.91066647 的偏差。因此， <math display="block" id="M103"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo><mo>=</mo><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo><mo>+</mo><mi>H</mi><mo stretchy="false">(</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo><mo>-</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544893&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544888&type= 54.10200119 3.72533321 <math display="block" id="M104"><mi>H</mi><mo stretchy="false">(</mo><mi>X</mi><mo>,</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo><mo>=</mo><mover accent="true"><mi>I</mi><mo>̂</mo></mover><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo><mo>-</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544902&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544896&type= 41.57133484 4.31799984 <math display="block" id="M105"><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mi mathvariant="italic">𝒦</mi></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544909&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544905&type= 29.46400070 8.29733276 若 <math id="M106"><mi mathvariant="italic">𝒩</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544915&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544912&type= 3.04800010 3.13266683 和 <math id="M107"><mi mathvariant="italic">𝒥</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544922&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544918&type= 2.53999996 3.04800010 是特征 <math id="M108"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M109"><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545021&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545018&type= 1.86266661 4.57200003 中的间隔数，可以得到 <math id="M110"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544939&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544936&type= 14.73200035 4.91066647 的偏差是 <math id="M111"><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mi mathvariant="italic">𝒦</mi></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544947&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544943&type= 29.46400070 8.29733276 。此时，在（4）式的目标函数中添加偏差项为 <math display="block" id="M112"><msub><mrow><mi>J</mi></mrow><mrow><mi mathvariant="normal">C</mi><mi mathvariant="normal">S</mi><mi mathvariant="normal">C</mi><mi mathvariant="normal">A</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒦</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>+</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544953&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544950&type= 65.10866547 8.29733276 <math display="block" id="M113"><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mo stretchy="false">|</mo><mtext> </mtext><mi mathvariant="bold-italic">f</mi><mtext> </mtext><mo stretchy="false">|</mo></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo>∈</mo><mi mathvariant="bold-italic">f</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo stretchy="false">|</mo><mi>C</mi></mrow><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mi mathvariant="italic">𝒦</mi></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>-</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544961&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544957&type= 64.85466003 9.73666668 <math display="block" id="M114"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo stretchy="false">)</mo><mo>+</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544969&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544964&type= 44.11133575 8.29733276 添加偏差后的目标函数有助于准确计算JMI，分别对 <math id="M115"><mi>R</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545878&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545874&type= 2.70933342 2.79399991 ， <math id="M116"><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545869&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545866&type= 1.52400005 2.79399991 和 <math id="M117"><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545861&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545857&type= 3.89466691 2.79399991 进行偏差校正，选出最优的特征子集。由（15）式可以发现，当 <math id="M118"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M119"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 统计独立时，相关性 <math id="M120"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544998&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24544995&type= 12.10733318 4.91066647 服从自由度为 <math id="M121"><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒦</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545006&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545002&type= 25.73866653 3.55599999 的 <math id="M122"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545412&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545408&type= 3.04800010 3.89466691 分布。当 <math id="M123"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M124"><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545021&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545018&type= 1.86266661 4.57200003 统计独立时，冗余 <math id="M125"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545030&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545025&type= 11.34533310 4.91066647 服从自由度为 <math id="M126"><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545038&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545032&type= 25.31533241 3.55599999 的 <math id="M127"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545412&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545408&type= 3.04800010 3.89466691 分布。临界值 <math id="M128"><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>R</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545051&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545047&type= 9.90600014 5.07999992 和 <math id="M129"><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>r</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545060&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545056&type= 8.80533314 5.07999992 分别如下 <math display="block" id="M130"><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>R</mi><mo stretchy="false">)</mo><mo>=</mo><mn mathvariant="normal">2</mn><mi>N</mi><mo stretchy="false">(</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn><mo stretchy="false">)</mo><mo>×</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545068&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545065&type= 44.61933517 5.07999992 <math display="block" id="M131"><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>r</mi><mo stretchy="false">)</mo><mo>=</mo><mn mathvariant="normal">2</mn><mi>N</mi><mo stretchy="false">(</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn><mo stretchy="false">)</mo><mo>×</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545074&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545071&type= 42.75666428 5.07999992 同理若给定变量 <math id="M132"><mi>Z</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545083&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545079&type= 2.37066650 2.79399991 ，两个独立随机变量 <math id="M133"><mi>X</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545089&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545086&type= 2.62466669 2.79399991 和 <math id="M134"><mi>Y</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545096&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545093&type= 2.87866688 2.79399991 的条件互信息为 <math display="block" id="M135"><mtable columnalign="left"><mtr><mtd><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>Z</mi><mo stretchy="false">)</mo><mo>=</mo></mtd></mtr><mtr><mtd><munder><mo movablelimits="false">∑</mo><mrow><mi>x</mi><mo>∈</mo><mi>X</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>y</mi><mo>∈</mo><mi>Y</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>z</mi><mo>∈</mo><mi>Z</mi></mrow></munder><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">o</mi><mi mathvariant="normal">g</mi><mfrac><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo></mrow><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo><mi>p</mi><mo stretchy="false">(</mo><mi>y</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo></mrow></mfrac></mrow></mrow></mrow><mo>=</mo></mtd></mtr><mtr><mtd><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>x</mi><mo>∈</mo><mi>X</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>y</mi><mo>∈</mo><mi>Y</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>z</mi><mo>∈</mo><mi>Z</mi></mrow></munder><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mfrac><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></mrow><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo><mi>p</mi><mo stretchy="false">(</mo><mi>y</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo><mi>P</mi><mo stretchy="false">(</mo><mi>z</mi><mo stretchy="false">)</mo></mrow></mfrac></mrow></mrow></mrow></mtd></mtr></mtable></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545104&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545099&type= 78.14733124 26.24666595 其中， <math id="M136"><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545112&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545108&type= 14.81666565 3.97933316 是联合概率， <math id="M137"><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545118&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545114&type= 9.99066734 3.97933316 和 <math id="M138"><mi>p</mi><mo stretchy="false">(</mo><mi>y</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545126&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545122&type= 9.82133293 3.97933316 是条件概率。假设 <math id="M139"><mi>q</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo><mo>≡</mo><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo><mi>p</mi><mo stretchy="false">(</mo><mi>y</mi><mo stretchy="false">|</mo><mi>z</mi><mo stretchy="false">)</mo><mi>p</mi><mo stretchy="false">(</mo><mi>z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545134&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545131&type= 48.59866714 3.97933316 ，则有 <math display="block" id="M140"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>Z</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>x</mi><mo>∈</mo><mi>X</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>y</mi><mo>∈</mo><mi>Y</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>z</mi><mo>∈</mo><mi>Z</mi></mrow></munder><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mfrac><mrow><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></mrow><mrow><mi>q</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></mrow></mfrac></mrow></mrow></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545143&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545138&type= 76.53866577 9.39799976 令 <math id="M141"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>=</mo><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545152&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545148&type= 22.60600090 4.91066647 ， <math id="M142"><msub><mrow><mi>q</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>=</mo><mi>q</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545161&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545156&type= 22.69066620 4.91066647 ，则 <math id="M143"><mi>f</mi><mo stretchy="false">(</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mfrac><mrow><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></mrow><mrow><msub><mrow><mi>q</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545169&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545165&type= 24.63800049 8.80533314 ， <math id="M144"><mi>f</mi><mo stretchy="false">(</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545179&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545175&type= 8.89000034 4.91066647 在 <math id="M145"><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>=</mo><msub><mrow><mi>q</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545185&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545182&type= 10.83733368 4.57200003 时泰勒展开，展开式第三项简化为 <math id="M146"><mfrac><mrow><mo stretchy="false">(</mo><msub><mrow><mi>p</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><mo>-</mo><msub><mrow><mi>q</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub><msup><mrow><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></mrow><mrow><mn mathvariant="normal">2</mn><msub><mrow><mi>q</mi></mrow><mrow><mn mathvariant="normal">1</mn></mrow></msub></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545196&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545189&type= 16.34066582 9.14400005 并忽略高阶项，得到下式 <math display="block" id="M147"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>Z</mi><mo stretchy="false">)</mo><mo>≈</mo><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><mi>x</mi><mo>∈</mo><mi>X</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>y</mi><mo>∈</mo><mi>Y</mi></mrow></munder><mrow><munder><mo movablelimits="false">∑</mo><mrow><mi>z</mi><mo>∈</mo><mi>Z</mi></mrow></munder><mrow><mfrac><mrow><mo stretchy="false">(</mo><mi>p</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo><mo>-</mo><mi>q</mi><msup><mrow><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></mrow><mrow><mn mathvariant="normal">2</mn><mi>q</mi><mo stretchy="false">(</mo><mi>x</mi><mo>,</mo><mi>y</mi><mo>,</mo><mi>z</mi><mo stretchy="false">)</mo></mrow></mfrac></mrow></mrow></mrow></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545209&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545202&type= 81.27999878 9.39799976 将（19）式化为标准表达式后， <math id="M148"><mi>I</mi><mo stretchy="false">(</mo><mi>X</mi><mo>;</mo><mi>Y</mi><mo stretchy="false">|</mo><mi>Z</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545217&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545213&type= 16.59466553 3.72533321 服从自由度为 <math id="M149"><mo stretchy="false">(</mo><mo stretchy="false">|</mo><mi>X</mi><mo stretchy="false">|</mo><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mo stretchy="false">|</mo><mi>Y</mi><mo stretchy="false">|</mo><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mi>Z</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545226&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545222&type= 30.47999954 3.72533321 的 <math id="M150"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545236&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545231&type= 3.04800010 3.89466691 分布。因此，对于给定类别 <math id="M151"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 且在 <math id="M152"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 和 <math id="M153"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 统计独立的情况下， <math id="M154"><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545271&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545266&type= 14.73200035 4.91066647 服从自由度为 <math id="M155"><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mi mathvariant="italic">𝒦</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545280&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545276&type= 28.70199966 3.55599999 的 <math id="M156"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545412&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545408&type= 3.04800010 3.89466691 分布。 <math id="M157"><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545299&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545294&type= 10.83733368 5.07999992 的计算如下 <math display="block" id="M158"><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi><mo stretchy="false">)</mo><mo>=</mo><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mo stretchy="false">(</mo><mn mathvariant="normal">2</mn><mo stretchy="false">)</mo><mo>×</mo><mi>I</mi><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>j</mi></mrow></msub><mo stretchy="false">|</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545308&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545304&type= 48.17533112 5.07999992 由于特征数据的 <math id="M159"><mi>R</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545878&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545874&type= 2.70933342 2.79399991 ， <math id="M160"><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545869&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545866&type= 1.52400005 2.79399991 和 <math id="M161"><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545336&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545331&type= 3.89466691 2.79399991 都服从 <math id="M162"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545412&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545408&type= 3.04800010 3.89466691 分布，将 <math id="M163"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545412&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545408&type= 3.04800010 3.89466691 分布同时用于检验离散化和特征选择，可以减少离散化的信息丢失，同时优化偏差得到最优目标函数［ 12 12 ］。因此，通过比较（15）式、（16）式和（20）式的临界值，可以做出特征选择和离散化水平决策。 2.2　算法实现及评估指标 2 首先，CSCA算法使用类信息 <math id="M164"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 计算其MI，根据其相关性分别计算每个特征的离散度 <math display="block" id="M165"><msub><mrow><mi>J</mi></mrow><mrow><mi>R</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><msubsup><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow><mrow><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msubsup><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545374&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545371&type= 28.19400024 5.24933338 其中， <math id="M166"><msubsup><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow><mrow><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></mrow></msubsup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545388&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545384&type= 3.80999994 4.31799984 表示特征 <math id="M167"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545396&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545391&type= 1.86266661 4.57200003 的离散化水平为 <math id="M168"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545405&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545401&type= 2.96333337 3.72533321 ，算法用 <math id="M169"><msup><mrow><mi>χ</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msup></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545412&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545408&type= 3.04800010 3.89466691 分布检验离散化可以选择最小的 <math id="M170"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545421&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545417&type= 2.96333337 3.72533321 ，即 <math id="M171"><msub><mrow><mi>J</mi></mrow><mrow><mi>R</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>></mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn mathvariant="normal">2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>R</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545430&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545424&type= 23.53733444 5.07999992 。根据特征对类别 <math id="M172"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 的依赖，每个特征中不同值的总数可以确定最大离散间隔 <math id="M173"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545454&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545450&type= 2.20133328 2.79399991 。因此，不同特征值有不同的离散间隔。但是，为了在最大的 <math id="M174"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545454&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545450&type= 2.20133328 2.79399991 内实现特征数据最佳的离散化水平，对所有特征使用相同的间隔。离散化水平设置和特征选择的主要步骤如算法1 算法1 。算法1 卡方校正算法(CSCA) 输入：特征数据集 <math id="M175"><mi mathvariant="bold-italic">F</mi><mo>=</mo><mo>{</mo><msub><mrow><mi>f</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>n</mi></mrow></msub><mo>}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545462&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545458&type= 23.11400032 4.14866638 ，类信息 <math id="M176"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545471&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545465&type= 2.20133328 2.37066650 ，最大间隔 <math id="M177"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545547&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545544&type= 1.86266661 2.37066650 输出：特征子集 <math id="M178"><mi mathvariant="bold-italic">f</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545753&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545749&type= 1.01600003 3.13266683 ，离散化水平 <math id="M179"><mi>D</mi><mo>=</mo><mo>{</mo><msub><mrow><mi>d</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>,</mo><mo>⋯</mo><mo>,</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>k</mi></mrow></msub><mo>}</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545490&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545486&type= 25.23066711 3.89466691 1 Begin 2 设定特征集 <math id="M180"><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>←</mo><mi mathvariant="normal">∅</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545496&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545494&type= 10.83733368 3.89466691 3 初始化 <math id="M181"><mi>j</mi><mo>=</mo><mn>2</mn></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545502&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545499&type= 6.68866634 3.13266683 ，用间隔 <math id="M182"><mi>j</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545654&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545651&type= 0.93133330 3.13266683 离散化 <math id="M183"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545661&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545657&type= 1.60866666 3.89466691 4 如果 <math id="M184"><msub><mrow><mi>J</mi></mrow><mrow><mi>R</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>></mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>R</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545519&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545516&type= 20.82799911 4.40266657 ,则 <math id="M185"><msub><mrow><mi>D</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>←</mo><msub><mrow><mi>D</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>⋃</mo><mi>j</mi><mo>;</mo><msub><mrow><mi>J</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>←</mo><msub><mrow><mi>J</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>⋃</mo><msub><mrow><mi>J</mi></mrow><mrow><mi>R</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>;</mo><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>←</mo><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>⋃</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545525&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545522&type= 55.71067047 4.14866638 5 <math id="M186"><mi>j</mi><mo>=</mo><mi>j</mi><mo>+</mo><mn>1</mn></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545532&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545530&type= 11.76866722 3.13266683 6 当 <math id="M187"><mi>j</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545654&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545651&type= 0.93133330 3.13266683 达到最大间隔 <math id="M188"><mi>d</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545547&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545544&type= 1.86266661 2.37066650 时，得到初始 <math id="M189"><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545557&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545552&type= 1.26999998 2.37066650 个特征集 <math id="M190"><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545586&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545582&type= 3.55599999 3.21733332 ，离散化水平 <math id="M191"><msub><mrow><mi>D</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545591&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545587&type= 3.80999994 3.21733332 和相关性 <math id="M192"><msub><mrow><mi>J</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545602&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545600&type= 2.70933342 3.21733332 7 <math id="M193"><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545586&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545582&type= 3.55599999 3.21733332 和 <math id="M194"><msub><mrow><mi>D</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545591&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545587&type= 3.80999994 3.21733332 根据对应的 <math id="M195"><msub><mrow><mi>J</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545602&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545600&type= 2.70933342 3.21733332 值降序排序，将最大 <math id="M196"><msub><mrow><mi>J</mi></mrow><mrow><mi>C</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545602&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545600&type= 2.70933342 3.21733332 值对应的特征和间隔定义为 <math id="M197"><msub><mrow><mi>f</mi></mrow><mrow><mn>1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545610&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545606&type= 2.03200006 3.89466691 和 <math id="M198"><msub><mrow><mi>d</mi></mrow><mrow><mn>1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545620&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545616&type= 2.87866688 3.21733332 <math id="M199"><mi mathvariant="bold-italic">f</mi><mo>←</mo><msub><mrow><mi>f</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>;</mo><mi>D</mi><mo>←</mo><msub><mrow><mi>d</mi></mrow><mrow><mn>1</mn></mrow></msub><mo>;</mo><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>←</mo><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>\</mo><msub><mrow><mi>f</mi></mrow><mrow><mn>1</mn></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545627&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545624&type= 34.37466812 4.14866638 8 初始化 <math id="M200"><mi>i</mi><mo>=</mo><mn>2</mn></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545634&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545631&type= 6.77333355 2.37066650 ， <math id="M201"><mi>T</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545707&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545704&type= 2.20133328 2.37066650 表示临界点，此时 <math id="M202"><mi>j</mi><mo>=</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>-</mo><mi>δ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545649&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545645&type= 13.37733269 3.89466691 9 用间隔 <math id="M203"><mi>j</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545654&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545651&type= 0.93133330 3.13266683 离散化特征 <math id="M204"><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545661&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545657&type= 1.60866666 3.89466691 ，如果 <math id="M205"><msub><mrow><mi>J</mi></mrow><mrow><mi mathvariant="normal">C</mi><mi mathvariant="normal">S</mi><mi mathvariant="normal">C</mi><mi mathvariant="normal">A</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>></mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>R</mi><mo stretchy="false">)</mo><mo>,</mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>r</mi><mo stretchy="false">)</mo><mo>,</mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545670&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545665&type= 44.28066635 4.40266657 则 <math id="M206"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>←</mo><mi>j</mi><mo>;</mo><msub><mrow><mi>J</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>←</mo><msub><mrow><mi>J</mi></mrow><mrow><mi>C</mi><mi>S</mi><mi>C</mi><mi>A</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545678&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545674&type= 27.00866890 4.14866638 ； <math id="M207"><mi>T</mi><mo>←</mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>R</mi><mo stretchy="false">)</mo><mo>,</mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi>r</mi><mo stretchy="false">)</mo><mo>,</mo><msubsup><mrow><mi>χ</mi></mrow><mrow><mi>C</mi></mrow><mrow><mn>2</mn></mrow></msubsup><mo stretchy="false">(</mo><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545686&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545682&type= 34.79800034 4.40266657 10 迭代计算直到 <math id="M208"><mi>j</mi><mo>=</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>+</mo><mi>δ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545693&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545690&type= 13.37733269 3.97933316 ,得到 <math id="M209"><msub><mrow><mi>J</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545700&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545697&type= 2.03200006 3.21733332 , <math id="M210"><mi>T</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545707&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545704&type= 2.20133328 2.37066650 和 <math id="M211"><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545715&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545712&type= 2.53999996 3.21733332 11 如果 <math id="M212"><msub><mrow><mi>J</mi></mrow><mrow><mi mathvariant="normal">C</mi><mi mathvariant="normal">S</mi><mi mathvariant="normal">C</mi><mi mathvariant="normal">A</mi></mrow></msub><mo>></mo><mi>T</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545723&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545719&type= 12.61533356 3.21733332 ，则 <math id="M213"><mi mathvariant="bold-italic">f</mi><mo>←</mo><mi mathvariant="bold-italic">f</mi><mo>⋃</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo>;</mo><mi>D</mi><mo>←</mo><mi>D</mi><mo>⋃</mo><msub><mrow><mi>d</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545729&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545725&type= 27.34733391 3.89466691 12 计算 <math id="M214"><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>←</mo><msub><mrow><mi>F</mi></mrow><mrow><mi>C</mi></mrow></msub><mo>\</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545737&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545734&type= 14.47799969 4.14866638 13 迭代训练 <math id="M215"><mi>i</mi><mo>=</mo><mn>3,4</mn><mo>,</mo><mo>⋯</mo><mo>,</mo><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545747&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545743&type= 17.52599907 2.79399991 ，得到特征子集 <math id="M216"><mi mathvariant="bold-italic">f</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545753&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545749&type= 1.01600003 3.13266683 和相应的离散度 <math id="M217"><mi>D</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545762&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545758&type= 2.45533323 2.37066650 14 End 在算法1 算法1 中，训练得到的特征按照相关性降序排序，选出相关性高的特征向量。为了使选择的特征和类变量 <math id="M218"><mi>C</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545779&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545775&type= 2.62466669 2.79399991 的联合性更加相关，通过偏移少量的 <math id="M219"><mi>δ</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545787&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545782&type= 1.94733346 2.79399991 改变新特征的离散化水平，若偏移过大则使特征数据将变得不相关。算法在 <math id="M220"><msub><mrow><mi>J</mi></mrow><mrow><mi mathvariant="normal">C</mi><mi mathvariant="normal">S</mi><mi mathvariant="normal">C</mi><mi mathvariant="normal">A</mi></mrow></msub></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545798&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545794&type= 7.19666624 3.72533321 大于特征选择标准的临界值时，数据特征被选择。 <math display="block" id="M221"><msub><mrow><mi>J</mi></mrow><mrow><mi mathvariant="normal">C</mi><mi mathvariant="normal">S</mi><mi mathvariant="normal">C</mi><mi mathvariant="normal">A</mi></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mi>I</mi><mo stretchy="false">(</mo><msubsup><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow><mrow><msub><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow></msub></mrow></msubsup><mo>;</mo><mi>C</mi><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒦</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>+</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545808&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545803&type= 67.05599976 8.29733276 <math display="block" id="M222"><mfrac><mrow><mn mathvariant="normal">1</mn></mrow><mrow><mfenced open="|" close="|" separators="|"><mrow><mi mathvariant="bold-italic">f</mi></mrow></mfenced></mrow></mfrac><munder><mo movablelimits="false">∑</mo><mrow><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo>∈</mo><mi mathvariant="bold-italic">f</mi></mrow></munder><mrow><mo stretchy="false">(</mo><mi>I</mi><mo stretchy="false">(</mo><msubsup><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow><mrow><msub><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow></msub></mrow></msubsup><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo stretchy="false">|</mo><mi>C</mi></mrow><mo stretchy="false">)</mo><mo>-</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mi mathvariant="italic">𝒦</mi></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo>-</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545821&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545816&type= 68.32599640 10.24466610 <math display="block" id="M223"><mi>I</mi><mo stretchy="false">(</mo><msubsup><mrow><mi>f</mi></mrow><mrow><mi>i</mi></mrow><mrow><msub><mrow><mi>d</mi></mrow><mrow><mi>j</mi></mrow></msub></mrow></msubsup><mo>;</mo><msub><mrow><mi>f</mi></mrow><mrow><mi>k</mi></mrow></msub><mo stretchy="false">)</mo><mo>+</mo><mfrac><mrow><mo stretchy="false">(</mo><mi mathvariant="italic">𝒩</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo><mo stretchy="false">(</mo><mi mathvariant="italic">𝒥</mi><mo>-</mo><mn mathvariant="normal">1</mn><mo stretchy="false">)</mo></mrow><mrow><mn mathvariant="normal">2</mn><mi>N</mi><mi mathvariant="normal">l</mi><mi mathvariant="normal">n</mi><mn mathvariant="normal">2</mn></mrow></mfrac><mo stretchy="false">)</mo></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545835&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545827&type= 46.05867004 8.29733276 通过算法1 算法1 检验所有特征的 <math id="M224"><mi>R</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545878&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545874&type= 2.70933342 2.79399991 ， <math id="M225"><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545869&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545866&type= 1.52400005 2.79399991 和 <math id="M226"><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545861&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545857&type= 3.89466691 2.79399991 来评估特征的重要程度，若某个特征的 <math id="M227"><mi>r</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545869&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545866&type= 1.52400005 2.79399991 高，则丢弃，若 <math id="M228"><mi>R</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545878&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545874&type= 2.70933342 2.79399991 和 <math id="M229"><mi mathvariant="normal">C</mi><mi mathvariant="normal">I</mi></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545890&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545882&type= 3.89466691 2.79399991 有意义，则选择该特征，最终可得到适当离散程度的最优特征子集。 CSCA在应对高维数据特征时，可以筛选出重要相关的约简特征，提高IDS的检测准确性。为了评估算法的性能，使用准确率（ A ）、召回率（ R ）、查准率（ P ）和误报率来衡量IDS的优劣。准确率为正确分类的样本数占总样本的比例，计算如下 <math display="block" id="M230"><mi>A</mi><mo>=</mo><mfrac><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">T</mi><mi mathvariant="normal">N</mi></mrow><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">T</mi><mi mathvariant="normal">N</mi><mo>+</mo><mi mathvariant="normal">F</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">F</mi><mi mathvariant="normal">N</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545902&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545898&type= 43.68799973 8.04333401 召回率表示被正确分类的样本中正常数据所占的比例，计算如下 <math display="block" id="M231"><mi>R</mi><mo>=</mo><mfrac><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi></mrow><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">F</mi><mi mathvariant="normal">N</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545914&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545909&type= 23.45266724 8.04333401 查准率表示预测为正确的样本数中，正确分类数据的占比，计算如下 <math display="block" id="M232"><mi>P</mi><mo>=</mo><mfrac><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi></mrow><mrow><mi mathvariant="normal">T</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">F</mi><mi mathvariant="normal">P</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545927&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545921&type= 22.69066620 8.04333401 以上式子中，TP表示准确分类正常数据的样本数，FP表示正常的数据中存在误报的样本数，TN表示准确分类攻击数据的样本数，FN表示攻击数据中被漏报的样本数［ 5 5 ， 13 13 ］。实际数据中 R 和 P 指标不会都达到理想值，需要加权调和平均 F 1值进行综合考虑，计算如下 <math display="block" id="M233"><mi>F</mi><mn mathvariant="normal">1</mn><mo>=</mo><mfrac><mrow><mn mathvariant="normal">2</mn><mo>⋅</mo><mi>R</mi><mo>⋅</mo><mi>P</mi></mrow><mrow><mi>R</mi><mo>+</mo><mi>P</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545944&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545935&type= 21.33600044 8.04333401 误报率是将正常的样本数据预测为攻击类别所占的比例，计算公式如下 <math display="block" id="M234"><mi mathvariant="normal">F</mi><mi mathvariant="normal">A</mi><mi mathvariant="normal">R</mi><mo>=</mo><mfrac><mrow><mi mathvariant="normal">F</mi><mi mathvariant="normal">P</mi></mrow><mrow><mi mathvariant="normal">F</mi><mi mathvariant="normal">P</mi><mo>+</mo><mi mathvariant="normal">T</mi><mi mathvariant="normal">N</mi></mrow></mfrac></math> http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545952&type= http://html.publish.founderss.cn/rc-pub/api/common/picture?pictureId=24545947&type= 28.61733437 8.04333401 高性能的IDS一定是低误报率。 3　仿真分析 1 3.1　数据集及仿真环境 2 为了研究数据特征选择的影响，选用两个常用的基准数据库NSL-KDD数据集［ 14 14 ］和UNSW-NB15数据集［ 15 15 ］进行实验仿真。NSL-KDD数据集有含125 973个训练样本和22 543个测试，有4个攻击类型，提取了41个分类特征，分为符号特征，0-1类型特征和百分比类型特征。UNSW-NB15数据集有175 341个训练数据和82 332个测试数据，包含9个攻击类，提取了44个数据特征，主要分为：时间特征，内容特征，流特征，基本特征，标记特征和其他原始特征。具体的数据特征分布如表1 表1 。 2 NSL-KDD数据集的特征 UNSW-NB15数据集的特征 2 protocol_type, service, flag, land, logged_in, root_shell, duration, dur, proto, service, state, spkts, dpkts, 2 su_attempted, is_host_login, diff_srv_rate, is_guest_login,src_bytes, sbytes,dbytes, rate, sttl, dtll, sload, dload, 2 dst_bytes, wrong_fragment, urgent, hot, num_root,num_failed_logins, sloss,dloss,sinpkt, dinpkt, sjit, djit, swin, 2 num_compromised, num_file_creations, num_shells, dst_host_count, stcpb,dtcpb, dwin,tcprtt, synack, 2 num_access_files, num_outbound_cmds, count, serror_rate, srv_count, ackdat, smean,dmean, trans_depth, 2 srv_serror_rate, rerror_rate, srv_rerror_rate, same_srv_rate, response_body_len,ct_srv_src, ct_state_ttl, 2 srv_diff_host_rate, dst_host_srv_count, dst_host_same_srv_rate, ct_dst_ltm,ct_src_dport_ltm, ct_dst_sport_ltm, 2 dst_host_diff_srv_rate, dst_host_same_src_port_rate, ct_dst_src_ltm, is_ftp_login, ct_ftp_cmd, 2 dst_host_srv_diff_host_rate, dst_host_serror_rate, ct_flw_http_mthd,ct_src_ltm, ct_srv_dst, 2 dst_host_srv_serror_rate, dst_host_rerror_rate, dst_host_srv_rerror_rate is_sm_ips_ports, attack_cat, label 合计 41个 44个为了提高模型的训练效率，在预处理中对特征数据进行归一化处理，将非数值特征数据转换为数值，使所有的特征值处于相同的数量级［ 16 16 ］，并采用10折交叉验证进行数据仿真，取10次交叉仿真结果的均值作为对算法性能的估计。 3.2　检测精度分析 2 IDS检测数据的准确率是分析算法精度的重要指标，可以评估不同特征选择方法的有效性。在两个不同的数据集中分别选择几组特征子集进行训练，可以提高算法的可靠性和稳定性。MIGM，M-DFIFS和CSCA在SVM分类器中的检测准确率结果如图2 图2 所示。根据图2 图2 所示，算法的检测准确率随着特征子集的增加而增加，达到稳定值后变化不明显。CSCA检测的准确率明显更高，平均准确率比另外两个算法提高了约5%，达到95.2%。MIGM算法受冗余特征的影响较大，在准确率达到峰值后，检测准确率随着特征的增加有明显下降。但CSCA达到收敛时选择的特征数更少，说明该算法降低了候选特征的冗余性。算法收敛时，IDS的召回率和查准率也能直接反映特征数据选择算法的检测精度。表2 表2 和表3 表3 记录了不同数据集中算法仿真的平均召回率、查准率和 F 1值。 2 特征个数 3 R 3 P 3 F 1值 CSCA MIGM M-DFIFS CSCA MIGM M-DFIFS CSCA MIGM M-DFIFS 10 0.953 0.930 0.919 0.949 0.926 0.911 0.951 0.928 0.915 20 0.960 0.942 0.934 0.963 0.944 0.932 0.961 0.943 0.933 30 0.984 0.958 0.957 0.980 0.957 0.951 0.982 0.956 0.954 40 0.879 0.753 0.679 0.977 0.945 0.947 0.925 0.838 0.971 均值 0.944 0.896 0.872 0.967 0.943 0.935 0.955 0.917 0.898 2 特征个数 3 R 3 P 3 F 1值 CSCA MIGM M-DFIFS CSCA MIGM M-DFIFS CSCA MIGM M-DFIFS 10 0.931 0.917 0.912 0.933 0.916 0.911 0.932 0.917 0.912 20 0.954 0.921 0.919 0.958 0.924 0.920 0.956 0.922 0.920 30 0.978 0.963 0.946 0.976 0.949 0.945 0.977 0.956 0.946 40 0.905 0.872 0.783 0.967 0.926 0.920 0.935 0.898 0.846 均值 0.942 0.918 0.890 0.959 0.929 0.924 0.950 0.923 0.906 在算法的仿真结果中，通过观察不同特征数在多次10折交叉验证中的综合指标的均值，可以对比不同算法检测正常数据的精度。从表2 表2 和表3 表3 中可以看到CSCA在两个数据集中的检测结果都比较稳定，相比于另外两个算法，该方法 R 和 P 的结果表现更好，且综合调整的 F 1值分别达到0.955和0.950，明显提高了模型的检测性能。 3.3　误报率分析 2 误报率是衡量IDS系统性能的关键指标，误报过高会导致系统检测性能下降。随着特征选择个数的变化，NSL-KDD数据集和UNSW-NB15数据集的误报率结果如图3 图3 所示。从图3 图3 可以看出，选择的特征过少时，特征子集不足以代表所有的类别信息，算法检测数据的误报率较高。随着特征个数的增多，算法的误报率整体呈下降趋势，但在某些区域会出现增长的情况，这主要是受到冗余特征的影响。CSCA通过偏差校正后减少了冗余特征，模型的平均误报率均比另外两个算法低3%左右，并且收敛性更快，训练选择最优的特征子集后的误报率都接近0.01，可以在数据动态变化时减少分类错误。 4　结语 1 在面临高维复杂的网络入侵数据时，传统MI特征选择算法的性能达不到理想的效果。基于动态离散化的CSCA能更好地减少特征冗余，选择最优的特征子集。利用离散化方法对所有候选特征预处理，再根据MI的特征选择标准计算相关偏差；然后通过卡方检验校正偏差目标函数，最后选出当前最重要的特征子集。仿真结果表明，CSCA精确提取重要的数据特征，使IDS具有更高的检测精度和更低的误报率。为了提高IDS对未知攻击的检测度，下一步将深入研究如何应对数据流中的漂移变化。参考文献 NANCY P ， MUTHURAJKUMAR S ， GANAPATHY S ， et al . Intrusion detection using dynamic feature selection and fuzzy temporal decision tree classification for wireless sensor networks ［J］. IET Communications ， 2020 ， 14 （ 5 ）： 888 - 895 . DOI： 10.1049/iet-com.2019.0172 http://dx.doi.org/10.1049/iet-com.2019.0172 . Intrusion detection using dynamic feature selection and fuzzy temporal decision tree classification for wireless sensor networks 刘云，肖雪 . 基于邻域维护准则的特征选择算法优化研究［J］. 重庆大学学报， 2019 ， 42 （ 3 ）： 58 - 64 . DOI： 10.11835/j.issn.1000-582X.2019.03.006 http://dx.doi.org/10.11835/j.issn.1000-582X.2019.03.006 . 基于邻域维护准则的特征选择算法优化研究 LIU Y ， XIAO X . Research on optimization of feature selection algorithm based on neighborhood preservation criterion ［J］. Journal of Chongqing University （Natural Science Edition）， 2019 ， 42 （ 3 ）： 58 - 64 . DOI： 10.11835/j.issn.1000-582X.2019.03.006 （Ch http://dx.doi.org/10.11835/j.issn.1000-582X.2019.03.006（Ch ）. Research on optimization of feature selection algorithm based on neighborhood preservation criterion ZHANG X ， MEI C L ， CHEN D G ， et al . Active incremental feature selection using a fuzzy-rough-set-based information entropy ［J］. IEEE Transactions on Fuzzy Systems ， 2020 ， 28 （ 5 ）： 901 - 915 . DOI： 10.1109/TFUZZ.2019.2959995 http://dx.doi.org/10.1109/TFUZZ.2019.2959995 . Active incremental feature selection using a fuzzy-rough-set-based information entropy WANG X Z ， GUO B ， SHEN Y ， et al . Input feature selection method based on feature set equivalence and mutual information gain maximization ［J］. IEEE Access ， 2019 ， 7 ： 151525 - 151538 . DOI： 10.1109/ACCESS.2019.2948095 http://dx.doi.org/10.1109/ACCESS.2019.2948095 . Input feature selection method based on feature set equivalence and mutual information gain maximization WEI G F ， ZHAO J ， FENG Y L ， et al . A novel hybrid feature selection method based on dynamic feature importance ［J］. Applied Soft Computing ， 2020 ， 93 ： 106337 . DOI： 10.1016/j.asoc.2020.106337 http://dx.doi.org/10.1016/j.asoc.2020.106337 . A novel hybrid feature selection method based on dynamic feature importance FAHY C ， YANG S X . Dynamic feature selection for clustering high dimensional data streams ［J］. IEEE Access ， 2019 ， 7 ： 127128 - 127140 . DOI： 10.1109/ACCESS.2019.2932308 http://dx.doi.org/10.1109/ACCESS.2019.2932308 . Dynamic feature selection for clustering high dimensional data streams TRAN B ， XUE B ， ZHANG M J . A new representation in PSO for discretization-based feature selection ［J］. IEEE Transactions on Cybernetics ， 2018 ， 48 （ 6 ）： 1733 - 1746 . DOI： 10.1109/TCYB.2017.2714145 http://dx.doi.org/10.1109/TCYB.2017.2714145 . A new representation in PSO for discretization-based feature selection ATIENCIA R S ， WEBER R . Dynamic rough-fuzzy support vector domain description for outlier detection ［C］// 2018 IEEE International Conference on Fuzzy Systems （FUZZ-IEEE） . New York ： IEEE Press ， 2018 ： 1 - 8 . DOI： 10.1109/FUZZ-IEEE.2018.8491618 http://dx.doi.org/10.1109/FUZZ-IEEE.2018.8491618 . Dynamic rough-fuzzy support vector domain description for outlier detection 李郅琴，杜建强，聂斌，等 . 特征选择方法综述［J］. 计算机工程与应用， 2019 ， 55 （ 24 ）： 10 - 19 . DOI： 10.3778/j.issn.1002-8331.1909-0066 http://dx.doi.org/10.3778/j.issn.1002-8331.1909-0066 . 特征选择方法综述 LI Z Q ， DU J Q ， NIE B ， et al . Summary of feature selection methods ［J］. Computer Engineering and Applications ， 2019 ， 55 （ 24 ）： 10 - 19 . DOI： 10.3778/j.issn.1002-8331.1909-0066（Ch http://dx.doi.org/10.3778/j.issn.1002-8331.1909-0066（Ch ）. Summary of feature selection methods YAN B H ， HAN G D . Effective feature extraction via stacked sparse autoencoder to improve intrusion detection system ［J］. IEEE Access ， 2018 ， 6 ： 41238 - 41248 . DOI： 10.1109/ACCESS.2018.2858277 http://dx.doi.org/10.1109/ACCESS.2018.2858277 . Effective feature extraction via stacked sparse autoencoder to improve intrusion detection system LUO C ， LI T R ， YI Z . An incremental feature selection approach based on information entropy for incomplete data ［C］// 2019 IEEE International Conference on Dependable， Autonomic and Secure Computing . New York ： IEEE Press ， 2019 ： 483 - 488 . DOI： 10.1109/dasc/picom/cbdcom/cyber http://dx.doi.org/10.1109/dasc/picom/cbdcom/cyber scitech.2019. 00097 . An incremental feature selection approach based on information entropy for incomplete data 陈谌，梁雪春 . 基于基尼指标和卡方检验的特征选择方法［J］. 计算机工程与设计， 2019 ， 40 （ 8 ）： 2342 - 2345 . DOI： 10.16208/j.issn1000-7024.2019.08.040 http://dx.doi.org/10.16208/j.issn1000-7024.2019.08.040 . 基于基尼指标和卡方检验的特征选择方法 CHEN C ， LIANG X C . Feature selection method based on Gini index and chi-square test ［J］. Computer Engineering and Design ， 2019 ， 40 （ 8 ）： 2342 - 2345 . DOI： 10.16208/j.issn1000-7024.2019.08.040（Ch http://dx.doi.org/10.16208/j.issn1000-7024.2019.08.040（Ch ）. Feature selection method based on Gini index and chi-square test GAO X W ， SHAN C ， HU C Z ， et al . An adaptive ensemble machine learning model for intrusion detection ［J］. IEEE Access ， 2019 ， 7 ： 82512 - 82521 . DOI： 10.1109/ACCESS.2019.2923640 http://dx.doi.org/10.1109/ACCESS.2019.2923640 . An adaptive ensemble machine learning model for intrusion detection NGUYEN M T ， KIM K . Genetic convolutional neural network for intrusion detection systems ［J］. Future Generation Computer Systems ， 2020 ， 113 ： 418 - 427 . DOI： 10.1016/j.future.2020.07.042 http://dx.doi.org/10.1016/j.future.2020.07.042 . Genetic convolutional neural network for intrusion detection systems MOUSTAFA N ， SLAY J . UNSW-NB15： A comprehensive data set for network intrusion detection systems （UNSW-NB15 network data set）［C］// 2015 Military Communications and Information Systems Conference （MilCIS） . New York ： IEEE Press ， 2015 ： 1 - 6 . DOI： 10.1109/MilCIS.2015.7348942 http://dx.doi.org/10.1109/MilCIS.2015.7348942 . UNSW-NB15： A comprehensive data set for network intrusion detection systems （UNSW-NB15 network data set）宋勇，蔡志平 . 大数据环境下基于信息论的入侵检测数据归一化方法［J］. 武汉大学学报（理学版）， 2018 ， 64 （ 2 ）： 121 - 126 . DOI： 10.14188/j.1671-8836.2018.02.004 http://dx.doi.org/10.14188/j.1671-8836.2018.02.004 . 大数据环境下基于信息论的入侵检测数据归一化方法 SONG Y ， CAI Z P . Normalized method of intrusion detection data based on information theory in big data environment ［J］. Journal of Wuhan University （Natural Science Edition）， 2018 ， 64 （ 2 ）： 121 - 126 . DOI： 10.14188/j.1671-8836.2018.02.004（Ch http://dx.doi.org/10.14188/j.1671-8836.2018.02.004（Ch ）. Normalized method of intrusion detection data based on information theory in big data environment 10.14188/j.1671-8836.2021.0111 TP311 A 1671-8836（2022）01-0065-08 LIU Yun，ZHENG Wenfeng，ZHANG Yi.Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection ［J］. J Wuhan Univ （ Nat Sci Ed ），2022，68（1）：65-72. DOI： 10.14188/j.1671-8836.2021.0111（Ch http://dx.doi.org/10.14188/j.1671-8836.2021.0111(Ch ）. LIU Yun，ZHENG Wenfeng，ZHANG Yi.Optimization of Chi-Square Correction Algorithm for Intrusion Detection Feature Selection ［J］.J Wuhan Univ （Nat Sci Ed），2022，68（1）：65-72. DOI：10.14188/j.1671-8836.2021.0111（Ch）. 刘云,郑文凤,张轶.卡方校正算法对入侵检测特征选择的优化[J].武汉大学学报（理学版）,2022,68(1):65-72. DOI:10.14188/j.1671-8836.2021.0111 刘云,郑文凤,张轶.卡方校正算法对入侵检测特征选择的优化[J].武汉大学学报（理学版）,2022,68(1):65-72. DOI:10.14188/j.1671-8836.2021.0111 国家自然科学基金（61761025），云南省重大科技专项计划项目资助（202002AD080002） 2021-05-06 2022-02-24 2022-02-22 武汉大学学报（理学版） 1 68

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Normalized Method of Intrusion Detection Data Based on Information Theory in Big Data Environment

Sentiment Orientation of Text: bfsmPMI-SVM

Intrusion Detection Model for Web Applications Based on Alive Entropy

Image Steganalysis Based on Mutual Information and Feature Fusion

Related Author

Zhiping CAI

Yong SONG

Juan DENG

Xin YE

Zhe LI

Jinshuo LIU

Jiamin CHEN

MO Xiuliang

Related Institution

College of Computer, National University of Defense Technology

Department of Engineering Technology, Hunan Vocational College for Nationalities

School of Computer,Wuhan University

International School of Software,Wuhan University

Key Laboratory of Computer Vision and System,Ministry of Education,Tianjin University of Technology

Bulk Citation

⁰