en
Book chapter
Open access
English

Asymmetric and sample size sensitive entropy measures for supervised learning

Published inAdvances in Intelligent Information Systems, Editors Ras Zbiniew and Li-Shiang Tsay, p. 26-42
PublisherSpringer
Collection
  • Studies in Computational Intelligence; 265
Publication date2010
Abstract

Many algorithms of machine learning use an entropy measure as optimization criterion. Among the widely used entropy measures, Shannon's is one of the most popular. In some real world applications, the use of such entropy measures without precautions, could lead to inconsistent results. Indeed, the measures of entropy are built upon some assumptions which are not fulfilled in many real cases. For instance, in supervised learning such as decision trees, the classification cost of the classes is not explicitly taken into account in the tree growing process. Thus, the misclassification costs are assumed to be the same for all classes. In the case where those costs are not equal on all classes, the maximum of entropy must be elsewhere than on the uniform probability distribution. Also, when the classes don't have the same a priori distribution of probability, the worst case (maximum of the entropy) must be elsewhere than on the uniform distribution. In this paper, starting from real world problems, we will show that classical entropy measures are not suitable for building a predictive model. Then, we examine the main axioms that define an entropy and discuss their inadequacy in machine learning. This we lead us to propose a new entropy measure that possesses more suitable proprieties. After what, we carry out some evaluations on data sets that illustrate the performance of the new measure of entropy.

Citation (ISO format)
ZIGHED, Djamel A., RITSCHARD, Gilbert. Asymmetric and sample size sensitive entropy measures for supervised learning. In: Advances in Intelligent Information Systems. [s.l.] : Springer, 2010. p. 26–42. (Studies in Computational Intelligence)
Main files (1)
Book chapter
accessLevelPublic
Identifiers
  • PID : unige:5381
ISBN978-3-642-05182-1
621views
772downloads

Technical informations

Creation2010/03/09 16:40:00
First validation2010/03/09 16:40:00
Update time2023/03/14 15:24:50
Status update2023/03/14 15:24:50
Last indexation2024/05/02 11:29:37
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack