The limitations of decision trees and automatic learning in real world medical decision making.
The decision tree approach is one of the most common approaches in automatic learning and decision making. It is popular for its simplicity in constructing, efficient use in decision making and for simple representation, which is easily understood by humans. The automatic learning of decision trees and their use usually show very good results in various "theoretical" environments. The training sets are usually large enough for learning algorithm to construct a hypothesis consistent with the underlying concept. But in real life it is often impossible to find the desired number of training objects for various reasons. The lack of possibilities to measure attribute values, high cost and complexity of such measurements, unavailability of all attributes at the same time are the typical representatives. There are different ways to deal with some of these problems, but in a delicate field of medical decision making, we cannot allow ourselves to make any inaccurate decisions. We have measured the values of 24 attributes before and after the 82 operations of children in age between 2 and 10 years. The aim was to find the dependencies between attribute values and a child's predisposition to acidemia--the decrease of blood's pH. Our main interest was in discovering predisposition to two forms of acidosis, the metabolic acidosis and the respiratory acidosis, which can both have serious effects on child's health. We decided to construct different decision trees from a set of training objects, which was complete (there were no missing attribute values), but on the other hand not large enough to avoid the effect of overfitting. A common approach to evaluation of a decision tree is the use of a test set. In our case we decided that instead of using a test set, we ask medical experts to take a closer look at the generated trees. They examined and evaluated the decision trees branch by branch. Their comments on the generated trees can be found in this paper. The comments show, that trees generated from available training set mainly have surprisingly good branches, but on the other hand some are very "stupid" and no medical explanation could be found. Thereafter we can conclude, that the decision tree concept and automatic learning can be successfully used in real world situations, constrained with the real world limitations, but they should be used only with the guidelines of appropriate medical experts.