This chapter identifies the differences in dimensionality reduction based on features, cases, and reduction of value techniques. It explains the advantages of data reduction in the preprocessing phase of a data-mining process. The chapter compares entropy-based technique and principal component analysis for feature ranking. For really large data sets, there is an increased likelihood that an intermediate, additional step—data reduction—should be performed prior to applying the data-mining techniques. Feature selection algorithms typically fall into two categories: feature ranking and subset selection. Examples of additional methodologies in feature extraction include factor analysis, independent component analysis, and multidimensional scaling. The core of the Relief algorithm is to estimate the quality of features according to how well their values distinguish between samples close to each other. A method for unsupervised feature selection or ranking based on entropy measure is a relatively simple technique; but with a large number of features its complexity increases significantly.