The traditional SVM does not support incremental learning. And the traditional training method of SVM is not working when the amount of training samples are so large that they can not be put into the RAM of computer. In order to solve this problem and improve the speed of training SVM, the natural characteristics of SV are analyzed in this paper. An incremental learning algorithm (I-SVM) for SVM with discarding part of history samples is presented. The theoretical analysis and experimental results show that this algorithm can not only speed up the training process, but also reduce the storage cost, while the classification precision is also guaranteed.