Knowledge base (KB) refinement is a suitable technique to support expert system (ES) validation. When used for validation, KB refinement should be guided not only by the number of errors to solve but also by the importance of those errors. Most serious errors should be solved first, even causing other errors of lower importance but assuring a neat validity gain. These are the bases for IMPROVER, a KB refinement tool designed to support ES validation. IMPROVER refines ES for medical diagnosis with this classification of error importance: false negative > false positive > ordering mismatch. IMPROVER has been used to support the validation of PNEUMON-IA, a real ES on the medical domain. After refinement, the ES validity has increased substantially. Detailed evidence of this improvement is provided, as well as examples of how the refinement process was performed. © 1996 John Wiley & Sons, Inc.