Although conformance checking is great at detecting process deviations, it still poses challenges that hinders adoption in auditing practice. A major challenge is that in real life a large number of deviating cases is often detected of which only a small amount are true anomalies and thus of real interest to auditors. The number of deviations are often too large to inspect one by one, which explains why auditing requires a sample-based approach. This paper contributes to the research on the practical feasibility of continuous auditing and studies the potential of weak supervision to classify deviations into anomalies and exceptions, allowing auditors to do a full-population analysis of the identified deviations. The Snorkel framework is applied which uses a set of imperfect domain expert rules to classify the set of deviations into anomalies and exceptions. A controlled and artificial experiment has been set up to explore the relation between the performance of this approach and the number and quality of domain expert rules. The results demonstrate the potential of this approach as a limited number of medium to high quality domain expert rules succeeds to classify deviations with acceptable accuracy.