A Robust and Adaptive Thresholding Method for Non-Parametric Statistical Tests

A decision stream in a hypothesis testing problem may be obtained by comparing a received data stream to a threshold. The threshold may be generated from a noise subset of the data stream based on certain characteristics of observed data. The probability distribution of the noise subset along with characteristics of the data stream may be used in determining the threshold. The determination of the threshold may be adaptive to maintain a user prescribed probability of error. A decision state machine may be used to control the manner in which noise characteristics are used to guide the hypothesis testing, increase the detection rate, and reduce the probability of error. The decision state machine evaluates the decision stream to determine falsely classified data samples and reclassify such items appropriately. The decision state machine may filter the decision stream to ensure that a lower decision error rate is achieved.