Many benchmark experiments of virtual screening techniques are biased by the composition of the benchmark datasets. Such benchmark dataset bias usually has two main causes:

1. The actives are to similar to each other ("analogue bias" [1]) regarding low-dimensional molecular properties.
2. The actives are too dissimilar from the decoys ("artificial enrichment" [2]) also regarding low-dimensional molecular properties.

Research from our lab recently introduced two tools for the statistical analysis of such benchmark dataset bias in chemical datasets:

1. "Simple Descriptors" are a vectorized form of a compound's sum-formula, the respective counts of H-Bond acceptors, H-Bond donors, chiral centers and ring systems and the compounds logP. These descriptors capture all low-dimensional molecular properties associated with analogue bias and artificial enrichment.[3]

2. Refined Nearest Neighbor analysis based on a simple descriptor representation of the respective datasets can be utilized to characterize the topology of the datasets, i.e. their clustering behavior and their relative position in chemical space.[3]

It was shown, that benchmark dataset bias is associated with "clumpy" topologies, i.e. datasets that form clusters in descriptor space and are separated from the decoys. Refined Nearest Neighbor analysis also provides tools for quantifying the "clumpiness" of datasets: The scalar ΣS has values smaller than 0, whenever a dataset is clumpy and values larger that 0, if it is dispersed. Values of ΣS near 0 indicate the special state of spatial randomness, i.e. a spatially randon distribution of actives and decoys in simple descriptor space.

It is obvious, that a collection of benchmark datasets, in which all datsets exhibit spatial randomness in simple descriptor space would be especially advantageous for validating VS methods.

1. A random distribution of actives and decoys regarding simple descriptors prevents low-dimensional molecular properties from distorting validation results.
2. Datasets that do not favor ligand based virtual screening methods because of trivial features of the molecules used for benchmarking greatly facilitate the comparison of ligand based VS methods and docking applications.

Based on these findings, the MUV collection of benchmark datasets was designed based on PubChem bioactivity data.[4] PubChem has several advantages for the design of VS benchmark datasets:

1. All data, including molecular structures and bio-assay readouts is publicly available.
2. Compounds tested inactive in the bio-assays are available as experimentally validated decoys.
3. The compound sets tested in the NIH roadmap effort are very diverse.
4. The majority of tested compounds is drug-like.

However, PubChem also has downsides as a datasource: Due to its origin from HTS, the bioactivity data in PubChem is notoriously noisy. Therefore, the MUV design workflow first ensures the maximum level of confidence by purging all bioactivity datasets from compounds, whose assigned bioactivity might be subject to doubts. The remaining datasets are obtimized spatially to generate the final MUV datasets.