Impact statement: Impact on GNN research . Our work brings in many valuable mindsets to the field of GNN research. For example, we fully adopt the principle of controlling model complexity when comparing different models, which is not yet adopted in most GNN papers. We focus on finding guidelines / principles when designing GNNs, rather than particular GNN instantiations. We emphasize that the best GNN designs can drastically differ across tasks (the state-of-the-art GNN model on one task may have poor performance on other tasks). We thus propose to evaluate models on diverse tasks measured by quantitative similarity metric. Rather than criticizing the weakness of existing GNN architectures, our goal is to build a framework that can help researchers understand GNN design choices when developing new models suitable for different applications. Our approach serves as a tool to demonstrate the innovation of a novel GNN model ( e.g. , in what kind of design spaces / task spaces, a proposed algorithmic advancement is helpful), or a novel GNN task ( e.g. , showing that the task is not similar to any existing tasks thus calls for new challenges of algorithmic development). Impact on machine learning research . Our approach is in fact applicable to general machine learning model design. Specifically, we hope the proposed controlled random search technique can assist fair evaluation of novel algorithmic advancements. To show whether a certain algorithmic advancement is useful, it is important to sample random model-task combinations, then investigate in what scenarios the algorithmic advancement indeed improves the performance. Additionally, the proposed task similarity metric can be used to understand similarities between general machine learning tasks, e.g. , classification of MNIST and CIFAR-10. Our ranking-based similarity metric is fully general, as long as different designs can be ranked by their performance. Impact on other research domains . Our framework provides an easier than ever support for experts in other disciplines to solve their problems via GNNs. Domain experts only need to provide properly formatted domain-specific datasets, then recommended GNN designs will be automatically picked and applied to the dataset. In the fastest mode, anchor GNN models will be applied to the novel task in order to measure its similarity with known GNN tasks, where the corresponding best GNN designs have been saved. Top GNN designs in the tasks with high similarity to the novel task will be applied. If computational resources permitted, a full grid search / random search over the design space can also be easily carried out to the new task. We believe this pipeline can significantly lower the barrier for applying GNN models, thus greatly promote the application of GNNs in other research domains. Impact on the society . As is discussed above, given its clarity and accessibility, we are confident that our general approach can inspire novel applications that are of high impact to the society. Additionally, its simplicity can also provide great opportunities for AI education, where students can learn from SOTA deep learning models and inspiring applications at ease.