chunked/nltk_chunking/tasks_token_classification/chunk_28.txt

import evaluate
seqeval = evaluate.load("seqeval")

Get the NER labels first, and then create a function that passes your true predictions and true labels to [~evaluate.EvaluationModule.compute] to calculate the scores:

import numpy as np
labels = [label_list[i] for i in example[f"ner_tags"]]
def compute_metrics(p):
     predictions, labels = p
     predictions = np.argmax(predictions, axis=2)

     true_predictions = [
         [label_list[p] for (p, l) in zip(prediction, label) if l != -100]
         for prediction, label in zip(predictions, labels)
     ]
     true_labels = [
         [label_list[l] for (p, l) in zip(prediction, label) if l != -100]
         for prediction, label in zip(predictions, labels)
     ]
     results = seqeval.compute(predictions=true_predictions, references=true_labels)
     return {
         "precision": results["overall_precision"],
         "recall": results["overall_recall"],
         "f1": results["overall_f1"],
         "accuracy": results["overall_accuracy"],
     }

Your compute_metrics function is ready to go now, and you'll return to it when you setup your training.