File size: 391 Bytes
5fa1a76 |
1 2 3 4 5 6 7 8 9 10 |
For ease of use, a generator is also possible:
thon
from transformers import pipeline
pipe = pipeline("text-classification")
def data():
while True:
# This could come from a dataset, a database, a queue or HTTP request
# in a server
# Caveat: because this is iterative, you cannot use num_workers > 1 variable
# to use multiple threads to preprocess data. |