Can I run this locally?
Hi,
Being a beginner on this, is it possible to run this model locally, with the current training, on my own computer?
It takes forever to start the space each time I want to use it - and it is really good!
If yes, how do I do it?
Hello,
Yes, it is possible! However, you probably might want to have a GPU with at least 12 GB vram (preferably Nvidia). You could get around with 6GB vram, if you use a smaller variant of TrOCR model. The models can be run on a CPU as well, but the processing times are a bit slow. It could take up to a minute or more to process, whereas with GPU it takes roughly ten seconds.
There is a minimal guide to how to run locally provided by huggingface in the upper right corner of the demo. When pressing the three dots, you can see “Run locally” and there you can see two options. Either you can try to run with docker or by cloning with git. If you are not familiar with docker, I suggest you try with cloning with git. In order to that work you need to have Python and Git installed on your computer.
Best regards,
National Archives of Finland AI team
Thank you for your response!
The tool is truly awesome and extremely helpful for my genealogy research!
Will it be made available on any other platform, or is this the online spot where I can use it (apart from locally, but I don't think my hardware is good enough...)
Kindest regards,
Bengt
Sweden
Hi,
Nice to hear that it helps. Unfortunately, we do not have plans to release this on other platforms:(
Regards,
National Archives of Finland AI team
I understand. It is an amazing tool! Better than any other tools for analyzing handwritten text from 1600-.
After the summer, I'll see if I can buy a better computer. :)
If I install it, will it be able to do all that the demo does? I guess I mean if it includes all the handwritten learning.
Kind regards,
Bengt
Yes, if you run the code locally, it will open up the same demo as in here. It also downloads the same models from here on to your local pc, so the results will be the same.
I really love the model, thank you, great work! I have been running the model on a local machine on a Nvidia GPU with only 4 GB of VRAM and it works rather well. I do the segmentation separately and then clear the GPU VRAM and then I run the actual OCR in batches of 4 lines at a time (could do more, but that batch size seems to bring the best overall performance). The performance is not that bad, but obviously not great. It takes about 30 seconds per image or less than 0.5 seconds per line. The VRAM seems to stay below 2 GB all the time, so I have still some room. My laptop itself has plenty or RAM (32GB). So I have obviously not encountered any issues on that side. I have implemented the code to run within my own python code, and thus have not tried to use the whole package and its user interface.