Performance diference versus unquantized 8b param version?

by ashercn97 - opened Jun 20

Jun 20

Hey! I was just wondering how much the performance degraded with the quantization? I didn't see anything on the page (correct me if I'm wrong!).

Thanks!

liamcripwell

NuMind org Jun 20

•

edited Jun 20

Hi :) on average we see ~85% the performance of the full precision model (in zero-shot). For many tasks it performs close to parity if they aren't very complicated or heavily visual. We will probably add more details to the card at some point.

ashercn97

Jun 22

Awesome!

ashercn97 changed discussion status to closed Jun 22

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment