microsoft

/

bitnet-b1.58-2B-4T

like 420

Follow

Microsoft 11.3k

same issue

To help me understand the issue better, could you please provide more information about how you're using the model? Specifically, it would be helpful to see the relevant code snippets or scripts.
Also, to ensure we're on the same page, can you confirm that you're using the version of transformers installed directly from the GitHub repository using this command?

pip install git+https://github.com/shumingma/transformers.git

Besides the version of transformers above, please ensure NOT to use trust_remote_code=True from the from_pretrained call.

@frontierai same issue i also facing I am running using docker like below commends

docker run --rm \
             --privileged=true \
             --shm-size=4g \
             -p 8000:8000 \
             vllm-cpu-env \
             --model=microsoft/bitnet-b1.58-2B-4T \
             --trust-remote-code
             --dtype=bfloat16 
      ```