You can read more about the optimizations in the original pull request | |
Combining Starcoder and Flash Attention 2 | |
First, make sure to install the latest version of Flash Attention 2 to include the sliding window attention feature. |
You can read more about the optimizations in the original pull request | |
Combining Starcoder and Flash Attention 2 | |
First, make sure to install the latest version of Flash Attention 2 to include the sliding window attention feature. |