CLIP uses a ViT like transformer to get visual features and a causal language model to get the text
features.