The input embeddings are passed through multiple encoder layers to output some final hidden states.