Each tokenizer works differently but the underlying mechanism remains the same.