Among the most important gains, according to Meta, originates from the usage of a tokenizer that has a vocabulary of 128,000 tokens. In the context of LLMs, tokens is usually a number of people, whole words, and even phrases. AIs stop working human enter into tokens, then use their vocabularies