vectorization. For example, assuming a vocabulary size of 10,000 and an average token of length 6 unicode characters, with each character being 2 bytes, the vocabulary would take up (10,000 * 6 * 2 = 120 NLP problems, we demonstrated a reproducible path to pre-train embeddings on a word, character, unicode etc. level tokenization and store them in an embedding table for easy lookup and use later. For many