3294 位用户此时在线
- 本站自动实时分享网络热点
- 24小时实时更新
- 所有言论不代表本站态度
- 欢迎对信息踊跃评论评分
- 评分越高，信息越新，排列越靠前
I'm more and more confident that tokenization will be gone.
Humans don't think in "tokens".
Tokens are hardcoded abstractions in LLMs that lead to weird behavior: LLMs can solve PhD-level math questions but cannot answer "Is 9.9 > 9.11?"
Meta is shifting LLMs to LCMs (Large
时政
(
twitter.com
)
I'm more and more confident that tokenization will be gone.
Humans don't think in "tokens".
Tokens are hardcoded abstractions in LLMs that lead to weird behavior: LLMs can solve PhD-level math questions but cannot answer "Is 9.9 > 9.11?"
Meta is shifting LLMs to LCMs (Large Concept Models), changing "next-token prediction" to "next-concept prediction". A "concept" is treated as a sentence, representing an abstract idea or action.
Two weeks ago, Meta also published the "Byte Latent Transformer", a new architecture that processes raw bytes grouped into patches instead of relying on tokenization.
My intuition says LCM is probably better at reasoning and modalities.
点击图片查看原图