Transformer+Web text → Large language model
merge · 11 Jun 2018
OpenAI GPT: a transformer pre-trained on text — openai.com ↗
Science
⚑ Backed by
- Work & jobs / Andrej Karpathy: “Around 6B people (~75% of the population) have barely come in contact with LLMs, while ~5,000 have access to frontier-grade systems internally”
- Science / Nick Cammarata: “The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread … Neural networks are already clean”
¶ c9p9gfxz6