In GPT-3's training data Wikipedia is only 3% of the mix (3 billion tokens), yet the model saw it 3.4 times, versus less than once (0.44) for the bulk Common Crawl web data
translated by AI, reviewed by the editors
· Media & information · Science
- technology
- Wikipedia after AI
- source
- arxiv.org ↗
¶ c8nn7qzx9