← Back to People
Author of "Why Stacking Sliding Windows Can't See Very Far" in Guangxuan Xiao's Blog
1
Mentions
1
Articles
1
Outlets
Associated Topics
Coverage Patterns
How media typically covers Guangxuan Xiao
Article Types
- research1
Media Angles
- technical1
Associated AI Models
GPT-OSS1Mistral1Llama-41Gemma 31
Articles
Most recent firstArticles Written
Guangxuan Xiao as author
Why Stacking Sliding Windows Can't See Very Far
Guangxuan Xiao's BlogresearchneutralMar 1, 2026
Stacking sliding window attention layers cannot achieve their theoretical context length due to information dilution and exponential barriers from residual connections, limiting effective memory to ~1,500 tokens despite theoretical capacity of 100,000+.
“Author of "Why Stacking Sliding Windows Can't See Very Far" in Guangxuan Xiao's Blog”
LLMsAI ResearchFoundation ModelsNLP