Architecture

Long Context

Long-context models have extended context windows (100K+ tokens), enabling processing of entire books, long conversations, or large documents. Extended context is enabled by efficient attention mechanisms (Flash Attention, sparse patterns, KV cache optimizations). Long context opens new applications: comprehensive RAG, detailed document analysis, and extended conversations. However, longer sequences increase latency and memory. Effective long-context use requires good retrieval or summarization. Quality of very long-range reasoning is an open question. Long context is valuable but not always necessary.