Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D

arXiv:2607.16072v1 Announce Type: new Abstract: While large language models (LLMs) can solve advanced reasoning problems in seconds, we show that even frontier models fail to perform a much simpler operation: exactly copying an input string that lies well within their context windows. We attribute this failure to positional encodings in Transformer architectures, whose inductive bias favors copying through a shortcut based on matching local contexts rather than carefully locating the correspondi...