I was surprised by a talk Yejin Choi (an NLP expert) gave yesterday in Berkeley, on some surprising weaknesses of GPT4:
As many humans know, 237*757=179,409
but GPT4 said 179,289.
For the easy problem of multiplying two 3 digit numbers, they measured GPT4 accuracy being only 59% accuracy on 3 digit number multiplication. Only 4% on 4 digit number multiplication and zero on 5x5. Adding scratchpad helped GPT4 but only to 92% accuracy on multiplying two 3 digit numbers.
Even more surprisingly, finetuning GPT3 on 1.8m examples of 3 digit multiplication still only gives 55 percent test accuracy (in distribution).
ÂŻ\_(â︿â)_/ÂŻ
So whats going on? Multiplication is algorithmically very challenging (as are less known algorithmic problems).
The authors hypothesize that Transformers have a hard time because they learn linear patterns that they can memorize, maybe compose, but not generally reason with. The paper raises interesting theoretical and practical questions on understanding what Transformers can learn.
The paper
"Faith and Fate: Limits of Transformers on Compositionality" says:
"Our empirical findings suggest that Transformers
solve compositional tasks by reducing multi-step compositional reasoning into
linearized subgraph matching, without necessarily developing systematic problem solving skills"