Indiana University cognitive scientist Douglas Hofstadter had a lifelong acquaintance with and admiration for the Swedish language and with the help of Swedish friends, became conversant with it. That led him in turn to try an experiment on machine translation programs such as Google Translate and DeepL. At Inference Review, he tells us, “although — or perhaps because — these programs have improved by leaps and bounds over the past few years, I greatly enjoy discovering and poking fun at their many unpredictable weaknesses.”

Thus the author of author of Gödel, Escher, Bach (1979) constructed a paragraph of pure nonsense in made-up Swedish, something like Lewis Carroll’s “Jabberwocky,” which plays around similarly with English:

All mimsy were the borogoves,

And the mome raths outgrabe.

Most of the words don’t exist and the sentence has no meaning. But it is phrased as grammatically correct English verse. Here’s Hofstadter’s similar treatment of a paragraph derived ultimately from Swedish in similar way:

Sista mällingen frädde jag mina skvallrutor på en eller två djyvelräckiga drammsniggor, men det knackraddes ogrinligt vålsent för spjulingarna, och sen med de inluppta trämplissorna blybbade det otvickligt. Kvältsmusarna tryckades för tjabbriga i och till de spyrlösa fjöllsidorna, och vi trömmades välmåset med våra innansätsingar. Sen över det ledvist häppligt fliknandet lömnade vi, och flingade den vålfredsskjutliga hjornarens knövboltar. Framligtvist inklågerade jag mig fräsinglöst om att knupa hur de där inblitsna värstningstjuperierna skylle prälla. Vi var yppligt klickforniga över att ingen brusspil eller mänskivel spjällde klotiskt, och att inga förförlingsmässa blyxor näbblades. Det var så att säga hultsamt och multsamt, och även ypperligen gnålfritt, så att jag truppade nästan frukkigt med en gnisskurlig sopperfläcke. Kan man särvist åtvända det där som en tvörlig frösénsstjukare, eller sjöll man brysa en döppel härnäss för att vidstycka sig med rejella äktningsfylla snippflyggor? Det där vulle jag inte bjärkna, inte alls.

Douglas Hofstadter, “Wacky Jabber” at Inference Review (September 2022)

On August 15, he fed it to Google Translate and asked for the English:

In the last mælling, I spread my gossip panes on one or two devilish drum snips, but it was knocked unbelievably late for the children, and then with the slivers of wood thrown in, it remained undisturbed. The voles were pressed too fast into and to the barren fjöllsides, and we were thoroughly beaten with our insinuations. Then over the sadly fortunate flapping, we slapped, and flake the knuckle-bolts of the peace-sick horned. Of course, I complained incessantly about how those blinded worst-case idiots were supposed to act. We were very happy that no noise arrow or man’s disc sputtered, and that no seduction fair lights were snagged. It was, so to speak, merry and merry, and also excellently free of whining, so that I trooped almost frantically with a squeamish soup stain. Can one particularly use that as a sleazy froséen swagger, or should one rush a doppelganger here in order to be swaggered with solid, full-bodied bow ties? I don’t want to spoil that, not at all.

Douglas Hofstadter, “Wacky Jabber” at Inference Review (September 2022)

DeepL and the Chinese translation giant Baidu came up with entirely different complete nonsense. The comparisons of utter nonsense are breathtaking.

Hofstadter’s point?

Of course, none of the three machine-produced paragraphs has any meaning whatsoever, but the systems aren’t aware of that flagrant lack. This is because they have no notion of what meaningfulness and meaninglessness are. They are not thinking while translating; they are just doing very complicated but knee-jerk reflex operations with pieces of text.

Douglas Hofstadter, “Wacky Jabber” at Inference Review (September 2022)

Well, in fairness, the machine translation systems could not have noticed that the original paragraph was meaningless either. A human translator, by contrast, would pick up the phone…

He draws out the significance:

And yet they were all produced by sober, no-nonsense, deadpan, tone-deaf, and stone-dead programs that have nonetheless been trumpeted in many prestigious and influential publications — such as the New York Times, the Economist, and others — as being astonishingly powerful and supremely accurate translators.

Douglas Hofstadter, “Wacky Jabber” at Inference Review (September 2022)

Along the lines of Robert J. Marks’s recent book, Non-Computable You, the difference between sense and nonsense is not a matter of computation. Pretending that it is won’t end well.

You may also wish to read: Turing Tests are terribly misleading. Black box algorithms are now being trusted to approve loans, price insurance, screen job applicants, trade stocks, determine prison sentences, and much more. Is that wise? My tests of a large language model (LLM) showed that the powerful computer could discuss a topic without showing any understanding at all. (Gary Smith)