Mind Matters Natural and Artificial Intelligence News and Analysis

TagGuess the Movie game and AI

3d-render-ai-artificial-intelligence-technology-cpu-central-processor-unit-chipset-on-the-printed-circuit-board-for-electronic-and-technology-concept-select-focus-shallow-depth-of-field-stockpack-adobe-stock
3D render AI artificial intelligence technology CPU central processor unit chipset on the printed circuit board for electronic and technology concept select focus shallow depth of field

How Well Do Researchers Say Chatbots and Other AI Really Perform?

The 400 researchers found that getting moderately high performance requires models with around 100 billion parameters, an exponentially hard problem

A vast team of over 400 researchers recently released a new open-access study on the performance of recent, popular text-based AI architectures such as GPT, the Pathways Language Model, the (recently controversial) LaMBDA architecture, and sparse expert models. The study, titled the “Beyond the Imitation Game,” or BIG, tries to provide a general benchmark for the state of text-based AI, how it compares to humans on the same tasks, and the effect of model size on the ability to perform the task. First, many of the results were interesting though not surprising: ● In all categories, the best humans outdid the best AIs (though that edge was smallest on translation problems from the International Language Olympiad).● Bigger models generally showed…