Large scale machine learning models are based on mathematical abstractions of biological neural networks. The highly successful GPT-3 model basically models the synapses of more than 8 million neurons (with preposterously large matrix calculations).
As a former psychology major, I know that there is more to the brain than synapses. Neurons have nuclei, membranes, and a trunk (axion) with branches (dendrites). And lots of chemicals inside and out. (And who knows what quantum effects we don’t know about.)
This makes me wonder what is being left out of the commonly used models, and why.
I’m pretty sure the “why” has a lot to do with practical considerations of contemporary engineering. The synaptic activity is modelled in discretized vectors that block together groups of synapses in blocks of time. The propagation of values (which natural neurons do via the other anatomical parts) is represented by big matrix computations. The output represents…
View original post 448 more words