Date: Mon, 22 Apr 2002 11:15:24 -0700 (PDT)
From: Grant Peacock 
Subject: Re: (urth) Mr Million v Oreb


--- Jeff Wilson wrote:
> Measuring raw switching events per second is misleading. The 10^11
> neurons in the brain might change state 20-30 times a second each, for
> 2.5 2.5 x 10^12 switching events a second. However, almost 10 years ago,
> Pentium processors with 3.3 million transistors were running at 100
> megahertz, for 3.3 x 10^14 switching events a second, and just consuming
> a few watts.

I'm not sure this is number makes any sense.  A 100 meg processor executes a
maximum of 10^8 operations per second, no matter how much memory it has
available.

My source of information is _I am Right, You are Wrong_ by Edward de Bono.  I
recommend this book.  Unfortunately my copy is on loan so I might miss a lot of
details here.  The computing knowledge is also shaky.  But here goes.  A neuron
has 2 states, firing and not firing, but don't let this fool you into thinking
of the brain as binary.  A neuron also has dendrites pointing to dozens of
other neurons, and it also has a variable stimulation threshold.  That is, it
needs a certain amount of "juice" from neurons that point to it before it
fires.  This limit changes depending on the amount of time since it fired
previously, and on levels of whatever chemicals to which it is sensitive. 
Finally, the amount of influence one neuron has over another can be increased
over time.  This happens when two connected neurons fire at the same time.

So, without too much effort to be efficient about the coding, let's estimate
how much computing power one would need to try to simulate a brain, using a
standard style of computer.  For each neuron, we need a list of other neurons
to which this one has a dendrite, and the sensitivity of each.  Here we're
talking probably 3 bytes * 20 or 30 neurons.  This is the hard part,
space-wise.  Keeping it on a disk would be way too slow -- I think it has to be
in memory.  So we need about 100 bytes per neuron, or 10^7 megs of RAM.  This
is 10,000 times the memory of today's PC but it is conceivable.

Now, the real bottleneck is that we only have one processor.  We can only look
at one neuron at a time.  The processor will have to make "passes" through the
entire list of neurons.  For each pass, on each neuron which is firing, it will
go down the list of dendrites and add the right amount of juice to all the
neighbors.  Let's suppose that a maximum of 10% of a brain's 10^11 neurons can
be firing at any given time.  Still, that's 10^10 * 20 additions that must be
performed in the slow part of the pass.  (I'm assuming an average of 20
dendrites/neuron, this may be too low.)  Adding probably takes more than 8 or
10 clock cycles but let's say 10.  So each pass takes on the order of 10^12
clock cycles.

Now, when we consider that a neuron might go on and off 20 times per second,
I'd say the passes I just described need to be performed at least 1000 times
per second to achieve realistic brain activity.  So our processor needs to run
at around 10^16 hertz, that is, do the work of 100 million pentiums.

The reason we're still behind by such a large factor is that each neuron is not
just a memory cell, it also performs some of the functions assumed by the
processor in a computer.  

-Peacock


__________________________________________________
Do You Yahoo!?
Yahoo! Games - play chess, backgammon, pool and more
http://games.yahoo.com/

--