Information theory was first described by Claude Shannon
in 1948 [5]. It sets out a mathematical
way to measure the choices made in a system. Although Shannon
concentrated on communications, the mathematics applies equally
well to other fields [6].
In particular, all of the theorems apply
in biology because the same constraints occur in biology
as in communication. For example, if I call you on the phone
and it is a bad connection, I may say `let me call you back'.
Then I hang up. I may even complain to the phone company
who then rips out the bad wires. So the process of
*killing the phone line*
is equivalent to
*selecting against a specific phenotype* in biology.

A second example is the copying of a key. In biology that's called `replication', and sometimes there are `mutations'. We go to a hardware store and have a key copied, but we get home only to find that it doesn't fit the door. When we return to the person who copied it, they throw the key away (kill it) and start fresh.

This kind of selection does not occur in straight physics.
It turns out that the
requirement of being able to make
distinct selections is critical to Shannon's channel capacity
theorem [7].
Shannon defined the channel capacity, *C* (bits per second)
as the maximum
rate that information can be sent through a communications
channel in the presence of thermal noise.
The theorem has two parts.
The first part says that
if the data rate one would like to send at,
*R*, is greater than *C*, one will fail.
At most *C* bits per second will get through.
The second part is surprising. It says that as long
as *R* is less than *or equal to* *C* the error rate
may be made as low as one desires.
The way that Shannon envisioned attaining this result was by encoding
the message before transmission
and decoding it afterwards.
Encoding methods have been explored in the ensuing 50 years
[8,9],
and their successful application is
responsible for the accuracy of our solar-system spanning
communications systems.

To construct the channel capacity theorem, Shannon assigned each message
to a point in a high dimensional space.
Suppose that we have a volt meter
that can be connected by a cable to a battery with a switch.
The switch has two
states, on and off, and so we can send 1 bit of information.
In geometrical terms, we can record the state
(voltage)
as one of two points on a line, such as *X*=0 and *X*=1.
Suppose now that we send two pulses, *X* and *Y*.
This allows for 4 possibilities,
00, 01, 10 and 11 and these form a square on a plane. If we send
100 pulses, then any particular sequence will be a point in
a 100 dimensional space (hyperspace).

If I send you a message, I first encode it as a string of 1s
and 0s and then send it down the wire. But the wire is hot
and this disturbs the signal [10,11].
So instead of *X* volts
you would receive
,
a variation around *X*.
There would be a different variation for *Y*:
.
and
are independent because thermal noise does not correlate over time.
Because they are the sum of many random molecular impacts,
for 100 pulses the s would have a Gaussian distribution
if they were plotted on one axis.
But because they are independent,
and the geometrical representation of independence is a right angle,
this represents 100 different directions in the high dimensional
space.
There is no particular direction in the high dimensional space
that is favored by the noise, so it turns out
that the original message will come to the receiver somewhere
on a sphere around the original point
[7,12,3].

What Shannon recognized is that these little noise
spheres have very sharply defined edges.
This is an effect of the high dimensionality:
in traversing from the center of the sphere to the surface
there are so many ways to go that essentially everything is on the surface
[13,14,12].
If one packs the message spheres
together so that they don't touch (with some error because they
are still somewhat fuzzy) then one can get the channel capacity.
The positions in hyperspace
that we choose for the messages is the *encoding*.
If we were to allow the spheres to intersect (by encoding in a poor way) then
the receiver wouldn't be able to distinguish overlapping messages.
The crucial
point is that we must choose non-overlapping spheres.
This only matters in human and animal communications systems
where failure can mean death.
It does not happen to rocks on the moon because there is
no consequence for `failure' in that case.
So Shannon's channel capacity theorem only applies when there is
a living creature associated with the system.
From this I conclude that Shannon is a biologist
and that his theorem is about biology.

The capacity theorem can be constructed for biological molecules
that interact or have different states [12].
This means that these molecular machines are capable of making precise
choices. Indeed, biologists know of many amazingly specific
interactions; the theorem shows that not only is this possible
but that
**biological systems can evolve to have as few
errors as necessary for survival**.

U.S. Department of Health and Human Services | National Institutes of Health | National Cancer Institute | USA.gov |

Policies | Viewing Files | Accessibility | FOIA