Information Theory for Understanding Living Systems
Of course, Information Theory lies at the heart of an information-based understanding of life. This theme explains and develops the information theoretic concepts needed for the other themes and incorporates the concepts that have been inspired by an information-focussed study of life.
Usually, information theory starts with Shannon's mathematical
description of communication. We have to start deeper, so as to develop
a more general understanding of information, which we can then apply to
the structure and function of biological systems, not constrained to
This more general understanding is mainly attributable to Luciano Floridi (one of our network). We begin with data, the element of which is a single binary difference: on / off or black / white. This single difference (on is different from off in one single respect) is the smallest element of data, so a set of n differences constitutes n-bits of data. We may say it takes n-bits of data to fully describe something that can be decomposed into a set of n binary differences.
Data is the ‘raw material’ of physical information. This is familiar to us as electronic computers store information in the form of strings of binary data. What the stored data actually consists of is a set of binary differences making a striped pattern in some material substance (e.g. the magnetisation of a thin disk of iron oxide). A DNA molecule stores data as a base-4 set of differences among the nucleic acids (A,G,C, and T), also making a pattern like a string of beads which can be ‘read’. The data is amenable to Shannon’s mathematical information theory and all that follows from it. But more generally, the pattern formed by the stored data is a configuration of matter in time and space that we say embodies and instantiates the data. By the same thinking, we can interpret any material object as a form in space and time defined by a pattern of differences: that is the object instantiates data.
Data instantiating the form of an object is equivalent (reversibly) with the object’s form embodying the data. Thus we can speak not only about data describing the object, but also the object being data: a pattern of matter in space and time. Very often the form of an object is functional, for example the shape of an enzyme molecule determines its function (indeed this is a general truth about molecules).
The definition of physical information to which Floridi points us is that of 'well formed and meaningful data'. This raises the difficult problem of ‘meaningful’, usually considered in the context of messages. In its most general sense, we take meaningful to be identical with ‘functional’, where function refers to the potential to cause a change in information pattern. We can think of meaningful data (for example a functional gene) as being able (in conjunction with an appropriate process) of creating a new pattern (for example a protein that the DNA codes for).
In mathematical information theory (which in fact deals only with
the statistics of data), meaning is never referred to (this blindness
to meaning is one of its founding axioms). When we refer to
information, we mean well formed and functional data, in the sense that
the pattern it consititutes may modify (or form) further pattern in
matter (or energy): changing the distribution of matter in space and
time. This is memorably captured in the phrase of Gregory Bateson:
information is “a difference which makes a difference”.These ideas are
taken further in the Philosophy theme,
here we concentrate on the application of mathematical information
theory (a the theory of data, without reference to meaning),
supplemented by a separate theory of function
relevant to biological systems.
The Mathematical Theory of Information
See how to calculate information content based on uncertainty -> here.
How this applies to nucleotides (the Molecular Biology Theme)