Information Theory for Understanding Living Systems


Of course, Information Theory lies at the heart of an information-based understanding of life. This theme explains and develops the information theoretic concepts needed for the other themes and incorporates the concepts that have been inspired by an information-focussed study of life.

Usually, information theory starts with Shannon's mathematical description of communication. We have to start deeper, so as to develop a more general understanding of information, which we can then apply to the structure and function of biological systems, not constrained to communication-like problems.

This more general understanding is mainly attributable to Luciano Floridi (one of our  network). We begin with data, the element of which is a single binary difference: on / off or black / white. This single difference (on is different from off in one single respect) is the smallest element of data, so a set of n differences constitutes n-bits of data. We may say it takes n-bits of data to fully describe something that can be decomposed into a set of n binary differences.

Data is the ‘raw material’ of physical information. This is familiar to us as electronic computers store information in the form of strings of binary data. What the stored data actually consists of is a set of binary differences making a striped pattern in some material substance (e.g. the magnetisation of a thin disk of iron oxide). A DNA molecule stores data as a base-4 set of differences among the nucleic acids (A,G,C, and T), also making a pattern like a string of beads which can be ‘read’. The data is amenable to Shannon’s mathematical information theory and all that follows from it. But more generally, the pattern formed by the stored data is a configuration of matter in time and space that we say embodies and instantiates the data. By the same thinking, we can interpret any material object as a form in space and time defined by a pattern of differences: that is the object instantiates data.

Data instantiating the form of an object is equivalent (reversibly) with the object’s form embodying the data. Thus we can speak not only about data describing the object, but also the object being data: a pattern of matter in space and time. Very often the form of an object is functional, for example the shape of an enzyme molecule determines its function (indeed this is a general truth about molecules).

The definition of physical information to which Floridi points us is that of 'well formed and meaningful data'. This raises the difficult problem of ‘meaningful’, usually considered in the context of messages. In its most general sense, we take meaningful to be identical with ‘functional’, where function refers to the potential to cause a change in information pattern. We can think of meaningful data (for example a functional gene) as being able (in conjunction with an appropriate process) of creating a new pattern (for example a protein that the DNA codes for).

In mathematical information theory (which in fact deals only with the statistics of data), meaning is never referred to (this blindness to meaning is one of its founding axioms). When we refer to information, we mean well formed and functional data, in the sense that the pattern it consititutes may modify (or form) further pattern in matter (or energy): changing the distribution of matter in space and time. This is memorably captured in the phrase of Gregory Bateson: information is “a difference which makes a difference”.These ideas are taken further in the Philosophy theme, here we concentrate on the application of mathematical information theory (a the theory of data, without reference to meaning), supplemented by a separate theory of function relevant to biological systems.



The Mathematical Theory of Information


See how to calculate information content based on uncertainty -> here.

See also:

How this applies to nucleotides (the Molecular Biology Theme)

Entropy

Function

The Theme is led by  Dr Keith Farnsworth, Dr Carlos Gershenson and Dr Thomas Schneider