Return to GLYPH's information content FAQ
Return to GLYPH's projects page

There Is No Absolute Measure of Information
There is often the misconception that there exists some ideal or absolute measure of information. A large purpose of these pages is to show that the measured information content of a signal is completely dependant on the model used for measuring. A further purpose is to show that there is no ideal model for measurement. This proof should put these misconceptions to rest.

I've reworded this proof on August 8, 1998, to make a few more people happy. Actually, I'm rather glad they complained, because I like this new wording.

---

Part One

Let us assume that there is an ideal measure of information, I.

Define I such that I(s) is the absolute amount of information in string s, in bits. I(s) is the minimum amount of information needed to send s.

Define a pair of black boxes, a sender and a receiver, which transmit a string s using exactly I(s) information. Then no device can send a string in less information than these black boxes.

Select any two strings, a and b.

Define a bijective mapping F and its inverse mapping G over the domain of all strings, such that F(a) = b, and thus G(b) = a.

Modify the black boxes into a pair of gray boxes by attaching F before the sender and G after the receiver.

If a is sent through the gray boxes, it cannot be sent using less than I(a) information, since then they would be beat the black boxes.

Therefore, I(F(a)) >= I(a)

This is equivalent to I(b) >= I(a)

Since this applies to any two strings a and b, we have the following:

Select any bijective mapping J and its inverse mapping K over the domain of all strings.

Select any string p.

Substituting J(p) for a and p for b, we get I(J(p)) >= I(p)

Substituting p for a and J(p) for b, we get I(p) >= I(J(p))

Combining I(J(p)) >= I(p) with I(p) >= I(J(p)) yields I(p) = I(J(p))

This means that (1) : applying an arbitrary bijective mapping to a string does not alter its absolute information measure.

---

Part Two

Define an arbitrary string c

Define another arbitrary string d

Define a bijective mapping M and its inverse N such that d = M(c)

There can exist a pair of gray boxes which use M and N.

So, by (1), I(c) = I(M(c)) = I(d)

Therefore, (2) : the absolute information measures of any two strings are equal.

---

Part Three

By (2), the black boxes transmit the same amount of information regardless of the string being sent. This means they use a uniform probability model.

We can build a pair of blue boxes which are identical to the black boxes, but which use a skewed probability model. The blue boxes will therefore do better than the black boxes for some strings.

Therefore the blue boxes sometimes send a string s in less than I(s) information.

Therefore I(s) is not the minimum amount of information needed to send s.

Therefore I(s) is not I(s).

Therefore I(s) cannot exist.

---