What is Information Theory?
53
Information Theory = Probability + Statistics
Yeah it's a pure matter of mathematics, I know that most of you don't like it, so let’s talk big guy "Pure mathematics is, in its way, the poetry of logical ideas." ~Albert Einstein
Ok I know that you are not convinced yet. So, let's forget about math. In practice in any electronic gadget, that you have, a magic touch of this theory is in, your computer (Of course you will say), you cell phone, iPod….
But let me before go further, pay tribute to the man in the photo, Claude Shannon (1916 - 2001the inventor who did almost every thing in this science..
What he did is about:
- Source codes: Compress
- Secrecy codes: encrypt
- Error control codes: Detect and Correct
The object of these three verbs is information.
A bit of Information
What is information? It’s really the hardest question to answer.
But the answer is simply is the tools that we use to describe our world. You have information stored in your mind (memory) and communicate it to the external world by the language (any kin of language is ok: speaking, dancing …,) but due our limited capacity it’s would be better to be smart when we use all these resources. And some need you will regret that you didn’t protect your information. You will send a letter to your boss about some issues in your job; you will try to make your message as short as possible, you will protect the content to be discovered by any contend of your company, and you will insure to deliver the message without any mistakes or errors, this is compress, encrypt and correct.
The Digital world
Any language uses a set of symbols as basic component (letters), and what is the simplest possible language? Yeah of course a one that has only two letters, that can true or false or black and white or 0 and 1 such a language is said to be binary, and from now we will speak only that language, the mathematics (again and again) behind it is called Boolean algebra. I think that it’s the easiest mathematics and we should start to teach it as first topics in primary school.
We say digital because we are using 0 and 1, and believe me; most of gadgets around you speak this language..
As object the information is measurable entity the unit for the binary language is the bit, all of you know Mb (mega bits = 2^20 = 2 * 2 * 2 …. Twenty times), Gb (giga bits = 2 ^30)and coming the Tb(tera bits ).
Now we are interested to communicate and to store these bits. Before him (The man: Shannon) we believe, that to transmit more information we need more power and there no other way. But what he showed that there is a smarter way to do it. Let’s say that you want to express your love to your women. The first way is that you say it one as strong as you can, or repeat it until she fall in love too. We call this a repletion code and it’s the easiest one, but not the most efficient one.
It seems from my short introduction that this theory is mainly related to communication but since the time of Shannon, and it was his motivation, but nowadays we can fin application and relation with other fields of science, Computer science (Kolmogorov complexity), physics (Thermodynamics), economics (Investement) and philosophy of sciences (Occam’s razor). I shall present these in coming hubs.
Let me know if you are interested to have more details or references.
|
|
An Introduction to Information Theory
Price: $6.00
List Price: $12.95 |
|
|
Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing)
Price: $54.22
List Price: $95.50 |
|
|
A First Course in Information Theory (Information Technology: Transmission, Processing and Storage)
Price: $52.29
List Price: $79.95 |
|
Decision Analysis, Game Theory, and Information (University Casebook Series)
Price: $18.93
List Price: $22.00 |
PrintShare it! — Rate it: up down flag this hub









Lgali says:
9 months ago
very informative article good to know
Information Theory = Probability + Statistics