This document explores various information theoretic concepts initially developed in physics and engineering and used today in many other fields. Information is defined as a quantity inversely proportional to the probability that an event happens and entropy is defined as the expected value of...