Question

In: Electrical Engineering

Briefly describe the significance of the Shannon limit for information capacity.

Briefly describe the significance of the Shannon limit for information capacity.

Solutions

Expert Solution

Consider the case in which the channel is noisy enough that a four-bit message requires an eight-bit code. The receiver, like the sender, would have a codebook that correlates the 16 possible four-bit messages with 16 eight-bit codes. Since there are 256 possible sequences of eight bits, there are at least 240 that don’t appear in the codebook. If the receiver receives one of those 240 sequences, she knows that an error has crept into the data. But of the 16 permitted codes, there’s likely to be only one that best fits the received sequence — that differs, say, by only a digit.
Shannon showed that, statistically, if you consider all possible assignments of random codes to messages, there must be at least one that approaches the Shannon limit. The longer the code, the closer you can get: eight-bit codes for four-bit messages wouldn’t actually get you very close, but two-thousand-bit codes for thousand-bit messages could.
Of course, the coding scheme Shannon described is totally impractical: a codebook with a separate, randomly assigned code for every possible thousand-bit message wouldn’t begin to fit on all the hard drives of all the computers in the world. But Shannon’s proof held out the tantalizing possibility that, since capacity-approaching codes must exist, there might be a more efficient way to find them.


Related Solutions

Briefly discuss the significance of two main factors (list those 2 factors) in strength limit state...
Briefly discuss the significance of two main factors (list those 2 factors) in strength limit state design method used for safety in the context of a real-world structural steel design example.
1. In the context of genetic variation and gene duplication, briefly describe the significance of the...
1. In the context of genetic variation and gene duplication, briefly describe the significance of the human hemoglobin genes. 2. Define, compare, and contrast “biological evolution” and “natural selection”
Summarize briefly the main research tool which led to discovery of the information and the significance...
Summarize briefly the main research tool which led to discovery of the information and the significance of the concept for our understanding of life. Describe one application of the concept in industry, medicine, or research? With reference
The nurse performs a tactile fremitus. Identify two assessment findings and briefly describe the clinical significance...
The nurse performs a tactile fremitus. Identify two assessment findings and briefly describe the clinical significance of each. (7marks)
Describe the theoretical capacity of a single lane of traffic. Describe the capacity at a traffic...
Describe the theoretical capacity of a single lane of traffic. Describe the capacity at a traffic signal. How might these values change with an automated car or other Connected Vehicle aps?
a. Explain briefly on the Branches of Accounting and describe the characteristics of useful information in...
a. Explain briefly on the Branches of Accounting and describe the characteristics of useful information in Accounting                                                          b. State other methods available in calculating depreciation apart from At Cost and at Net Book based in measuring the assets of an organization.           
Briefly describe the impact of trauma on a person's ability to understand and for retaining information
Briefly describe the impact of trauma on a person's ability to understand and for retaining information
briefly describe the Urey-Miller experiment, then identify the significance (i.e. what did it tell us about the origin of life?).
  briefly describe the Urey-Miller experiment, then identify the significance (i.e. what did it tell us about the origin of life?). Pick an animal, then define the species using the biological species concept. Compare and contrast analogous and homologous structures. Compare and contrast microevolution and macroevolution. Compare and contrast background and mass extinctions.
In deregulating capital-intensive industries, why might it be a sensible policy to limit investment in capacity...
In deregulating capital-intensive industries, why might it be a sensible policy to limit investment in capacity by incumbent firms or require the dominant incumbent firm to divest assets?
Explain briefly the importance of serviceability limit state in the design stage for the structures
Explain briefly the importance of serviceability limit state in the design stage for the structures
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT