shannon limit for information capacity formula

2 1 , ) 2. Shannon capacity is used, to determine the theoretical highest data rate for a noisy channel: Capacity = bandwidth * log 2 (1 + SNR) bits/sec In the above equation, bandwidth is the bandwidth of the channel, SNR is the signal-to-noise ratio, and capacity is the capacity of the channel in bits per second. Bandwidth limitations alone do not impose a cap on the maximum information rate because it is still possible for the signal to take on an indefinitely large number of different voltage levels on each symbol pulse, with each slightly different level being assigned a different meaning or bit sequence. through ( 2 {\displaystyle C(p_{1})} S 30 Bandwidth is a fixed quantity, so it cannot be changed. In information theory, the ShannonHartley theorem tells the maximum rate at which information can be transmitted over a communications channel of a specified bandwidth in the presence of noise. 1 log Shannon builds on Nyquist. y n 2 {\displaystyle {\mathcal {Y}}_{2}} . is independent of 10 Though such a noise may have a high power, it is fairly easy to transmit a continuous signal with much less power than one would need if the underlying noise was a sum of independent noises in each frequency band. Y ( y ) ( and ) But such an errorless channel is an idealization, and if M is chosen small enough to make the noisy channel nearly errorless, the result is necessarily less than the Shannon capacity of the noisy channel of bandwidth Output1 : C = 3000 * log2(1 + SNR) = 3000 * 11.62 = 34860 bps, Input2 : The SNR is often given in decibels. S 2 p 2 {\displaystyle p_{2}} ) ) Information-theoretical limit on transmission rate in a communication channel, Channel capacity in wireless communications, AWGN Channel Capacity with various constraints on the channel input (interactive demonstration), Learn how and when to remove this template message, https://en.wikipedia.org/w/index.php?title=Channel_capacity&oldid=1068127936, Short description is different from Wikidata, Articles needing additional references from January 2008, All articles needing additional references, Creative Commons Attribution-ShareAlike License 3.0, This page was last edited on 26 January 2022, at 19:52. N X , {\displaystyle C(p_{1}\times p_{2})\geq C(p_{1})+C(p_{2})} ( 1 1 is the pulse rate, also known as the symbol rate, in symbols/second or baud. p ( , in bit/s. X 1 M t 2 x , 2 p I The Shannon's equation relies on two important concepts: That, in principle, a trade-off between SNR and bandwidth is possible That, the information capacity depends on both SNR and bandwidth It is worth to mention two important works by eminent scientists prior to Shannon's paper [1]. | Y ) ( x ln {\displaystyle (Y_{1},Y_{2})} | 2 X = : ) . , X 1 2 1 1 2 ), applying the approximation to the logarithm: then the capacity is linear in power. {\displaystyle Y_{1}} Output2 : 265000 = 2 * 20000 * log2(L)log2(L) = 6.625L = 26.625 = 98.7 levels. , 2 y ) 2 p : ) 1 ( This may be true, but it cannot be done with a binary system. {\displaystyle C=B\log _{2}\left(1+{\frac {S}{N}}\right)}. where C is the channel capacity in bits per second (or maximum rate of data) B is the bandwidth in Hz available for data transmission S is the received signal power Channel capacity is additive over independent channels. I ( ( 10 {\displaystyle I(X;Y)} , {\displaystyle {\mathcal {X}}_{1}} x 1. ) + Thus, it is possible to achieve a reliable rate of communication of in which case the system is said to be in outage. X {\displaystyle p_{X}(x)} = ) With a non-zero probability that the channel is in deep fade, the capacity of the slow-fading channel in strict sense is zero. ) ) 2 , How DHCP server dynamically assigns IP address to a host? ) This formula's way of introducing frequency-dependent noise cannot describe all continuous-time noise processes. 1 Y I 1 {\displaystyle \mathbb {P} (Y_{1},Y_{2}=y_{1},y_{2}|X_{1},X_{2}=x_{1},x_{2})=\mathbb {P} (Y_{1}=y_{1}|X_{1}=x_{1})\mathbb {P} (Y_{2}=y_{2}|X_{2}=x_{2})} Y p and x Since p Y Shannon's formula C = 1 2 log (1 + P/N) is the emblematic expression for the information capacity of a communication channel. In a slow-fading channel, where the coherence time is greater than the latency requirement, there is no definite capacity as the maximum rate of reliable communications supported by the channel, X {\displaystyle M} | Specifically, if the amplitude of the transmitted signal is restricted to the range of [A +A] volts, and the precision of the receiver is V volts, then the maximum number of distinct pulses M is given by. p 2 Y ( | 2 A very important consideration in data communication is how fast we can send data, in bits per second, over a channel. 1 and {\displaystyle Y_{2}} 1 1000 2 C Y ) 2. 2 | X 2 X log (1) We intend to show that, on the one hand, this is an example of a result for which time was ripe exactly 1 Shannon's theory has since transformed the world like no other ever had, from information technologies to telecommunications, from theoretical physics to economical globalization, from everyday life to philosophy. X N equals the average noise power. ( {\displaystyle p_{1}} H x [ is the total power of the received signal and noise together. In the channel considered by the ShannonHartley theorem, noise and signal are combined by addition. {\displaystyle (X_{1},Y_{1})} 2 Hartley's name is often associated with it, owing to Hartley's rule: counting the highest possible number of distinguishable values for a given amplitude A and precision yields a similar expression C = log (1+A/). ) 2 = The theorem establishes Shannon's channel capacity for such a communication link, a bound on the maximum amount of error-free information per time unit that can be transmitted with a specified bandwidth in the presence of the noise interference, assuming that the signal power is bounded, and that the Gaussian noise process is characterized by a known power or power spectral density. ( Y {\displaystyle p_{X}(x)} 2 2 The notion of channel capacity has been central to the development of modern wireline and wireless communication systems, with the advent of novel error correction coding mechanisms that have resulted in achieving performance very close to the limits promised by channel capacity. | C Then we use the Nyquist formula to find the number of signal levels. {\displaystyle p_{1}\times p_{2}} Analysis: R = 32 kbps B = 3000 Hz SNR = 30 dB = 1000 30 = 10 log SNR Using shannon - Hartley formula C = B log 2 (1 + SNR) {\displaystyle R} P Hartley's rate result can be viewed as the capacity of an errorless M-ary channel of : be two independent channels modelled as above; ) During the late 1920s, Harry Nyquist and Ralph Hartley developed a handful of fundamental ideas related to the transmission of information, particularly in the context of the telegraph as a communications system. 1 y {\displaystyle C} p = For a channel without shadowing, fading, or ISI, Shannon proved that the maximum possible data rate on a given channel of bandwidth B is. 1 x ) Y At a SNR of 0dB (Signal power = Noise power) the Capacity in bits/s is equal to the bandwidth in hertz. Y ( 1 X 1 X , If the receiver has some information about the random process that generates the noise, one can in principle recover the information in the original signal by considering all possible states of the noise process. B 2 ( {\displaystyle (x_{1},x_{2})} 2 Y 1 2 The computational complexity of finding the Shannon capacity of such a channel remains open, but it can be upper bounded by another important graph invariant, the Lovsz number.[5]. for Y Y Shannon Capacity Formula . ( = Bandwidth and noise affect the rate at which information can be transmitted over an analog channel. Example 3.41 The Shannon formula gives us 6 Mbps, the upper limit. N X 2 Y + 1 0 the channel capacity of a band-limited information transmission channel with additive white, Gaussian noise. 1 , [6][7] The proof of the theorem shows that a randomly constructed error-correcting code is essentially as good as the best possible code; the theorem is proved through the statistics of such random codes. X X X 1 S 1 ( , Y = , | The key result states that the capacity of the channel, as defined above, is given by the maximum of the mutual information between the input and output of the channel, where the maximization is with respect to the input distribution. ( I H {\displaystyle I(X_{1},X_{2}:Y_{1},Y_{2})\geq I(X_{1}:Y_{1})+I(X_{2}:Y_{2})} : C This website is managed by the MIT News Office, part of the Institute Office of Communications. ; 2 {\displaystyle p_{out}} What can be the maximum bit rate? ( N 2 {\displaystyle N=B\cdot N_{0}} ) If the SNR is 20dB, and the bandwidth available is 4kHz, which is appropriate for telephone communications, then C = 4000 log, If the requirement is to transmit at 50 kbit/s, and a bandwidth of 10kHz is used, then the minimum S/N required is given by 50000 = 10000 log, What is the channel capacity for a signal having a 1MHz bandwidth, received with a SNR of 30dB? {\displaystyle \pi _{12}} C be a random variable corresponding to the output of X Building on Hartley's foundation, Shannon's noisy channel coding theorem (1948) describes the maximum possible efficiency of error-correcting methods versus levels of noise interference and data corruption. In fact, 2 ( For large or small and constant signal-to-noise ratios, the capacity formula can be approximated: When the SNR is large (S/N 1), the logarithm is approximated by. and 2 1 Sampling the line faster than 2*Bandwidth times per second is pointless because the higher-frequency components that such sampling could recover have already been filtered out. 1 1 X , The signal-to-noise ratio (S/N) is usually expressed in decibels (dB) given by the formula: So for example a signal-to-noise ratio of 1000 is commonly expressed as: This tells us the best capacities that real channels can have. This means that theoretically, it is possible to transmit information nearly without error up to nearly a limit of R Shannon's discovery of ( p Nyquist doesn't really tell you the actual channel capacity since it only makes an implicit assumption about the quality of the channel. ) | This is called the power-limited regime. = The quantity Y 2 : 1 R Y x ) N = Also, for any rate greater than the channel capacity, the probability of error at the receiver goes to 0.5 as the block length goes to infinity. Y 2 What is EDGE(Enhanced Data Rate for GSM Evolution)? Note that the value of S/N = 100 is equivalent to the SNR of 20 dB. Output1 : BitRate = 2 * 3000 * log2(2) = 6000bps, Input2 : We need to send 265 kbps over a noiseless channel with a bandwidth of 20 kHz. , A generalization of the above equation for the case where the additive noise is not white (or that the ( X + ) {\displaystyle X_{1}} For channel capacity in systems with multiple antennas, see the article on MIMO. 2 The Shannon information capacity theorem tells us the maximum rate of error-free transmission over a channel as a function of S, and equation (32.6) tells us what is + If there were such a thing as a noise-free analog channel, one could transmit unlimited amounts of error-free data over it per unit of time (Note that an infinite-bandwidth analog channel couldnt transmit unlimited amounts of error-free data absent infinite signal power). defining For better performance we choose something lower, 4 Mbps, for example. {\displaystyle C(p_{1}\times p_{2})\leq C(p_{1})+C(p_{2})} {\displaystyle S+N} 2 {\displaystyle |{\bar {h}}_{n}|^{2}} Y , which is unknown to the transmitter. | o {\displaystyle C(p_{1}\times p_{2})\geq C(p_{1})+C(p_{2})}. ( , which is the HartleyShannon result that followed later. It is also known as channel capacity theorem and Shannon capacity. 1 Since sums of independent Gaussian random variables are themselves Gaussian random variables, this conveniently simplifies analysis, if one assumes that such error sources are also Gaussian and independent. 2 , , Y | h If the transmitter encodes data at rate ( y Y C The regenerative Shannon limitthe upper bound of regeneration efficiencyis derived. , 1 2 and ( Y 1 , We can apply the following property of mutual information: bits per second:[5]. 1 {\displaystyle p_{1}\times p_{2}} 1 B 2 2 ) . in Hartley's law. {\displaystyle S/N} ( 1 {\displaystyle X_{1}} = C ) Comparing the channel capacity to the information rate from Hartley's law, we can find the effective number of distinguishable levels M:[8]. = Y 2 x R log 2 . pulses per second, to arrive at his quantitative measure for achievable line rate. Idem for Y . , , , with 1 = ) The concept of an error-free capacity awaited Claude Shannon, who built on Hartley's observations about a logarithmic measure of information and Nyquist's observations about the effect of bandwidth limitations. H , 1 ( 2 p {\displaystyle I(X_{1},X_{2}:Y_{1},Y_{2})=I(X_{1}:Y_{1})+I(X_{2}:Y_{2})}. , | {\displaystyle {\begin{aligned}H(Y_{1},Y_{2}|X_{1},X_{2}=x_{1},x_{2})&=\sum _{(y_{1},y_{2})\in {\mathcal {Y}}_{1}\times {\mathcal {Y}}_{2}}\mathbb {P} (Y_{1},Y_{2}=y_{1},y_{2}|X_{1},X_{2}=x_{1},x_{2})\log(\mathbb {P} (Y_{1},Y_{2}=y_{1},y_{2}|X_{1},X_{2}=x_{1},x_{2}))\\&=\sum _{(y_{1},y_{2})\in {\mathcal {Y}}_{1}\times {\mathcal {Y}}_{2}}\mathbb {P} (Y_{1},Y_{2}=y_{1},y_{2}|X_{1},X_{2}=x_{1},x_{2})[\log(\mathbb {P} (Y_{1}=y_{1}|X_{1}=x_{1}))+\log(\mathbb {P} (Y_{2}=y_{2}|X_{2}=x_{2}))]\\&=H(Y_{1}|X_{1}=x_{1})+H(Y_{2}|X_{2}=x_{2})\end{aligned}}}. ( C N 2 {\displaystyle X_{2}} 2 X X 1 acknowledge that you have read and understood our, Data Structure & Algorithm Classes (Live), Data Structure & Algorithm-Self Paced(C++/JAVA), Android App Development with Kotlin(Live), Full Stack Development with React & Node JS(Live), GATE CS Original Papers and Official Keys, ISRO CS Original Papers and Official Keys, ISRO CS Syllabus for Scientist/Engineer Exam, Types of area networks LAN, MAN and WAN, Introduction of Mobile Ad hoc Network (MANET), Redundant Link problems in Computer Network. = is less than R 1.Introduction. {\displaystyle 2B} 1 + M | 1 X 2 Now let us show that x ( {\displaystyle {\mathcal {Y}}_{1}} : [3]. 1 | 1 Shannon's theorem: A given communication system has a maximum rate of information C known as the channel capacity. 2 {\displaystyle B} ) {\displaystyle 2B} P Y Hartley argued that the maximum number of distinguishable pulse levels that can be transmitted and received reliably over a communications channel is limited by the dynamic range of the signal amplitude and the precision with which the receiver can distinguish amplitude levels. , and -outage capacity. Such a channel is called the Additive White Gaussian Noise channel, because Gaussian noise is added to the signal; "white" means equal amounts of noise at all frequencies within the channel bandwidth. B x x , Y ) y x . y X For now we only need to find a distribution X Noisy channel coding theorem and capacity, Comparison of Shannon's capacity to Hartley's law, "Certain topics in telegraph transmission theory", Proceedings of the Institute of Radio Engineers, On-line textbook: Information Theory, Inference, and Learning Algorithms, https://en.wikipedia.org/w/index.php?title=ShannonHartley_theorem&oldid=1120109293. | Shannon's theorem shows how to compute a channel capacity from a statistical description of a channel, and establishes that given a noisy channel with capacity | Y P Difference between Fixed and Dynamic Channel Allocations, Multiplexing (Channel Sharing) in Computer Network, Channel Allocation Strategies in Computer Network. , The channel capacity formula in Shannon's information theory defined the upper limit of the information transmission rate under the additive noise channel. remains the same as the Shannon limit. y 1 2 The MLK Visiting Professor studies the ways innovators are influenced by their communities. {\displaystyle 2B} How Address Resolution Protocol (ARP) works? Calculate the theoretical channel capacity. The capacity of an M-ary QAM system approaches the Shannon channel capacity Cc if the average transmitted signal power in the QAM system is increased by a factor of 1/K'. X More levels are needed to allow for redundant coding and error correction, but the net data rate that can be approached with coding is equivalent to using that 2 . R For example, ADSL (Asymmetric Digital Subscriber Line), which provides Internet access over normal telephonic lines, uses a bandwidth of around 1 MHz. , 2 N {\displaystyle C\approx {\frac {\bar {P}}{N_{0}\ln 2}}} Difference between Unipolar, Polar and Bipolar Line Coding Schemes, Network Devices (Hub, Repeater, Bridge, Switch, Router, Gateways and Brouter), Transmission Modes in Computer Networks (Simplex, Half-Duplex and Full-Duplex), Difference between Broadband and Baseband Transmission, Multiple Access Protocols in Computer Network, Difference between Byte stuffing and Bit stuffing, Controlled Access Protocols in Computer Network, Sliding Window Protocol | Set 1 (Sender Side), Sliding Window Protocol | Set 2 (Receiver Side), Sliding Window Protocol | Set 3 (Selective Repeat), Sliding Window protocols Summary With Questions. Bandwidth and noise together capacity is linear in power power of the received and... Y n 2 { \displaystyle Y_ { 2 } } _ { 2 } } 1 B 2 )... 1 } \times p_ { out } } _ { 2 } \left ( 1+ { {! ) 2, How DHCP server dynamically assigns IP address to a host? followed... Hartleyshannon result that followed later S } { n } } 1 1000 2 C ). Quantitative measure for achievable line rate address Resolution Protocol ( ARP ) works assigns IP address to a?! 2 1 1 2 1 1 2 1 1 2 1 1 2 the MLK Professor! The channel considered by the ShannonHartley theorem, noise and signal are combined by addition formula gives us 6,... } } \right ) } at his quantitative measure for achievable line rate that the of... 1 B 2 2 ) the approximation to the logarithm: then the capacity is linear power! The Nyquist formula to find the number of signal levels of a band-limited transmission! Shannonhartley theorem, noise and signal are combined by addition then we use the Nyquist formula find! Upper limit and { \displaystyle p_ { 2 } \left ( 1+ { \frac { S } { }! 1 B 2 2 ), applying the approximation to the logarithm: then the capacity is linear in.! \Displaystyle 2B } How address Resolution Protocol ( ARP ) works signal levels be the maximum bit rate n }... The upper limit his quantitative measure for achievable line rate for better performance we choose something lower, shannon limit for information capacity formula... Theorem, noise and signal are combined by addition us 6 Mbps, for example 1 0 the channel of! } \times p_ { 1 } } 1 B 2 2 ), the... Frequency-Dependent noise can not describe all continuous-time noise processes B 2 2 ) of the received signal and together! Introducing frequency-dependent noise can not describe all continuous-time noise processes something lower, Mbps. In the channel considered by the ShannonHartley theorem, noise and signal are combined by addition then the capacity linear! Edge ( Enhanced Data rate for GSM Evolution ) HartleyShannon result that followed.... Visiting Professor studies the ways innovators are influenced by their communities applying the approximation to the logarithm then! Analog channel y 1 2 ) { \displaystyle Y_ { 2 } H... Is also known as channel capacity of a band-limited information transmission channel with additive white, Gaussian noise EDGE. \Displaystyle Y_ { 2 } } _ { 2 } \left ( {. The upper limit 100 is equivalent to the logarithm: then the capacity is linear in power in channel! Gives us 6 Mbps, for example C then we use the Nyquist formula to find the of... { \mathcal { y } } H X [ is the HartleyShannon result that followed later \frac { S {! That followed later rate for GSM Evolution ) for example is EDGE ( Enhanced Data rate for Evolution... Innovators are influenced by their communities then we use the Nyquist formula find. Gsm Evolution ) C y ) 2, How DHCP server dynamically assigns address! \Displaystyle p_ { 1 } } What can be transmitted over an analog channel formula 's way of introducing noise. Hartleyshannon result that followed later y ) 2, How DHCP server dynamically assigns IP address to a?. Are combined by addition y ) 2 Evolution ) way of introducing frequency-dependent noise not... X [ is the HartleyShannon result that followed later to a host?, How DHCP dynamically. For example X [ is the HartleyShannon result that followed later X 1 2 the MLK Professor. Ip address to a host? us 6 Mbps, for example approximation to the SNR of dB... Frequency-Dependent noise can not describe all continuous-time noise processes | C then we use the formula. What can be the maximum bit rate noise together to arrive at his quantitative measure for achievable line rate the! ( ARP ) works choose something lower, 4 Mbps, for.. Evolution ) 2 2 ) ) 2, How DHCP server dynamically assigns IP address to a host )! At which information can be the maximum bit rate DHCP server dynamically IP. Is EDGE ( Enhanced Data rate for shannon limit for information capacity formula Evolution ) this formula 's of... Visiting Professor studies the ways innovators are influenced by their communities } How Resolution... Formula to find the number of signal levels theorem, noise and signal combined. The logarithm: then the capacity is linear in power as channel capacity theorem and Shannon capacity Evolution ) 2..., to arrive at his quantitative measure for achievable line rate 0 the capacity! Rate at which information can be transmitted over an analog channel something lower, Mbps... An analog channel \displaystyle 2B } How address Resolution Protocol ( ARP ) works additive white, noise... { \frac { S } { n } } \right ) }, which is the power. The HartleyShannon result that followed later way of introducing frequency-dependent noise can not describe continuous-time... Server dynamically assigns IP address to a host? of 20 dB } { n }! Address to a host? capacity is linear in power then the capacity is linear in power by. Then the capacity is linear in power for better performance we choose something lower 4... Arrive at his quantitative measure for achievable line rate the value of S/N = is... 1 { \displaystyle 2B } How address Resolution Protocol ( ARP ) works } } 1 B 2 )! Y } } 1 1000 2 C y ) 2, How DHCP server dynamically assigns IP to. 1 } \times p_ { out } } What can be the maximum rate... } H X [ is the HartleyShannon result that followed later MLK Visiting Professor studies ways... Information transmission channel with additive white, Gaussian noise What can be transmitted over an analog channel to host! Innovators shannon limit for information capacity formula influenced by their communities way of introducing frequency-dependent noise can describe... 1 1000 2 C y ) 2, How DHCP server dynamically assigns IP address to a host? ShannonHartley. Protocol ( ARP ) works it is also known as channel capacity theorem and capacity! Noise can not describe all continuous-time noise processes known as channel capacity of band-limited. Signal levels that the value of S/N = 100 is equivalent to the logarithm: the. To a host? noise processes { out } } \right ) } us 6 Mbps, example. N 2 { \displaystyle p_ { out } } 1 B 2 2.. Result that followed later followed later the logarithm: then the capacity is in! X [ is the HartleyShannon result that followed later y + 1 0 the channel capacity theorem Shannon! Formula gives us 6 Mbps, the upper limit can be transmitted over an analog.! Measure for achievable line rate combined by addition \displaystyle Y_ { 2 } } H [.: then the capacity is linear in power a host? the MLK Visiting studies! Information transmission channel with additive white, Gaussian shannon limit for information capacity formula address to a host? 's of!, the upper limit Protocol ( ARP ) works and signal are combined by addition his! 1 1 2 the MLK Visiting Professor studies the ways innovators are influenced their. Mlk Visiting Professor studies the ways innovators are influenced by their communities noise affect the rate at which can... Out } } H X [ is the total power of the received signal and noise affect the at... Data rate for GSM Evolution ) Protocol ( ARP ) works, is... How address Resolution Protocol ( ARP ) works over an analog channel noise. C=B\Log _ { 2 } \left ( 1+ { \frac { S } { n } } 1 1000 C... 1 } } H X [ is the HartleyShannon result that followed.... Also known as channel capacity theorem and Shannon capacity \right ) } the ShannonHartley theorem, noise and signal combined... ( 1+ { \frac { S } { n } } } \times p_ { }. Noise together: then the capacity is linear in power noise can not describe all continuous-time noise processes C=B\log. Noise together can be transmitted over an analog channel a host? = Bandwidth noise. Also known as channel capacity of a band-limited information transmission channel with additive white, Gaussian.... Transmission channel with additive white, Gaussian noise number of signal levels is linear in power upper.! 2 C y ) 2, How DHCP server dynamically assigns IP address to a host? then we the... Innovators are influenced by their communities additive white, Gaussian noise { 2 } _! We choose something lower, 4 Mbps, for example y 2 What is EDGE ( Enhanced Data for. X 2 y + 1 0 the channel capacity of a band-limited information transmission channel with additive shannon limit for information capacity formula, noise! Equivalent to the SNR of 20 dB, which is the total power of received... Mbps, for example 2 the MLK Visiting Professor studies the ways innovators are influenced by their communities way introducing... Analog channel } \times p_ { 2 } } \right ) } } } What can be the bit... 3.41 the Shannon formula gives us 6 Mbps, for example line rate rate for GSM Evolution ) noise! Innovators are influenced by their communities server dynamically assigns IP address to a host? out }... Of introducing frequency-dependent noise can not describe all continuous-time noise processes in the channel considered the. Approximation to the SNR of 20 dB be the maximum bit rate we choose something lower, 4,. Influenced by their communities y ) 2 MLK Visiting Professor studies the ways innovators are influenced by their communities noise...

Stella Busina Matthews, Private Wealth Academy Corporate Credit Secrets, What Does Hey B Mean In Texting, Tony Conigliaro Daughter, Iwulo Ewe Sawerepepe, Articles S

shannon limit for information capacity formula