Is concept of "bit" in computer programming similar to the concept of "bit" in information theory? until today I knew that one bit is a variable, or a space in memory that can hold a value of either One (high) or Zero (low). This is the concept I learned from studying computer programming, microprocessor or DATA bus etc. 

But after starting the course on information theory, I found out that bit is expressed as the information content of a symbol in message. This is calculated taking the logarithm (base 2) of the inverse of the probability of occurrence of the symbol.

Are these two concepts same ? On one hand one bit is a variable that can store either zero or one. On the other hand, one bit is the uncertainty associated with one of two symbols with probability of occurrence of 0.5. So, does 1 bit in computer programming or ASCII code mean 1 bit in information content of source or information theory?  

A little edit:
here is one thing I am finding trouble understanding this topic. See, in data transfer of English alphabets, if we use ASCII code, we basically represent each symbol with 8 bits. Suppose that's 00000000 for a, 00000001 for b etc. So we are essentially allocating 8 quantization levels for each symbol.

But when the information theory comes into play, we take the probability of each symbol into account. 'E' has the highest frequency, where 'Z' has the lowest. So average information content comes down to 3 or 4 bits, right ?

My book says, 'Entropy or average information content is the minimum average number of bits required to represent each sample without distortion'.So, in this case, for efficient data transfer, are we creating maximum four quantization levels for each symbols? Because, on an average they carry information worth 4 bits. If that's so, isn't bit in information theory the same as the one in computer programming, data transfer or ASCII code etc ?

You probably get that I am clearly a noob here :p
They are not the same, but they're related. In particular, if you look at a computer memory holding $M$ "computer" bits, where each bit can be considered random and independent of all other bits, and there are roughly 50% of zeros, then the memory also holds roughly $M$ "information theory" bits.

Of course, this is often not the case: computer bits are usually correlated, and not uniformly random. This is why they can be compressed. Compressor programs such as LZW ("source coders" in information theory parlance) work, in a sense, by making each computer bit hold one information bit.

Edited to add: This example may make the distinction clearer. Consider a memoryless source with two outputs, $m_1=000$ and $m_2=001$, with probability 0.5 for each. Clearly, the information in each message is one (information) bit, but its length is three (computer) bits. A source coder, such as the Huffman algorithm, will readily code the messages to $c_1=0$ and $c_2=1$, compressing the source output. You can easily extrapolate this example to a source that produces ASCII-encoded text.

Note that, in the case of written languages in general and English in particular, nobody knows what the actual source entropy is, because there is no model for it. That is why there are contests for the best compression of large bodies of text; nobody is really sure what the optimum compression algorithm for English is.