# Are the bias term b and the Bias in "Bias–variance tradeoff" the same thing?

In ADALINE algorithm , with $y=x*w+b$, where $x$ is the feature vector of a sample and $w$ is the weight vector, the update rule (SGD) for the bias $b$ is: $b \leftarrow b + \eta(o - y)$.

With Gradient Descent, the final $b$ is actually: $$b \leftarrow b_{init} + \eta\sum_{i=1}^n(o_i - y_i)$$ where $b_{init}$ is a random initial value for $b$.

On the other hand, according to Bias–variance tradeoff, the mean square error decomposition is: $$Err(x)=Bias^2+Variance+Irreducible\ Error$$, where $Bias$ is: $$Bias=E[\hat{f}(x)]-f(x)$$

• What is the relation between $b$ and $Bias$? Are they the same thing?
• Regarding the equation calculation, is $E[\hat{f}(x)]-f(x)$ the sum of errors like $b$?

In the first example, bias refers to the intercept of a linear model. It is simply a parameter just like $w$ that needs to be learned.
• So it seems they don't have any relation except they both are called bias. For the second $Bias$, do people actually calculate/analyze it for a certain purpose? Or it is simply an abstract concept for understanding the $Bias VS. Variance$? Jun 18, 2018 at 14:47
{"ft_lang_label":"__label__en","ft_lang_prob":0.76765037,"math_prob":0.99984217,"size":695,"snap":"2022-05-2022-21","text_gpt3_token_len":229,"char_repetition_ratio":0.105643995,"word_repetition_ratio":0.0,"special_character_ratio":0.32517985,"punctuation_ratio":0.11643836,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":1.0000048,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2022-05-17T09:02:50Z\",\"WARC-Record-ID\":\"<urn:uuid:5ba0978f-c74c-4ead-85b4-250be59c0761>\",\"Content-Length\":\"223534\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:98a7ed34-fbb3-4791-8ea0-1b7ad89ee5f4>\",\"WARC-Concurrent-To\":\"<urn:uuid:1ed4014d-8d73-42fe-9aee-d0df3cd1e82c>\",\"WARC-IP-Address\":\"151.101.193.69\",\"WARC-Target-URI\":\"https://stats.stackexchange.com/questions/351904/are-the-bias-term-b-and-the-bias-in-bias-variance-tradeoff-the-same-thing\",\"WARC-Payload-Digest\":\"sha1:QMJLZ3XTI3O4PLYILAHVG3UVWKSI56NX\",\"WARC-Block-Digest\":\"sha1:2HMLP22LAEOGMNU7TT3KY3JD3VNNK3JZ\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2022/CC-MAIN-2022-21/CC-MAIN-2022-21_segments_1652662517018.29_warc_CC-MAIN-20220517063528-20220517093528-00347.warc.gz\"}"}