# Huber loss / np.where

Yao Yao on June 12, 2019

Huber loss:

$L_{\delta }(y,f(x)) = {\begin{cases}{ \frac{1}{2}}(y-f(x))^{2} & {\textrm {for}}|y-f(x)| \leq \delta, \newline \delta \,|y-f(x)|-{\frac{1}{2}} \delta^{2} & {\textrm {otherwise.}} \end{cases}}$
• Usually set $\delta = 1$
• Huber loss is less sensitive to outliers (those $y$ such that $\vert y-f(x) \vert > \delta$) than the MSE
• And converges faster than the mean absolute error (because of larger gradients).

A python implementation:

def huber_fn(y_true, y_pred):
# Suppose delta == 1
error = y_true - y_pred

is_small_error = tf.abs(error) < 1

squared_loss = tf.square(error) / 2
linear_loss = tf.abs(error) - 0.5

return tf.where(is_small_error, squared_loss, linear_loss)


If all the arrays are 1-D, np.where(condition, X, Y) is equivalent to:

for (c, x, y) in zip(condition, X, Y):
yield c ? x : y
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 2, "mathjax_display_tex": 0, "mathjax_asciimath": 1, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.9512661099433899, "perplexity": 14479.073709129483}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2021-17/segments/1618038887646.69/warc/CC-MAIN-20210419142428-20210419172428-00609.warc.gz"}