title: Unexpected result while using lowess to smooth a data.table column in R
body: <p>I have a data.table <code>test_dt</code> in which I want to smooth the y column using <code>lowess</code> function.</p>
  <pre><code>test_dt &lt;- structure(list(x = c(28.75, 30, 31.25, 32.5, 33.75, 35, 36.25, 
  37.5, 38.75, 40, 41.25, 42.5, 43.75, 45, 46.25, 47.5, 48.75, 
  50, 52.5, 55, 57.5, 60, 62.5, 63.75, 65, 67.5, 70, 72.5, 75, 
  77.5, 80, 82.5, 85, 87.5, 90, 92.5, 95, 97.5, 100, 102.5, 103.75, 
  105, 106.25, 107.5, 108.75, 110, 111.25, 112.5, 113.75, 115, 
  116.25, 117.5, 118.75, 120, 121.25, 122.5, 125, 130, 135, 140, 
  145), y = c(116.78, 115.53, 114.28, 113.05, 111.78, 110.53, 109.28, 
  108.05, 106.78, 105.53, 104.28, 103.025, 101.775, 100.525, 99.28, 
  98.05, 96.8, 95.525, 93.1, 90.65, 88.225, 85.775, 83.35, 82.15, 
  80.9, 78.5, 76.075, 73.675, 71.25, 68.85, 66.5, 64.075, 61.725, 
  59.4, 57.075, 54.725, 52.475, 50.225, 48, 45.75, 44.65, 43.55, 
  42.475, 41.45, 40.35, 39.275, 38.25, 37.225, 36.175, 35.175, 
  34.175, 33.225, 32.275, 31.3, 30.35, 29.45, 27.625, 24.175, 21, 
  18.125, 15.55), z = c(116.778248424972, 115.531456655985, 114.284502467544, 
  113.034850770519, 111.784500981402, 110.533319511795, 109.284500954429, 
  108.034850457264, 106.784502297216, 105.531265565238, 104.278221015846, 
  103.026780249377, 101.775992395759, 100.528761292272, 99.2853168637851, 
  98.043586202838, 96.8021989104315, 95.5702032427799, 93.1041279347743, 
  90.6575956222915, 88.2179393348852, 85.783500434839, 83.3503011023971, 
  82.136280706039, 80.922846825298, 78.4965179152157, 76.0823895453039, 
  73.6686672097464, 71.264486719796, 68.8702598156142, 66.4865368523571, 
  64.1182523898466, 61.7552221811808, 59.4004347738795, 57.0823289450761, 
  54.7908645949795, 52.5071096685879, 50.2308279167219, 47.9940967492558, 
  45.7658417529877, 44.6514226583931, 43.5622751034012, 42.4876666190815, 
  41.4173110074806, 40.3555584369672, 39.3004471381618, 38.2552969838653, 
  37.2202353638959, 36.1963659189447, 35.1889616530209, 34.2004259883859, 
  33.2295174626826, 32.2669278456991, 31.3171387914754, 30.3742375589802, 
  29.4555719783757, 27.6243725086786, 23.9784367995753, 27.625, 
  27.625, 27.625)), row.names = c(NA, -61L), class = c(&quot;data.table&quot;, 
  &quot;data.frame&quot;))
  </code></pre>
  <p>As can be seen in the image below, I am getting an unexpected result. The expected result is that the line (<code>z</code> column) in the graph below should closely follow the points (<code>y</code> column).</p>
  <p><a href="https://i.stack.imgur.com/xg4bf.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xg4bf.png" alt="Smoothing using lowess" /></a></p>
  <p>Here is my code -</p>
  <pre><code>library(data.table)
  library(ggplot2)
  test_dt[, z := lowess(x = x, y = y, f = 0.1)$y]
  ggplot(test_dt) + geom_point(aes(x, y)) + geom_line(aes(x, z))
  </code></pre>
  <p>Q1. Can someone suggest why <code>lowess</code> is not smoothing properly?</p>
  <p>Q2. Since <code>lowess</code> is not working as expected, is there any other function in R that would be more efficient in smoothing the <code>y</code> column without producing a spike (as <code>lowess</code> did on the boundary points)?</p>