{"id":18,"date":"2013-09-05T01:00:54","date_gmt":"2013-09-05T01:00:54","guid":{"rendered":"http:\/\/sonny..ogi.edu\/~kgorman\/blog\/?p=18"},"modified":"2013-09-05T01:00:54","modified_gmt":"2013-09-05T01:00:54","slug":"loess-hyperparameters-without-tears","status":"publish","type":"post","link":"https:\/\/www.wellformedness.com\/blog\/loess-hyperparameters-without-tears\/","title":{"rendered":"LOESS hyperparameters without tears"},"content":{"rendered":"

LOESS<\/a> is a classic non-parametric regression technique. One potential issue that arises is that LOESS fits depend on several hyperparameters<\/i> (i.e., parameters set by the experimenter a priori<\/i>). In this post I’ll take a quick look at how to set these.<\/p>\n

At each point in a LOESS curve, the y<\/i>-value is derived from a local, low-degree polynomial weighted regression. The first hyperparameter refers to the degree of the local fits. Most users set degree to 2 (i.e., use local quadratic curves), and with good reason. At degree 1, you’re just computing a local average. Higher degrees than 2 (e.g., cubic) tend to not have much of an effect.<\/p>\n

The other hyperparameter is “span”, which controls the degree of smoothing. A value of 0 uses no context and a value of 1 uses the entire sample (so it will be similar to fitting a single quadratic function to the data). The choice of this value has a major effect on the quality of the fit obtained:<\/p>\n

\"gbu\"<\/a><\/p>\n

For the randomly generated data here, large values of the span parameter (“bad”) produce a LOESS which fails to follow the larger trend, whereas small values (“ugly”) primarily model noise. For this reason alone, the experimenter should probably not be permitted to select the span hyperparameter herself.<\/p>\n

Fortunately, there are several objectives used to determine an “optimal” setting for the span parameter. Hurvich et al. (1998)<\/a> propose a particularly privileged objective, namely minimizing AICC<\/sub><\/i>. This has been used to generate the “good” curve above. Here’s how I did it (adapted from this<\/a> post to R-help):<\/p>\n