Osama bin Laden and bias in median filters

Posted on Updated on

Nielsen2011median

After the recent killing of Osama bin Laden one issue remains: Has this terrorist leader been involved in the implementation of median filters?

Likely not. However, there is still something fishy about median filters: In a recent blog post I would have liked to use median filtering, but found to my surprise that the standard implementation could be quite biased to an extent that I found it useless for the present application.

Implementations in Python (“medfilt” from “scipy.signal”) and medfilt1 from Matlab are biased towards zero as the signal is zeropadded at the beginning and the end (as far as I can determine). It means that with large median filter kernels and signals with a mean different from zero you get a drop at the end and the beginning of the signal. You see this effect in the plot. The thine black line is the original signal with mean 12, a bit of ordinary noise and a few outliers, while the thicker lines are the outputs from different median filters. The Python standard implementation (green curve) drops at the beginning and end. It is possible to avoid this effect either by padding with values from the signal or shrinking the kernel. Shrinking the kernel seems to perform well in my case: If you look on the blue curve you see it follows the y=12 line (the “truth”) rather nicely. The variance at the boundary will increase though.

The present version of the “median filter” Wikipedia article has a short discussion on this boundary issue.

Surprisingly, Octave’s implementation of “medfilt1” is different from Matlab’s implementation. If you write “plot(medfilt1(data, 51))” you get different results in Matlab and Octave. It seems that Octave implements what I here have termed the “shrink kernel” version of median filtering.

https://gist.github.com/955159

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s