Dusting off an old artificial neural network code

Posted on Updated on

During my Master Thesis I constructed a ‘standard’ two-layer artificial neural network. It was later expanded and added to our Lyngby Toolbox, that is a Matlab toolbox for analysis of functional neuroimages (and other things you can think of).

Today I heard that Morten Arngren was working with a neural network with pruning and regularization optimization, and I mentioned our Lyngby code.

Now I just wanted to see if it still worked alright, but saw that it lack a small example to get people started. So I made a small noiseless one-dimensional example with a sine wave:

X = 3*rand(50,1); T = sin(X);
[V,W,EAcc,Info] = lyngby_nn_qmain(X, T, 'genoptim', ...
'EarlyStop', 'trainset', 1:25);
X0 = (0:0.1:3)';
[Y, H] = lyngby_nn_qforward(X0, V, W);
plot(X,T,'o', X0,Y,'-')

Here the generalization is just controlled with so-called ‘early stopping’ using the first half of the data set as a training set (25 data points) and the second half for ‘validation’. The forward function generates the output (Y) of the neural network.

The more complicated and longer running ‘Pruning2DRegGridSearch’ option both prunes and optimizes two regularization parameters (ridge regularization parameters): one for the input layer of parameters (weights) and one for the output:

X = 3*rand(50,1); T = sin(X);
[V,W,EAcc,Info] = lyngby_nn_qmain(X, T, ...
'genoptim', 'Pruning2DRegGridSearch', ...
'reg', [10^-5 10 10^-5 10], ...
'trainset', 1:25, 'info', 1);
X0 = (0:0.1:3)';
[Y, H] = lyngby_nn_qforward(X0, V, W);
figure, plot(X,T,'o', X0,Y,'-')
figure, lyngby_nn_plotnet(V,W)

This latter example plots the output and also the optimized neural network structure (see the figures).

In this case the one-dimensional example presents no problem for either approaches. When I used it for funtional magnetic resonance imaging (fMRI) data in my thesis and in the article Plurality and Resemblance in fMRI Data Analysis my recollection is that neural network optimization was a long-winded process.

The Lyngby Toolbox can be downloaded from here.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s