Neural Network and Connectionism

Gediz GÜRSU
Aug 22, 2016
3 min read

Artificial Intelligence is a vast subject. There are many papers published since 1950's,. Main topics are biological and artificial neuron models, machine learning algorithms and different network and learning approaches. There are hundreds of computational neuron models inspired from biological neurons. There is also identical realistic models mimicking biological neuron. Some of them are in ModelDB. You can also find very detailed biological models in neuromorpho.org . A realistic digital reconstruction of rat somatosensory cortex and its experimental data set can be obtained from Neocortical Microcircuit Collaboration Portal. Clustering of neuron types based on electrophysiological similarities can be found in neuroelectro.org.

To make things easily understandable, we can state that advances in neurocognitive science are not entirely straightforward. In 2004, it is discovered that neurons can signal backwards. It is a very significant finding. It opens a new era for neuronal and connectionist learning techniques and algorithms. There was already a back-propagation learning algorithm for computational models, however, backwards propagating action potential explains a lot.

From software lists NEURON 0.74, Nengo , NeurophStudio and Simbrain are ones I have been using. I am also using sickit-machine learning library in python. Apart from that I am coding from scratch using pythons numpy, scipy and numba libraries.

Logic of neuronal encoding and decoding is very related to control systems, signal processing, chaos theory, nonlinear systems and machine learning. Its essentially a pattern recognition, reduction and an efficient state storing mechanism. In addition to computational neural network studies, I have a project in neural networks and it is greatly related to knowledge base design, language learning and creative intelligence. The aim of my study is to create a creative thinking and joking AI which would have a superior pattern recognition capability. At the hearth of it, a cognitive architectural model with neural circuitry and knowledge base system is the final aim of my AI project. I will mention it in this site as a project. However this limited post is about the ability to simulate neural networks.

Some simple examples from what I have done so far are :

An Electronic Neuron (Harmon Model) :

I have created and simulated this electronic neuron with five excitatory input and two inhibitory inputs. As it can be seen that simulating using electronic models are somewhat cumbersome due to physical port and connection limitations.

A Spiking Neural Network Learning a 3D Orbit :

I have simulated this 3d orbital path in nengo using spiking (realtime working) neurons. Using 2000 neurons , 3d orbital path could be taught to this neural network ensemble easily. Texture like graph is the firing of spiking neurons. You can zoom in using control and trackball to trace it better.

An Odor Smelling RAT Simulation in Simbrain :

In this simulation rat tries to distinguish cheese odor from other items. No it doesn't depend if the rat is hungry or not :)). I have thousands of pages for encyclopedic and academic explanations for neural networks and machine learning. Thus it is impossible to share all of them here. However I can share some of the interesting ones here.

There are software lists neural network design and computation : https://wiki.python.org/moin/PythonForArtificialIntelligence

https://www.dmoz.org/Computers/Artificial_Intelligence/Neural_Networks/Software/

http://www.scipy.org/topical-software.html

https://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators

http://zhar.net/howto/html/ A glimpse to hardware implementations : http://www.artificialbrains.com/

http://web.stanford.edu/group/brainsinsilicon/neurogrid.html

http://apt.cs.manchester.ac.uk/projects/SpiNNaker/

https://en.wikipedia.org/wiki/Optical_computing

Some Books:

http://www.bem.fi/book/

http://hagan.okstate.edu/NNDesign.pdf

http://www.labri.fr/perso/nrougier/downloads/Erice-2016.pdf