This article was originally published at MathWorks’ website. It is reprinted here with the permission of MathWorks.
3 Things You Need to Know
A neural network is a computing model whose layered structure resembles the networked structure of neurons in the brain, with layers of connected nodes. A neural network can learn from data—so it can be trained to recognize patterns, classify data, and forecast future events.
A neural network breaks down your input into layers of abstraction. It can be trained over many examples to recognize patterns in speech or images, for example, just as the human brain does. Its behavior is defined by the way its individual elements are connected and by the strength, or weights, of those connections. These weights are automatically adjusted during training according to a specified learning rule until the neural network performs the desired task correctly.
Why Do Neural Networks Matter?
Neural networks are especially well suited to perform pattern recognition to identify and classify objects or signals in speech, vision, and control systems. They can also be used for performing time-series prediction and modeling.
Here are just a few examples of how neural networks are used:
- Power companies accurately forecast the load on their electric grids to ensure reliability and optimize efficiency of the power generators they operate.
- ATMs reliably accept bank deposits by reading the account number and deposit amount on your check.
- Pathologists rely on cancer detection applications to guide them in classifying tumors as benign or malignant, based on uniformity of cell size, clump thickness, mitosis, and other factors.
Deep Learning
Neural networks that operate on two or three layers of connected neuron layers are known as shallow neural networks. Deep learning networks can have many layers, even hundreds. Both are machine learning techniques that learn directly from input data.
Deep learning is getting lots of attention, and for good reason. It’s achieving results that were not possible before.
Deep learning is especially well suited to complex identification applications such as face recognition, text translation, and voice recognition. It’s also a key technology used in advanced driver assistance systems and tasks including lane classification and traffic sign recognition.
- Introduction to Deep Learning: Machine Learning vs. Deep Learning (3:47)
- Deep Neural Networks (2 Videos)
How Do Neural Networks Work?
A neural network combines several processing layers, using simple elements operating in parallel and inspired by biological nervous systems. It consists of an input layer, one or more hidden layers, and an output layer. The layers are interconnected via nodes, or neurons, with each layer using the output of the previous layer as its input.
Typical neural network architecture.
Techniques Used with Neural Networks
Common machine learning techniques for designing neural network applications include supervised and unsupervised learning, classification, regression, pattern recognition, and clustering.
Supervised Learning
Supervised neural networks are trained to produce desired outputs in response to sample inputs, making them particularly well suited for modeling and controlling dynamic systems, classifying noisy data, and predicting future events. Deep Learning Toolbox™ includes four types of supervised networks: feedforward, radial basis, dynamic, and learning vector quantization.
Classification
Classification is a type of supervised machine learning in which an algorithm “learns” to classify new observations from examples of labeled data.
Regression
Regression models describe the relationship between a response (output) variable and one or more predictor (input) variables.
Pattern Recognition
Pattern recognition is an important component of neural network applications in computer vision, radar processing, speech recognition, and text classification. It works by classifying input data into objects or classes based on key features, using either supervised or unsupervised classification.
For example, in computer vision, supervised pattern recognition techniques are used for optical character recognition (OCR), face detection, face recognition, object detection, and object classification. In image processing and computer vision, unsupervised pattern recognition techniques are used for object detection and image segmentation.
Unsupervised Learning
Unsupervised neural networks are trained by letting the neural network continually adjust itself to new inputs. They are used to draw inferences from data sets consisting of input data without labeled responses. You can use them to discover natural distributions, categories, and category relationships within data.
Deep Learning Toolbox includes two types unsupervised networks: competitive layers and self-organizing maps.
Clustering
Clustering is an unsupervised learning approach in which neural networks can be used for exploratory data analysis to find hidden patterns or groupings in data. This process involves grouping data by similarity. Applications for cluster analysis include gene sequence analysis, market research, and object recognition.
Developing Shallow Neural Networks with MATLAB
With tools and functions for managing large data sets, MATLAB® offers specialized toolboxes for working with machine learning, neural networks, deep learning, computer vision, and automated driving.
With just a few lines of code, MATLAB lets you develop neural networks without being an expert. Get started quickly, create and visualize models, and deploy models to servers and embedded devices.
With MATLAB, you can integrate results into your existing applications. MATLAB automates deploying your neural network models on enterprise systems, clusters, clouds, and embedded devices.
- Workflow for Neural Network Design
- What Is Deep Learning Toolbox? (2:10)
- Wine Classification with Neural Net Pattern Recognition App (3:36)
Typical Workflow for Designing Neural Networks
Each neural network application is unique, but developing the network typically follows these steps:
- Access and prepare your data
- Create the neural network
- Configure the network’s inputs and outputs
- Tune the network parameters (the weights and biases) to optimize performance
- Train the network
- Validate the network’s results
- Integrate the network into a production system
Classification and Clustering of Shallow Networks
MATLAB and Deep Learning Toolbox provide command-line functions and apps for creating, training, and simulating shallow neural networks. The apps make it easy to develop neural networks for tasks such as classification, regression (including time-series regression), and clustering. After creating your networks in these tools, you can automatically generate MATLAB code to capture your work and automate tasks.
Preprocessing, Postprocessing, and Improving Your Network
Preprocessing the network inputs and targets improves the efficiency of shallow neural network training. Postprocessing enables detailed analysis of network performance. MATLAB and Simulink® provide tools to help you:
- Reduce the dimensions of input vectors using principal component analysis
- Perform regression analysis between the network response and the corresponding targets
- Scale inputs and targets so they fall in the range [-1,1]
- Normalize the mean and standard deviation of the training data set
- Use automated data preprocessing and data division when creating your networks
Improving the network’s ability to generalize helps prevent overfitting, a common problem in neural network design. Overfitting occurs when a network has memorized the training set but has not learned to generalize to new inputs. Overfitting produces a relatively small error on the training set but a much larger error when new data is presented to the network.
Two solutions to improve generalization include:
- Regularization modifies the network’s perforance function (the measure of error that the training process minimizes). By including the sizes of the weights and biases, regularization produces a network that performs well with the training data and exhibits smoother behavior when presented with new data.
- Early stopping uses two different data sets: the training set, to update the weights and biases, and the validation set, to stop training when the network begins to overfit the data
Postprocessing plots for analyzing network performance, including mean squared error validation performance for successive training epochs (top left), error histogram (top right), and confusion matrices (bottom) for training, validation, and test phases.
Simulink Support
Deep Learning Toolbox provides a set of blocks for building shallow neural networks in Simulink. All blocks are compatible with Simulink Coder™. These blocks are divided into four libraries:
- Transfer function blocks, which take a net input vector and generate a corresponding output vector
- Net input function blocks, which take any number of weighted input vectors, weight-layer output vectors, and bias vectors, and return a net input vector
- Weight function blocks, which apply a neuron’s weight vector to an input vector (or a layer output vector) to get a weighted input value for a neuron
- Data preprocessing blocks, which map input and output data into the ranges best suited for the neural network to handle directly
Alternatively, you can create and train your networks in the MATLAB environment and automatically generate network simulation blocks for use with Simulink. This approach also enables you to view your networks graphically.
