ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel

Artificial Neural Network | Artificial Neural Network Architecture |Binary |Bipolar Sigmoid Function

Updated on January 30, 2011

Neural Networks

Before going to the artificial neural networks let me introduce about neural networks. It is an interconnected group of neurons. The main example that I can use here is the human brain. In technical field neural networks are often refers to Artificial Neural Network (ANN) or neural nets. It is mostly used in artificial intelligence and for medical treatments. Artificial Neural Networks is considered as one of the fast growing field in artificial intelligence and have a great future.

Biological Neuron

Our brain is a collection of about 10 billion interconnected neurons. Each neuron is a cell that uses biochemical reactions to receive, process, and transmit information. A neuron dendrite tree is connected to thousands of neighboring neurons. When any one of the neurons fire a positive or negative charge is received by any one of the dendrites. The strengths of all received of all received charges are added together through the process of temporal or spatial summation. Spatial summation is done when several weak signals are converted into a large signal. Temporal summation converts a rapid series of weak pulses from one source into one large signal. The aggregate input is then passed to the soma (cell body).

Single Layer Network

Artificial Neural Network Architecture

Neurons can be referred as arranged in layers. Normally neurons in the same layer behave in the same manner. The main factors which decide the behavior of a neuron are its activation function and pattern of weighted connections over which it sends and receives signals. The arrangement of neurons into layers and the connection patterns within and between the layers is called the net architecture.

Based on the no of layers the network structure is broadly classified into 2 categories depends on the connection establishment in the network. They are

1)Feed-forward network

a) Single layer network

b) Multi layer network

2)Recurrent network

Feed-forward network

a) Single layer network

It has only one layer of connection weights. The input unit will receive signals from the outside world and the response to a signal can be read from the output unit.The input units are directly connected with the output unit and they are generally used for pattern classification.

b) Multilayer network

A multilayer network has more layers of nodes between the input units and output units.Typically there is a layer of weights between 2 adjacent levels of units. Multilayer nets can solve more complicated than single-layer nets, but its training is more difficult.

Recurrent network

Here each node is a processing elementor unit, it may be anyone of the 2 states (Black-Active, White-Inactive). Units are connected to each other with weighted symmetric connection. A positive weighted connection indicates that 2 units tend to activate each other. A negative connection allows an active unit to deactivate from a neighboring unit.

Working:  A random unit is chosen. If any of its neighbors are active, the unit computes the sum of the weights on the connections to those active neighbors. If the sum is a positive result, then the unit becomes active, otherwise it becomes inactive.

Feature: Fault-tolerance: If a new processing element fails completely, the network will still function properly.  


It composed of a number of nodes or units, connected by a link, where each link is associated with a numeric weight. Each unit consists of a set of inputs and only one output or a set of outputs, where each level the weights are updated, named as activation level.

Constructing a neural network for the desired task

Depends on the required task, the number of units, the kind of unit, the type of networks are decided and constructed.  Next the weights are initialized to the network using a learning algorithm. Outputs are derived for a set of training samples.

Units in neural networks

Neural networks are composed of nodes or units connected by links. A link from unit j to unit i will propagate the activation function aj from j to i. Every link also has a numeric weight Wji, which denotes the strength and sign of the connection.  Unit i will first computes a weighted sum of its input

Then it applies an activation function g to this sum to derive the output.The activation function g is used for 2 things. To make the unit to be active (near +1) when the right inputs are given and to make inactive (near 0) when the wrong inputs are given. Second to make the activation nonlinear, otherwise the entire neural network collapses into a simple linear function.

The choices for ginclude two logistic functions called 1) threshold function and 2) sigmoid function. Threshold function will provide output as 1 when the input is positive or 0.

Binary Step Function
Binary Step Function
Binary Sigmoid Function
Binary Sigmoid Function
Bipolar Sigmoid function
Bipolar Sigmoid function

Activation Functions

Activation functions are also called as transfer functions. It is the function that is applied to the net output of any node before it is fed to the next layer. There various types of activation functions used and some are

1) Binary Step Function

Single layer network use a step function to convert the net input, which is a continuously valued variable, to an output unit that is a binary or bipolar signal. This function is called as the threshold function that I have mentioned earlier.

2) Binary Sigmoid function

Sigmoid functions are useful activation functions. Logistic function and hyperbolic function are most common ones in this category. It is very useful in neural networks trained by backpropogation.Logistic function will range from 0 to 1, is usually used as the activation function for neural networks in which the desired output values either binary or between 0 and 1. To emphasize the range of the function, we call it as binary sigmoid.

3) Bipolar sigmoid function

The logistic sigmoid function can be scaled to have any range of the values that is appropriate for a given problem. The most common range is from -1 to 1. We call this as bipolar sigmoid function.

Backpropagation network

It is a general purpose learning algorithm, which is powerful but expensive in terms of computational requirements for training. Training a network using backpropagation involves 3 stages.

1) Feedforward of the input training pattern

2) Backpropagation of the associated error.

3) Adjustment of weights.

In feedforward, every input unit receives an input signal and sends the signals to each of the hidden units. Each output unit computes its activation to form the response of the network for the given input pattern.

While training, each output unit compares its computed activation with its target value to determine the error associated with the pattern with that unit. The mathematical basis for the backpropagation algorithm is the optimization technique known as gradient descent.

Choice of activation functions

An activation function for a backpropogation network should have several important characteristics.

It should be continuous, differentiable and monotonically non-decreasing. It is very advantageous if it’s derivative is easy to compute.

Choice of initial weights and biases

The choice of initial weight will influence whether the network reaches a global or only local minimum of the error. The update of the weight between 2 units depends on both, the derivative of the upper unit’s depth on both, the derivative of the upper unit’s activation function, and the activation function of the lower unit. For this reason, it is important to avoid choices of initial weight that would make it likely that either, activations or derivatives of activations are zero. The values of the initial weight must not be too large; the in initial input signals to each hidden or output unit will be likely to fall in the region where the derivative of the sigmoid function has a very small value. On the other hand, if the initial weights are too small, the net input to a hidden or output unit will be close to 0, which also causes extremely slow learning.

 How long to train the network

Since the common interest for applying a backpropogation network is to achieve a balance between correct responses to training patterns and good responses to new input patterns, it is not necessarily advantageous to continue training until the total squared error usually reaches a minimum. As long as the error for training and testing patterns decreases, training continues. When the error begins to increase, the network is starting to memorize the training patterns. At this point training can be terminated. 

Applications of Neural Networks

Neural network is a fast growing field and most of the complex automated systems have their base in tis. Artificial Intelligence works are expanded by using the functionality of neural networks. Main applications are :

Robotics, Function approximation, time series prediction, Patten recognition, Novelty detection, Fitness approximation

Other applications are


It can be used to pronounce a written text. It can be achieved by mapping the text stream to phonemes basic sound elements and then passing the phonemes to an electric speech generator. It is one of the wonderful application of neural network and it works approximately correct.

Handwritten Character Recognition

An artificial neural network can be trained to recognize the handwritten characters.

Autonomous land Vehicle In a Neural Network

It can learn to steer a vehicle along a road by observing the performance of a human.

Computer data processing

It involves entering, analyzing, summarizing and converting data into other forms which is desirable for a user.

Artificial neural networks summary

Artificial neural networks are studied by considering human brain and it's neural networks. In computer field artificial neural networks have a good future. Researches are going in artificial neural networks field to enhance the use of it.

If you need to know further about Artificial Neural Network feel free to ask your questions in the comment column below.


    0 of 8192 characters used
    Post Comment

    No comments yet.


    This website uses cookies

    As a user in the EEA, your approval is needed on a few things. To provide a better website experience, uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

    For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at:

    Show Details
    HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
    LoginThis is necessary to sign in to the HubPages Service.
    Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
    AkismetThis is used to detect comment spam. (Privacy Policy)
    HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
    HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
    Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
    CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
    Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the or domains, for performance and efficiency reasons. (Privacy Policy)
    Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
    Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
    Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
    Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
    Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
    VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
    PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
    Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
    MavenThis supports the Maven widget and search functionality. (Privacy Policy)
    Google AdSenseThis is an ad network. (Privacy Policy)
    Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
    Index ExchangeThis is an ad network. (Privacy Policy)
    SovrnThis is an ad network. (Privacy Policy)
    Facebook AdsThis is an ad network. (Privacy Policy)
    Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
    AppNexusThis is an ad network. (Privacy Policy)
    OpenxThis is an ad network. (Privacy Policy)
    Rubicon ProjectThis is an ad network. (Privacy Policy)
    TripleLiftThis is an ad network. (Privacy Policy)
    Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
    Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
    Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
    Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
    ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
    Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)
    ClickscoThis is a data management platform studying reader behavior (Privacy Policy)