11/28/2017 Creating Neural Networks in Python | Electronics360 · 11/28/2017 Creating Neural...
-
Upload
hoangduong -
Category
Documents
-
view
226 -
download
2
Transcript of 11/28/2017 Creating Neural Networks in Python | Electronics360 · 11/28/2017 Creating Neural...
11/28/2017 Creating Neural Networks in Python | Electronics360
http://electronics360.globalspec.com/article/8956/creating-neural-networks-in-python 1/3
HOME INDUSTRIES INFORMATION TECHNOLOGY ARTICLE
An illustrative example of an artificial neural network showingnodes and the links between them. Image credit: Jonathan
Heathcote
Information Technology
Creating Neural Networks in PythonEric Olson
16 June 2017
Artificial neural networks are machine learningframeworks that simulate the biological functions ofnatural brains to solve complex problems like image andspeech recognition with a computer. In real brains,information is processed by building block cells calledneurons. A detailed understanding of exactly how thisoccurs is the subject of ongoing research. But at a basiclevel, neurons receive signals through their dendritesand output signals to other neurons through axons oversynapses.
Neural networks attempt to mimic the behavior of realneurons by modeling the transmission of informationbetween nodes (simulated neurons) of an artificial neural network. The networks “learn” in an adaptive way,continuously adjusting parameters until the correct output is produced for a given input. Signal intensities alongnode connections are adjusted through activation functions by evaluating prediction errors in a trial and errorprocess until the network learns the best answer.
Library Packages
Packages for coding neural networks exist in most popular programming languages, including Matlab, Octave,C, C++, C#, Ruby, Perl, Java, Javascript, PHP and Python. Python is a highlevel programming languagedesigned for code readability and efficient syntax that allows expression of concepts in fewer lines of code thanlanguages like C++ or Java. Two Python libraries that have particular relevance to creating neural networks areNumPy and Theano.
NumPy is a Python package that contains a variety of tools for scientific computing, including an Ndimensionalarray object, broadcasting functions, and linear algebra and random number capabilities. NumPy offers anefficient multidimensional container of generic data with arbitrary definition of data types, allowing seamlessintegration with a wide variety of databases. NumPy’s functions serve as fundamental building blocks forscientific computing, and many of its features are integral in developing neural networks in Python.
Weekly NewsletterGet news, research, and analysis on the Electronics industry in your
inbox every week for FREE
Sign up for our FREE eNewsletter
Get the Free eNewsletter
Date Event Location
30 Nov01 Dec2017
Slush Helsinki,Finland
2327 Apr2018
2018 IEEE RadarConference
OklahomaCity,Oklahoma
1822 Jun2018
2018 Symposiaon VLSITechnology &Circuits
Honolulu,Hawaii
IEEE.org About Advertise
Search Electronics360 Acquired Electronics360
Share
CALENDAR OF EVENTS
Find Free Electronics Datasheets
Enter a part number
Home Industries Supply Chain Product Watch Teardowns Hot Topics Calendar Multimedia
Engineering Newsletter Signup
11/28/2017 Creating Neural Networks in Python | Electronics360
http://electronics360.globalspec.com/article/8956/creating-neural-networks-in-python 2/3
A three layer neural network in Python. Image credit: iamtrask (Click to enlarge)
NumPy is a library package for thePython programming language that
can be used to develop neuralnetworks, among other scientific
computing tasks.
Theano—a Python library that defines, optimizes and evaluates mathematicalexpressions—integrates neatly with NumPy. It has unittesting and selfverification features to identify errors; efficient symbolic differentiation; dynamicgeneration of C code for quick expression evaluation; and is capable of utilizingthe graphics processing unit (GPU) transparently for very fast dataintensivecomputations. It can model computations as graphs and use the chain rule tocalculate complex gradients. This is particularly applicable to neural networksbecause they are readily expressed as graphs of computations. Utilizing theTheano library, neural networks can be written more concisely and executemuch faster, especially if a GPU is employed.
Other Python libraries designed to facilitate machine learning include:
· scikitlearn: a machine learning library for Python built on NumPy, SciPy and matplotlib.
· TensorFlow: a machine intelligence library that uses data flow graphs to model neural networks, capable ofdistributed computing across multiple CPUs or GPUs.
· Keras: a high level neural network application program interface (API) able to run on top of TensorFlow orTheano.
· Lasagne: a lightweight library for constructing neural networks in Theano. Lasagne serves as a middleground between the high level abstractions of Keras and the lowlevel programming of Theano.
· mxnet: a deep learning framework capable of distributed computing with a large selection of languagebindings, including C++, R, Scala and Julia.
Three Layer Neural Network
A simple three layer neural network can be programmed in Python as seen in the accompanying image fromiamtrask’s neural network python tutorial. This basic network’s only external library is NumPy (assigned to ‘np’).It has an input layer (represented as X), a hidden layer (l1) and an output layer (l2). The input layer is specifiedby the input data, which the network attempts to correlate with the output layer across the second hidden layerby approximating the correct answer in a trial and error process as the network is trained.
The first twolines of thisPython neuralnetwork assignvalues to theinput (X) andoutput (Y)datasets. Thenext two linesinitialize therandom weights(syn0 and syn1),which areanalogous tosynapses
connecting the network’s layers and are used to predict the output given the input data. Line 5 sets up the mainloop of the neural network that iterates many times (60,000 iterations in this case) to train the network for thecorrect solution. Lines 6 and 7 create a predicted answer using a sigmoid activation function and feed itforward through layers l1 and l2. Lines 8 and 9 assess the errors in layers l2 and l1. Lines 10 and 11 updatethe weights syn1 and syn0 with the error information from lines 8 and 9. The process happening in lines 8through 11 is referred to as backpropagation, which uses the chain rule for derivative calculation. In thisapproach, each of the network’s weights is updated to be closer to the real output, minimizing the error of each“synapse” as the iterative procedure progresses.
Additional operations that can improve this basic neural network for certain datasets include adding a bias andnormalizing the input data. A bias allows shifting the activation function to the left or right to better fit the data.Normalizing the input data, meanwhile, is important for obtaining accurate results for some datasets, as manyneural network implementations are sensitive to feature scaling. Examples of data normalization includescaling each input value to fall between 0 and 1, or standardizing the inputs to have a mean of 0 and avariance of 1.
With the capability to adaptively learn, neural networks have a powerful advantage over conventionalprogramming techniques. In particular, they are useful in applications that are difficult for traditional code tohandle like image recognition, speech synthesis, decision making and forecasting. As the processing power of
Get the Engineering360Electronics360 Newsletter
Stay up to date on:Features the top stories, latest news, charts, insightsand more on the endtoend electronics value chain.
SUBSCRIBE NOW
Business Email Address
SIGN UP
You can change your email preferences at any time.Read our full privacy policy.
United States
11/28/2017 Creating Neural Networks in Python | Electronics360
http://electronics360.globalspec.com/article/8956/creating-neural-networks-in-python 3/3
Powered by CR4, the Engineering Community
Barracuda Goes PrivateDATA CENTER AND CRITICAL INFRASTRUCTURE
New Animation Method Develops Better Lightin Computer GraphicsDATA CENTER AND CRITICAL INFRASTRUCTURE
Facebook Developing Method to AnimateProfile Photos Based on Reaction to PostsDATA CENTER AND CRITICAL INFRASTRUCTURE
HighSpeed Quantum Encryption Could Makethe Future Internet Even More SecureDATA CENTER AND CRITICAL INFRASTRUCTURE
Researchers Developed Faster, LongerLasting BatteriesINFORMATION TECHNOLOGY
Ceva Has Vision For Neural NetworkProcessingPROCESSORS
Intel Follows Qualcomm Down NeuralNetwork PathCOMMENTARY
BrainInspired AI Super Computing SystemDevelopedINFORMATION TECHNOLOGY
IBM Seeks Customers For Neural NetworkBreakthroughPROCESSORS
Here is the First Embedded Neural Networkon a USB StickINFORMATION TECHNOLOGY
computer hardware continues to increase and new architectures are developed, neural networks promise to beinfluential tools able to handle increasingly complex tasks that previously only humans could manage.
Discussion – 0 commentsBy posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Add your comment
INFORMATION TECHNOLOGY RELATED ARTICLES
Editorial Team AdvertisingClient Services Terms of Use
Electronics360Datasheets360 Engineering360360 Websites
Home | Site Map | Accessibility | Nondiscrimination Policy | Privacy & Opting Out of Cookies
© Copyright 2017 IEEE GlobalSpec All rights reserved. Use of this website signifies your agreement to the IEEE Terms and Conditions.
Get the Engineering360Electronics360 Newsletter