All about Deep Learning Tutorial

Azure Cognitive Services are APIs, SDKs, and services available to help developers build intelligent applications without having direct AI or data science skills or knowledge. Azure Cognitive Services enable developers to easily add cognitive features into their applications. The goal of Azure Cognitive Services is to help developers create applications that can see, hear, speak, understand, and even begin to reason. The catalog of services within Azure Cognitive Services can be categorized into five main pillars - Vision, Speech, Language, Web Search, and Decision.

7. PyTorch

PyTorch is an open-source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing. It is primarily developed by Facebook's artificial intelligence research group. It is free and open-source software released under the Modified BSD license.

8. Eclipse DeepLearning4J (DL4J)

Eclipse Deeplearning4j is the first commercial-grade, open-source, distributed deep-learning library written for Java and Scala. Integrated with Hadoop and Apache Spark, DL4J brings AI to business environments for use on distributed GPUs and CPUs.

9. Lasagne

Lasagne is a lightweight library to build and train neural networks in Theano. Its main features are:

• Supports feed-forward networks such as Convolutional Neural Networks (CNNs), recurrent networks including Long Short-Term Memory (LSTM), and any combination thereof
• Allows architectures of multiple inputs and multiple outputs, including auxiliary classifiers
• Many optimization methods including Nesterov momentum, RMSprop, and ADAM
• Freely definable cost function and no need to derive gradients due to Theano's symbolic differentiation
• Transparent support of CPUs and GPUs due to Theano's expression compiler

10. noLearn

nolearn contains a number of wrappers and abstractions around existing neural network libraries, most notably Lasagne, along with a few machine learning utility modules. All code is written to be compatible with scikit-learn.

11. PyLearn2

Pylearn2 is a machine learning library. Most of its functionality is built on top of Theano. This means you can write Pylearn2 plugins (new models, algorithms, etc) using mathematical expressions, and Theano will optimize and stabilize those expressions for you, and compile them to a backend of your choice (CPU or GPU).

Artificial Neural Networks

A neural network is a series of algorithms that endeavors to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial. Neural networks can adapt to changing input; so the network generates the best possible result without needing to redesign the output criteria. The concept of neural networks, which has its roots in artificial intelligence, is swiftly gaining popularity in the development of trading systems.

Artificial Neural Networks Process

Artificial Neural Networks consist of the following four main parts:

1. Neuron

Artificial Neural Networks contain layers of neurons. A neuron is a computational unit that calculates a piece of information based on weighted input parameters. Inputs accepted by the neuron are separately weighted.

Inputs are summed and passed through a non-linear function to produce output. Each layer of neurons detects some additional information, such as edges of things in a picture or tumors in a human body. Multiple layers of neurons can be used to detect additional information about input parameters.

2. Nodes

Artificial Neural Network is an interconnected group of nodes akin to the vast network of layers of neurons in a brain. Each circular node represents an artificial neuron and an arrow represents a connection from the output of one neuron to the input of another.

3. Inputs

Inputs are passed into the first layer. Individual neurons receive the inputs, with each of them receiving a specific value. After this, an output is produced based on these values.

4. Outputs

The outputs from the first layer are then passed into the second layer to be processed. This continues until the final output is produced. The assumption is that the correct output is predefined.

Each time data is passed through the network, the result is compared with the correct one, and tweaks are made to their values until the network creates the correct final output each time.

Classification of Artificial Neural Networks

1. Shallow neural network

The Shallow neural network has only one hidden layer between the input and output.

2. Deep neural network:

Deep neural networks have more than one layer. For instance, the Google LeNet model for image recognition counts 22 layers.

Types of Deep Learning Networks

1. Feed-forward Neural Networks

The simplest type of artificial neural network. With this type of architecture, information flows in only one direction, forward. It means, the information flows starts at the input layer, goes to the "hidden" layers, and ends at the output layer. The network does not have a loop. Information stops at the output layers.

2. Recurrent neural networks (RNNs)

RNN is a multi-layered neural network that can store information in context nodes, allowing it to learn data sequences and output a number or another sequence. In simple words it an Artificial neural network whose connections between neurons include loops. RNNs are well suited for processing sequences of inputs.

For example, if the task is to predict the next word in the sentence "Do you want a…………?

• The RNN neurons will receive a signal that points to the start of the sentence.
• The network receives the word "Do" as an input and produces a vector of the number. This vector is fed back to the neuron to provide a memory to the network. This stage helps the network to remember it received "Do" and it received it in the first position.
• The network will similarly proceed to the next words. It takes the word "you" and "want." The state of the neurons is updated upon receiving each word.
• The final stage occurs after receiving the word "a." The neural network will provide a probability for each English word that can be used to complete the sentence. A well-trained RNN probably assigns a high probability to "café", "drink," "burger," etc.

Common uses of RNN

• Help securities traders to generate analytic reports
• Detect abnormalities in the contract of financial statement
• Detect fraudulent credit-card transaction
• Provide a caption for images
• Power chatbots
• The standard uses of RNN occur when the practitioners are working with time-series data or sequences (e.g., audio recordings or text).

3. Convolutional neural networks (CNN)

CNN is a multi-layered neural network with a unique architecture designed to extract increasingly complex features of the data at each layer to determine the output. CNN's are well suited for perceptual tasks.

CNN is mostly used when there is an unstructured data set (e.g., images) and the practitioners need to extract information from it

For instance, if the task is to predict an image caption:

• The CNN receives an image of let's say a cat, this image, in computer term, is a collection of the pixel. Generally, one layer for the greyscale picture and three layers for a color picture.
• During the feature learning (i.e., hidden layers), the network will identify unique features, for instance, the tail of the cat, the ear, etc.
• When the network thoroughly learned how to recognize a picture, it can provide a probability for each image it knows. The label with the highest probability will become the prediction of the network.

4. Reinforcement Learning

Reinforcement learning is a subfield of machine learning in which systems are trained by receiving virtual "rewards" or "punishments," essentially learning by trial and error.

Google's DeepMind has used reinforcement learning to beat a human champion in the Go games. Reinforcement learning is also used in video games to improve the gaming experience by providing smarter bot.

One of the most famous algorithms are:

• Q-learning
• Deep Q network
• State-Action-Reward-State-Action (SARSA)
• Deep Deterministic Policy Gradient (DDPG)

The following is a list of Neural Networks with their corresponding use cases

Neural Network	Use Case
ANN (Artificial Neural Network)	Computational Neuroscience
CNN (Convolutional Neural Network)	Image Processing
RNN (Recurrent Neural Network)	Speech Recognition
DNN (Deep Neural Network)	Acoustic Modeling
DBN (Deep Brief Network)	Drug Discovery

Computational Graph

Computational graphs are a nice way to think about mathematical expressions. For example, consider the expression e=(a+b)∗(b+1). There are three operations: two additions and one multiplication. To help us talk about this, let’s introduce two intermediary variables, c, and d so that every function’s output has a variable. We now have:

c=a+b

d=b+1

e=c∗d

To create a computational graph, we make each of these operations, along with the input variables, into nodes. When one node’s value is the input to another node, an arrow goes from one to another.

These sorts of graphs come up all the time in computer science, especially in talking about functional programs. They are very closely related to the notions of dependency graphs and call graphs. They’re also the core abstraction behind the popular deep learning framework Theano.

We can evaluate the expression by setting the input variables to certain values and computing nodes up through the graph. For example, let’s set a=2 and b=1

The output comes out to be 6.

Python Implementation of Computational Graph

from dask import delayed
import dask

@delayed
def cube(num):
    print("Cube function:", num)
    print()
    return num * num * num


@delayed
def sum_list(args):
    print("Sum_list function:", args)
    return sum(args)


items = [1, 2, 3]
computation_graph = sum_list([cube(i) for i in items])
computation_graph.visualize()

Output

Examples of deep learning applications

1. AI in Finance

The financial technology sector has already started using AI to save time, reduce costs, and add value. Deep learning is changing the lending industry by using more robust credit scoring. Credit decision-makers can use AI for robust credit lending applications to achieve faster, more accurate risk assessment, using machine intelligence to factor in the character and capacity of applicants.

Underwrite is a Fintech company providing an AI solution for credit makers company. underwrite.ai uses AI to detect which applicant is more likely to pay back a loan.

2. AI in HR

Under Armour, a sportswear company revolutionizes hiring and modernizes the candidate experience with the help of AI. They had, on average, 30000 resumes a month. Reading all of those applications and begin to start the screening and interview process was taking too long.

At that time, Under Armour had all of the 'must-have' HR technology in place such as transactional solutions for sourcing, applying, tracking and onboarding but those tools weren't useful enough. Under armor choose HireVue, an AI provider for HR solution, for both on-demand and live interviews. The results were bluffing; they managed to decrease by 35% the time to fill. In return, the hired higher quality staff.

3. AI in Marketing

AI is a valuable tool for customer service management and personalization challenges. Improved speech recognition in call-center management and call routing as a result of the application of AI techniques allows a more seamless experience for customers.

For example, deep-learning analysis of audio allows systems to assess a customer's emotional tone. If the customer is responding poorly to the AI chatbot, the system can be rerouted the conversation to real, human operators that take over the issue.

Conclusion

In this article, we studied deep learning, what is deep learning, who invented deep learning, advantages of deep learning, disadvantages of deep learning, drivers of deep learning, the difference between machine learning and deep learning, python deep learning libraries, artificial neural network, artificial neural network process, classification of artificial neural network, types of deep learning networks, computational graph and examples of deep learning applications . Hope you were able to understand each and everything. For any doubts, please comment on your query.

In the next article, we will learn about Data Science.

Congratulations!!! you have climbed your next step in becoming a successful ML Engineer.

Next Article In this Series >> Data Science