Introduction To Deep Learning

Today, Artificial intelligence(AI) is a thriving field with many practical applications and active research topics. The true challenge to artificial intelligence is to solve problems that human solve intuitively and by observing things like spoken accent and faces in an image.

The solution to the above problem is to allow computers to learn from experience and understand the world in terms of a hierarchy of concepts, with each concept defined in terms of its relation to simpler concepts. By gathering knowledge from experience, this approach avoids the need for human operators to formally specify all of the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason, we call this approach to AI deep learning.

Several artificial intelligence projects have sought to hard-code knowledge about the world in formal languages. A computer can reason about statements in these formal languages automatically using logical inference rules. This is known as the knowledge base approach to artificial intelligence. None of these projects has led to a major success. One of the most famous such projects is Cyc (Lenat and Guha, 1989).

Now, this is not always possible to hard-core each feature in our machine. So the ability to acquire their own knowledge is necessary, can be gained by extracting patterns from raw data. This capability is known as machine learning. The performance of the simple machine learning algorithms depends heavily on the representation of the data they are given. Each piece of information included in the representation of our desired problem is known as a feature.

Importance of features is very crucial, for instance, take an example of human beings, we can easily perform arithmetic on Arabic numbers, but doing arithmetic on Roman numerals is much more time-consuming. It is not surprising that the choice of representation has an enormous effect on the performance of machine learning algorithms.

Now to solve this problem we can use machine learning not only to discover mapping from representation to output but also representation itself. This is called as representation learning. Representation learning, i.e., learning representations of the data that make it easier to extract useful information when building classifiers or other predictors. In the case of probabilistic models, a good representation is often one that captures the posterior distribution of the underlying explanatory factors for the observed input(We will revisit this topic later in greater detail). Talking about representational learning the autoencoder is a good example. An autoencoder is the combination of an encoder function that converts the input data into a different representation, and a decoder function that converts the new representation back into the original format.

Of course, it can be very difficult to extract such high-level, abstract features from raw data. Many of representations, such as a speaker’s accent, can be identified only using sophisticated, nearly human-level understanding of the data. It is nearly as difficult to obtain a representation as to solve the original problem, representation learning does not, at first glance, seem to help us.

Deep learning solves this central problem in representation learning by introducing representations that are expressed in terms of other, simpler representations. Deep learning allows the computer to build complex concepts out of simpler concepts. There are two main ways of measuring the depth of a model:

   i. A number of sequential instructions that must be executed to evaluate the architecture.
   ii. The depth of the graph describing how concepts are related to each other.

It is not always clear which of these two views, the depth of the computational graph, or the depth of the probabilistic modeling graph   is most relevant, and because different people choose different sets of smallest elements from which to construct their graphs, there is no single correct value for the depth of an architecture, just as there is no single correct value for the length of a computer program. Nor is there a consensus about how much depth a model requires to qualify as “deep.”