Python Machine Learning Deep Learning Data Structure

 

Here are the most popular and usually askable questions about Deep Learning during a job interview or an internship program. I have written each answer in simple language to ensure clarity and ease of understanding.

1. What is Deep Learning?

Deep learning is a subclass of Machine Learning that employs the Neural Network concept. The idea behind deep learning is to build learning algorithms that mimic the human brain. The 'deep' in deep learning refers to the layers of layers in the network.

2. What is Neural Network?

A neural network is a circuit of neurons. Neural Networks are classified into two categories. They are:

  • Biological Neural Network
  • Artificial Neural Network

Biological Neural Network:


In a biological neural network, the input signals are transferred to the cell body through dendrites. The cell body processes the information that they have received from dendrites. And Axon is like a cable through which neurons send information. Finally, synapses allow the nerve cell to pass an electrical and chemical signal to another neuron. This is how biological neuron works.

Artificial Neural Network:

With Neural Networks, obviously, we want to talk about Artificial One. An artificial neural network (ANN) is a collection of artificial neurons, modeled loosely after the human brain, that are designed to recognize patterns.

ANN mainly consists of three layers:

  • input layer
  • one or more hidden layer
  • output layer

 

3. What is the role of the activation function in the Neural Network?

An activation function decides whether a neuron should be activated (fired) or not at the most basic level. The process involves calculating the weighted sum and further adding bias to it on the input before sending it to the next layer.

Some of the activation functions are Linear, Sigmoid, SoftMax, ReLU, etc.

4. What is a convolutional Neural Network? How it is different from a fully connected Neural Network?

Convolutional Neural Networks are a type of artificial neural network that is frequently used for analyzing visual information. Convolutional Neural Network includes the following four stages:

  • Convolution Layer
  • Normalization
  • Pooling
  • Fully Connected

When it comes to large-dimension images, the number of neurons to process an entire complex image set will be very high. This leads to overfitting because all of the hidden layers are connected in a feed-forward neural network. This is not practical. The solution is Convolutional Neural Network (CNN).

In CNNs, every neuron in a layer connects only to a small portion of the preceding layer, rather than to all of its neurons.

So, in convolution neural networks, only the region which is significant are connected to each other.

5. What is Recurrent Neural Network?

A recurrent neural network (RNN) is a sort of artificial neural network (ANN) that makes use of time series or sequential data to learn. The previous step's output is provided as input to the next phase in RNN.

For example, while predicting the next word of a sentence, the previous words are necessary, and so the previous words must be remembered. As a result, RNN was created, which solved the problem with the help of a Hidden Layer.

The evolution of RNN:

For more details, watch this video:

6. What is gradient descent? What are its type?

A gradient measures how much the output of a function changes if we change the input a little bit. Gradient descent is an iterative optimization algorithm for finding the local minimum of a function. This algorithm tries to minimize the error by taking small steps toward the minimum value. These steps involve updating the weights and biases in a neural network.

There are three types of gradient descent. They are:

  • Batch Gradient Descent
  • Stochastic Gradient Descent
  • Mini-Batch Gradient Descent

Batch Gradient Descent:

In batch gradient descent, we go through all training samples and calculate cumulative error and then we backpropagate to adjust the weight. This technique works well with small training sets. Batch Gradient Descent is often referred to as Vanilla Gradient Descent.

Stochastic Gradient Descent (SGD):

This method uses one (randomly picked) sample for a forward pass and then adjusts weight. It is good when the training set is very big and we don't want too much computation.

Mini batch gradient descent:

It is a combination of batch gradient descent and SGD. In this method, instead of choosing one randomly picked training sample, we will use a batch of randomly picked training samples.

7. Difference between the loss function and cost function?

The cost function is the average of the loss functions for all training sets, whereas the loss function computes the error for a single training set.

8. What is the role of Adam Optimizer?

Basically, the optimizer updates network weights iteratively based on the training data. Adam is a stochastic gradient descent replacement optimization technique for deep learning model training. It is a combination of gradient descent with the momentum algorithm and the RMSP (Root Mean Square Propagation) algorithm.

9. What is the learning rate in neural networks?

The step size or learning rate determines how often the weights are adjusted during training. It's a hyperparameter that governs the extent of model changes in response to estimated errors whenever the model weights are updated.

10. What is categorical cross-entropy? When to use it?

Cross entropy measures the efficiency of a classification model whose final result is a probability that varies from 0 to 1.

Categorical cross-entropy is a loss function that is used in multi-class classification tasks. This method is appropriate when an example can only belong to one out of many possible categories, and the model must decide which one.

11. What is overfitting and underfitting?

A model is said to be overfitted when the model is fitted to training data too well. A model begins to learn from the noise and wrong information when it is trained with such a large amount of data. To solve this, we need to resample the data and estimate the model accuracy using techniques like k-fold cross-validation.

A model is said to be underfitted when the model is unable to understand or extract patterns from the data. It happens when the algorithm used to extract patterns is inappropriate or the data to train the model is very less.

12. What is backpropagation?

  • Backpropagation is a supervised learning method used to train neural networks. 
  • It transfers the error information from the output node to the inner node. It makes it possible to calculate the gradient effectively.
  • It is most useful for feed-forward networks (networks that have no feedback, or simply, have no connections that loop).
  • Backpropagation requires that the activation function used by artificial neurons is differentiable.