Machine Learning Interview Questions

The Naive Bayes method is a supervised learning algorithm, it is naive since it makes assumptions by applying Bayes’ theorem that all attributes are independent of each other.

Bayes’ theorem states the following relationship, given class variable y and dependent vector x1  through xn:

P(yi | x1,..., xn) =P(yi)P(x1,..., xn | yi)(P(x1,..., xn)

Using the naive conditional independence assumption that each xiis independent: for all I this relationship is simplified to:

P(xi | yi, x1, ..., xi-1, xi+1, ...., xn) = P(xi | yi)

Since, P(x1,..., xn) is a constant given the input, we can use the following classification rule:

P(yi | x1, ..., xn) = P(y) ni=1P(xi | yi)P(x1,...,xn) and we can also use Maximum A Posteriori (MAP) estimation to estimate P(yi)and P(yi | xi) the former is then the relative frequency of class yin the training set.

P(yi | x1,..., xn)  P(yi) ni=1P(xi | yi)

y = arg max P(yi)ni=1P(xi | yi)

The different naive Bayes classifiers mainly differ by the assumptions they make regarding the distribution of P(yi | xi): can be Bernoulli, binomial, Gaussian, and so on.

Unsupervised learning is also a type of machine learning algorithm used to find patterns on the set of data given. In this, we don’t have any dependent variable or label to predict. Unsupervised Learning Algorithms:

Clustering,

Anomaly Detection,

Neural Networks and Latent Variable Models.

Example:

In the same example, a T-shirt clustering will categorize as “collar style and V neck style”, “crew neck style” and “sleeve types”.

There are various types of dimensionality reduction techniques including:

Factor Analysis

Random forest

Low variance filter

Missing value ratio

Forward feature selection

Backward feature elimination

Principal Component Analysis (PCA)

Supervised learning is a machine learning algorithm of inferring a function from labeled training data. The training data consists of a set of training examples.

Example: 01

Knowing the height and weight identifying the gender of the person. Below are the popular supervised learning algorithms.

Support Vector Machines

Regression

Naive Bayes

Decision Trees

K-nearest Neighbour Algorithm and Neural Networks.

Example: 02

If you build a T-shirt classifier, the labels will be “this is an S, this is an M and this is L”, based on showing the classifier examples of S, M, and L.

There are six types of kernels in SVM:

Linear kernel - used when data is linearly separable.

Polynomial kernel - When you have discrete data that has no natural notion of smoothness.

Radial basis kernel - Create a decision boundary able to do a much better job of separating two classes than the linear kernel.

Sigmoid kernel - used as an activation function for neural networks.

An Artificial Neural Network (ANN) is an advanced computational model inspired by the human brain. It's characterized by its structure of interconnected nodes and their ability to perform tasks, such as pattern recognition, through learning algorithms.

Key Components

• Nodes (Neurons)
• Receives input signals, processes them, and generates an output signal.
• Applies an activation function to the combination of inputs and node weights to determine its output.

• Edges (Synaptic Weights)
• Represent the strength of the connection between nodes.
• Adjusted during learning to optimize task performance.

Neural Network Kinds

• Feedforward Neural Network (FNN)
• Information flows in one direction without loops.
• Commonly used in basic classification tasks.

• Recurrent Neural Network (RNN)
• Internally possesses recurrent connections, enabling them to exhibit temporal dynamic behavior.
• Best suited for sequence-related tasks, such as natural language processing and speech recognition.

• Convolutional Neural Network (CNN)
• Uses spatial arrangement of data in images to its advantage by applying a series of convolutional, pooling, and fully connected layers.
• Specialized for tasks involving images, such as object detection and classification.

• Modular Neural Network
• Comprises multiple, autonomous neural networks or neural-like circuits, each responsible for a specific sub-task.

• Hybrid Neural Network
• Combines multiple neural network types to derive the benefits of each in solving particular kinds of problems.

Training Mechanism

• Supervised Learning: Labeled input data guides network learning.
• Unsupervised Learning: The network processes unlabeled data to find patterns and structures.
• Reinforcement Learning: Network learns via a feedback loop where it's rewarded or penalized based on its actions.

Mathematical Underpinnings

The functioning of an ANN is mathematically informed by methods such as the backpropagation algorithm and gradient descent, which iteratively optimizes network parameters to minimize error.

Activation Functions

Activation functions, like the sigmoid or rectified linear unit (ReLU), produce nonlinear outputs critical for enabling ANNs to model complex relationships efficiently.

Node Operations

Nodes integrate weighted inputs, typically summing them up, then apply an activation function to the result. The weighted sum is often represented as:

$$ z = \sum_{i=1}^{n} w_{i}x_{i} + b $$

Where:

• $w_{i}$ and $x_{i}$ are the weights and inputs, respectively
• $b$ is the bias term
• $n$ is the number of inputs

The simplest answer is to make our lives easier. In the early days of “intelligent” applications, many systems used hardcoded rules of “if” and “else” decisions to process data or adjust the user input. Think of a spam filter whose job is to move the appropriate incoming email messages to a spam folder.

But with the machine learning algorithms, we are given ample information for the data to learn and identify the patterns from the data.

Unlike the normal problems we don’t need to write the new rules for each problem in machine learning, we just need to use the same workflow but with a different dataset.

Let’s talk about Alan Turing, in his 1950 paper, “Computing Machinery and Intelligence”, Alan asked, “Can machines think?”

The paper describes the “Imitation Game”, which includes three participants -

Human acting as a judge,

Another human, and

A computer is an attempt to convince the judge that it is human.

The judge asks the other two participants to talk. While they respond the judge needs to decide which response came from the computer. If the judge could not tell the difference the computer won the game.

The test continues today as an annual competition in artificial intelligence. The aim is simple enough: convince the judge that they are chatting to a human instead of a computer chatbot program.

The algorithm learns mathematical parameters directly during the training phase. Conversely, the developer sets architectural hyperparameters manually before the training phase begins to guide the learning process.

A Support Vector Machine (SVM) is an algorithm that tries to fit a line (or plane or hyperplane) between the different classes that maximizes the distance from the line to the points of the classes.

In this way, it tries to find a robust separation between the classes. The Support Vectors are the points of the edge of the dividing hyperplane as in the below figure.

A Support Vector Machine (SVM) is a very powerful and versatile supervised machine learning model, capable of performing linear or non-linear classification, regression, and even outlier detection.

Suppose we have given some data points that each belong to one of two classes, and the goal is to separate two classes based on a set of examples.

In SVM, a data point is viewed as a p-dimensional vector (a list of p numbers), and we wanted to know whether we can separate such points with a (p-1)-dimensional hyperplane. This is called a linear classifier.

There are many hyperplanes that classify the data. To choose the best hyperplane that represents the largest separation or margin between the two classes. If such a hyperplane exists, it is known as a maximum-margin hyperplane and the linear classifier it defines is known as a maximum margin classifier. The best hyperplane that divides the data in H3

We have data (x1, y1), ..., (xn, yn), and different features (xii, ..., xip), and yiis either 1 or -1.

The equation of the hyperplane H3 is the set of points satisfying:

w. x-b = 0

Where w is the normal vector of the hyperplane. The parameter b||w||determines the offset of the hyperplane from the original along the normal vector w

So for each i, either xiis in the hyperplane of 1 or -1. Basically, xisatisfies:

w . xi - b = 1  or   w. xi - b = -1

Some of the most commonly used supervised techniques are:

Naive Bayes'

Random forest

Logistic regression

K nearest neighbor

Multiple linear regression

Support Vector Machines

Some of the commonly used unsupervised techniques are:

Association rules

Clustering techniques

Recommendation systems

Principal Component Analysis

Machine Learning is generally divided into three main categories including:

A. Supervised Learning: It is where an algorithm learns to map input data to a specific output based on example input-output pairs. This process involves training a model using a labeled dataset, which means each input in the dataset is associated with a known and correct output.

B. Unsupervised Learning: It involves analyzing and finding patterns in unlabeled data without any prior training. What makes it different from the supervised learning is that it does not require a teacher to correct its output

C. Reinforcement Learning: Reinforcement learning is an area concerned with how an agent ought to take actions in an environment to maximize the notion of cumulative reward.

Cross-validation is a method of splitting all your data into three parts: training, testing, and validation data. Data is split into k subsets, and the model has trained on k-1of those datasets.

The last subset is held for testing. This is done for each of the subsets. This is k-fold cross-validation. Finally, the scores from all the k-folds are averaged to produce the final score.

Principal component analysis (PCA) is most commonly used for dimension reduction.

In this case, PCA measures the variation in each variable (or column in the table). If there is little variation, it throws the variable out, as illustrated in the figure below:

Thus making the dataset easier to visualize. PCA is used in finance, neuroscience, and pharmacology.

It is very useful as a preprocessing step, especially when there are linear correlations between features.

Underfitting occurs when a model lacks the complexity to capture the underlying data pattern. Simple linear models often underfit highly non-linear data, leading to extremely weak performance on both training and validation sets. Engineers fix this problem by adding better features and increasing the overall model capacity.

These concepts represent different levels of abstraction in computer science.

Term

Meaning

Example

AI

The broad goal of making machines perform intelligent tasks

Planning and language perception

Machine Learning

Systems learn patterns from historical data

Fraud detection and churn prediction

Deep Learning

Neural networks with many internal layers

Image recognition and language models

There are generally two paradigms of ensemble methods including:

Sequential ensemble methods

Parallel ensemble methods

Classification is used to produce discrete results, classification is used to classify data into some specific categories. For example, classifying emails into spam and non-spam categories.

Whereas, regression deals with continuous data. For example, predicting stock prices at a certain point in time.

Classification is used to predict the output into a group of classes. For example, Is it Hot or Cold tomorrow?

Whereas, regression is used to predict the relationship that data represents. For example, What is the temperature tomorrow?

Both scalers bring features onto a comparable scale, but they do it differently and suit different situations.

StandardScaler transforms each feature to have zero mean and unit variance, which is commonly called z-score normalization. It works well when the data is roughly normally distributed and is the right choice for algorithms like SVM , logistic regression, and PCA , which are sensitive to the magnitude of features but not to a fixed range.

MinMaxScaler compresses values into a fixed range, typically 0 to 1. This is preferred for neural networks and KNN , where bounded inputs either speed up convergence or directly affect distance calculations. If your data has hard boundaries that carry meaning, pixel values in an image, for instance, MinMaxScaler preserves that structure better than StandardScaler .

The one case where you skip scaling entirely is tree-based models. Random Forest , XGBoost , and LightGBM split features based on thresholds, not magnitudes. Multiplying every value in a column by a constant does not change where the best split is, so scaling has no effect on their predictions and adds unnecessary complexity to the pipeline.

Residual Networks, or ResNets, are a meaningful architectural development that have significantly impacted the field of deep learning by alleviating the vanishing gradient problem. This divergence from traditional architectures has enabled substantially deeper neural networks, leading to improved accuracy in both training and testing stages.

Vanishing Gradient Problem

In conventional feedforward neural networks, as the gradients back-propagate from the output layer to the input layer during training, they can become exceedingly small. This effect, known as the vanishing gradient problem, leads to sluggish learning, especially in deep networks.

Mathematically, the vanishing gradient problem can be represented as:

$$ \frac{\partial L}{\partial W_{l}} = \frac{\partial L}{\partial a_{l}} \cdot \frac{\partial a_{l}}{\partial W_{l}} \cdot \sigma'(z_{l}) \prod_{i=l+1}^{L} f'(z_i)W_{i}^T $$

where $\frac{\partial L}{\partial a_{l}}$ diminishes as $l$ grows.

Impact of ResNets

ResNets address the vanishing gradient problem using shortcut connections, also referred to as skip connections. These unique features allow for direct information flow between layers, mitigating the issue of vanishing gradients and thereby improving training of very deep networks.

The skip connection can be defined as:

$$ $$ z_{l} &= W_{l} a_{l-1} + b_{l} \\ a_{l} &= g(z_{l}) + a_{l-1} $$ $$

Mathematically, this helps to prevent the vanishing gradient problem, ensuring that $W_{l} a_{l-1}$ does not dominate the activation $a_{l}$.

The structure of a ResNet block is often illustrated as follows:

Practical Accuracy

ResNets, with their characteristic deep layering, have produced state-of-the-art performance on several benchmark image classification tasks. These tasks include the well-known ImageNet dataset.

The consistent top rankings in these competitions have secured ResNets as a vital component of modern deep learning toolkits.