ML Interview Part 1: Foundations, Linear & Logistic Regression

1. Q1. What is Machine Learning, and how does it work internally?

Machine Learning (ML) is a branch of artificial intelligence in which a computer system learns patterns from historical data and uses those patterns to make predictions or decisions on new data.

In traditional programming, a developer explicitly writes rules. In machine learning, a developer provides training data and chooses an algorithm that learns the relationships between inputs and outputs.

Example: a bank wants to predict whether a customer will repay a loan. Traditional programming might use rules such as: reject if the credit score is below 600; reject if monthly income is below ₹20,000; approve if the customer has no previous defaults. These rules can be useful, but they may not capture complicated relationships among income, existing debt, repayment history, employment stability and loan amount.

A machine learning model can learn those relationships from historical loan records, subject to data quality, regulatory requirements and appropriate model validation.

How machine learning works, step by step

1. 1. Collect historical data: income, credit history, debt, repayment outcomes
2. 2. Clean and prepare data: handle missing values, encode categories, scale features if needed
3. 3. Train the algorithm: learn model parameters by minimising a loss function
4. 4. Evaluate on unseen data: measure generalisation, not just training performance
5. 5. Deploy and monitor: generate predictions and watch for drift and errors

What happens during training? Each loan application is represented by a feature vector. The known historical outcome is y, where 1 means default and 0 means no default. The model computes a prediction from the features and its learned parameters.

Feature vector and prediction

x  = [income, credit score, debt]        input features
ŷ  = f_θ(x)                               the model's prediction

f  = the selected model or algorithm
θ  = the parameters learned during training
ŷ  = the predicted outcome (here, probability of default)

The training algorithm compares predictions with actual outcomes using a loss function, then adjusts the parameters to reduce the loss. Gradient descent updates the parameters approximately as follows, where J is the loss, η (eta) is the learning rate and the gradient points in the direction in which the loss increases most rapidly, so we step the opposite way:

Gradient descent update

θ(t+1) = θ(t) − η · ∇θ J(θ(t))

Repeat until a stopping condition: maximum iterations reached, or no meaningful improvement.

Practical Python example: train, test and report

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

# X contains numerical features.
# y contains historical outcomes: 0 or 1.
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,
    random_state=42,
    stratify=y
)

model = LogisticRegression(max_iter=1000)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print(classification_report(y_test, predictions))

This example assumes X and y have already been prepared. Real credit-risk systems need additional preprocessing, leakage checks, calibration, fairness assessments, validation and appropriate approval controls.

2. Q2. Explain Linear Regression, including its mathematics and real-world applications

Linear Regression is a supervised machine learning algorithm used to predict a continuous numerical value: house prices, monthly sales, demand forecasts, delivery times.

It assumes the target can be approximated by a linear combination of the input features. A company might predict monthly sales from advertising spend, the number of sales representatives and website traffic.

The model

One feature:        ŷ = w·x + b
Many features:      ŷ = b + w₁x₁ + w₂x₂ + … + wₙxₙ

x = input feature    w = coefficient (weight)    b = intercept (bias)    ŷ = predicted value

How does it learn? The model starts with initial coefficients, predicts, measures the errors against the known values, and finds the coefficients that minimise the loss. The usual loss is Mean Squared Error. Squaring penalises large mistakes more heavily and stops positive and negative errors from cancelling out.

Mean Squared Error

J(w, b) = (1/n) · Σᵢ ( yᵢ − (w·xᵢ + b) )²

Numerical example. Suppose a simplified fitted model predicts house prices as: Price (lakhs) = 10 + 5 × Bedrooms.

The coefficient 5 means the model associates each additional bedroom with ₹5 lakh more, given the features in this simplified model. It does not establish that adding a bedroom will causally raise a real house's value by that amount.

Fit and evaluate with scikit-learn

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score

# Feature: number of bedrooms
X = np.array([[1], [2], [3], [4], [5]])
y = np.array([15, 20, 25, 30, 35])

model = LinearRegression()
model.fit(X, y)

prediction = model.predict([[3]])

print("Coefficient:", model.coef_[0])
print("Intercept:", model.intercept_)
print("Predicted price:", prediction[0])

print("Training MSE:", mean_squared_error(y, model.predict(X)))
print("Training R2:", r2_score(y, model.predict(X)))

Expected fitted relationship: coefficient approximately 5 and intercept approximately 10. This tiny example demonstrates the mechanics; it does not evaluate real-world predictive performance.

Classical statistical inference also relies on assumptions such as linearity, independent errors and constant error variance. Regularised versions (Ridge, Lasso, Elastic Net) help manage overfitting and correlated features.

3. Q3. Explain Logistic Regression. Why is it used for classification instead of Linear Regression?

Despite its name, Logistic Regression is primarily used for classification. It estimates the probability that an observation belongs to a particular class.

A bank predicting whether a customer will default needs a class (default or no default), but the model first estimates a probability. Linear Regression can output values below 0 or above 1, which are not valid probabilities. Logistic Regression fixes this by passing a linear score through the sigmoid function.

Linear score, then sigmoid

z = w₁x₁ + w₂x₂ + … + wₙxₙ + b

p = σ(z) = 1 / (1 + e^(−z))         always strictly between 0 and 1

Explore the sigmoid function

Training minimises binary cross-entropy (log loss). yᵢ is the actual outcome and pᵢ the predicted probability. The loss heavily penalises confident predictions that are wrong; an iterative optimiser adjusts the coefficients to reduce it.

Binary cross-entropy (log loss)

J = −(1/n) · Σᵢ [ yᵢ·log(pᵢ) + (1 − yᵢ)·log(1 − pᵢ) ]

How classification happens: if the model estimates a default probability of 0.72 and the decision threshold is 0.50, the application is classified as default risk because 0.72 ≥ 0.50. The threshold is a separate business decision from estimating the probability, chosen from validated performance, the cost of each error, fairness and regulatory obligations. A prediction should not decide a loan automatically without suitable controls.

Probabilities and ROC-AUC

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score

# X: prepared customer features
# y: historical binary outcome
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

# Estimated probability of class 1
probabilities = model.predict_proba(X_test)[:, 1]

# Default classification threshold is normally 0.5
predictions = model.predict(X_test)

print("First five probabilities:", probabilities[:5])
print("ROC-AUC:", roc_auc_score(y_test, probabilities))

For imbalanced outcomes also examine precision, recall, PR-AUC and probability calibration. ROC-AUC alone does not tell you whether a particular threshold is suitable for deployment.

4. Series plan