Exams

{0}------------------------------------------------

Description

Consider a dataset with 2 objects per class and $\mathbf 2$ features. For the negative class, the objects are located $$ at [0,0] and [-0.2,-0.1] and for positive class at [1,1.5] and [1,1]. Give the weight vector $\boldsymbol{\mathrm{w}}$ and the number of support vectors (SVs) that the support vector machine $$ will find.

$\mathbb{W}$ What is $\mathbf{w}$ ?

$\Box \ \mathbf{w} = [-1,-1], w_0 = 1$ $\mathbf{w}=[1,1],w_0=-1$

$A$ Model Answer

weight: 1.0

Image /page/0/Picture/7 description: The image shows a light blue speech bubble with an information icon on the left and the text "Nov 10 23:16:36" on the right.

{1}------------------------------------------------

Consider the following image that represents an image that is 8 pixels high and 24 pixels wide, where white pixels represent a value of 1 and black pixels a value of 0:

Image /page/1/Picture/2 description: The image shows the word "EXAM" in white letters on a black background. The letters are pixelated.

Suppose we apply a convolution on the image, using the following 3 by 3 kernel:

$K = \begin{bmatrix} 1, 0, 1 \ 0, 2, 1 \ 1, 0, 1 \end{bmatrix}.$

What will the maximum value in the convolved image be?

{2}------------------------------------------------

Gradient Descent

Indicate which claim is TRUE

$\text{Select } \textbf{one } \text{option}$

• ☐ Gradient descent is used to maximize the value of a function
• $\Box$ Gradient descent is guaranteed to end up in a (local) minimum if we take a sufficient number of steps
• $\hfill \Box$ <br> Gradient descent is usually preferred over solving the normal equations to train linear regression models
• Gradient descent only uses information on the first order derivatives of the objective function, and does not use information about the second-order derivative

$\mathbf A$ Model Answe

{3}------------------------------------------------

{4}------------------------------------------------

Image /page/4/Figure/1 description: The image shows a neural network diagram. The network has three input nodes labeled x0=1, x1, and x2. There are two hidden nodes labeled h1 and h2, and an output node labeled y-hat. There is also a bias node h0=1 connected to the output node. The connections between the nodes are represented by arrows, with associated weights. The weights are as follows: x0 to h1 is 0.5, x1 to h1 is -0.5, x2 to h1 is 0.5, x0 to h2 is -5, x1 to h2 is 1, x2 to h2 is 1, h0 to y-hat is 0, h1 to y-hat is -1, and h2 to y-hat is -1.

Consider the multilayer perceptron depicted in the figure, where the numbers next to the edges denote the weight of each connection. Suppose we use a logistic activation function for both the hidden nodes and the output node.

If the input to this model is $\mathbf{x}^T = [\,\infty, 0\,],$ what will the value for $\hat{y}$ be?

♣ collapse $A$ Model An Select one option $\Box \; \hat{y} = \infty$ $\Box \; \hat{y} = 1$ Explanation and SO:

weight: 1.0

{5}------------------------------------------------

The difference between the apparent error and the true error is called: $$

Select one option

• Overfitting
• Underfitting
• $\Box$ Bayes error
• $\hfill \Box$ <br> Training error

Difference is overfitting, indicating that the predictor has learned the peculiarities of the training set and does not generalise well to new, unseen data.

weight: 1.0

$A$ Model Answer

{6}------------------------------------------------

Fairness: independence & separation

Select one statement that is $\ensuremath{\mathsf{FALSE}}$

$\text{Select } \textbf{one } \text{option}$

• In the context of predicting whether people should receive a loan, the Independence criterion allows the bank to make a larger profit than the Separation criterion.
• ☐ The independence criterion is satisfied if in each subgroup (race, gender, etc.) the same percentage of people receive a positive classification result.
• ☐ The Separation criterion can be evaluated using an ROC curve.
• ☐ Satisfying one of the statistical fairness notions, such as the Independence criterion or the Separation criterion, doesn't guarantee that the machine learning model makes fair decisions.

$A$ Model Answer

{7}------------------------------------------------

weight: 1.0

Consider training a so-called "large language model" (for instance, a simplified ChatGPT). Our model needs to take in the previous word in a sentence and predict the next word. Suppose there are 30,000 unique words in our language, we have a training dataset of 5 million words, we train the model for 20 epochs and we use a learning rate of 0.001. How many classes does this classification problem have?

{8}------------------------------------------------

Fairness: sensitive attributes

Imagine a loan approval system that considers a person's credit score and their gender, among several factors for making loan decisions. In the context of this story, what would be the most appropriate action in fair machine learning?

$\text{Select } \textbf{one } \text{option}$

$\mathbf A$ Model Answer

weight: 1.0

☐ Include gender as a significant feature in the model to ensure equal representation.

☐ Ignore gender entirely and rely on other attributes for decision-making.

Utilize sensitive attributes like gender in the model but apply fairness-aware techniques to mitigate bias.

☐ Share gender information publicly for transparency in loan approval processes.

{9}------------------------------------------------

Layout: III ▾

Gini index

Rather than using the entropy to measure node impurity, in this assignment we are going to use the gini index in our splitting criterion to construct decision trees. If we let $p_c$ be the proportion of objects belonging to class $c$ in a set of objects $S$ , the Gini index is defined as:

$I(S) = \sum_{c \in C} p_c (1-p_c).$

Just like in the case of entropy, the gain in purity is given by: $$

$Gain(S) = I(S) - \sum_{v \in V} \frac{|S_v|}{|S|} I(S_v)$

where $V$ contains the indices of the subsets obtained after splitting, $S_v$ are the objects in the subset $v$ and $|S|$ indicates the number of objects in a set $$

Implement the following two functions:

• 1. Function $\texttt{compute\_gini\_index(labels)}$ that calculates the Gini index for a set of objects.
• Function compute_gain(labels, subsets) that calculates the gain in purity in terms of the Gini index for a given set of objects and division into subsets.

labels is a numpy.ndarray that consists of labels corresponding to the objects. subsets is a numpy.ndarray that consists of the subset IDs of each object.

Note: In order to call the functions in your Solution class use: Solution.compute_gini_index(labels) and <pre>Solution.compute_gain(labels, subsets).</pre>

Example:

Given an array labels of 3 classes: $[0,1,2,0,1]$ <code>compute_gini_index</code> should return: $I(S) = \frac{2}{5} \cdot \frac{3}{5} + \frac{2}{5} \cdot \frac{3}{5} + \frac{1}{5} \cdot \frac{4}{5} = 0.64$

Given a division into <code>subsets</code> : $[0,1,2,0,2]$ (i.e., first object is in <code>subset</code> 0, etc.) <code>compute_gain</code> should return: $Gain(S) = 0.64 - 0.2 = 0.44$

X Nov 10 '23 16:36

{10}------------------------------------------------

Naive Bayes

Identify a FALSE statement about Naive Bayes.

Select one option

The Naive Bayes classifier using a Gaussian distribution per feature will always give a linear decision boundary.

$\Box$ Naive Bayes is not affected by feature scaling.

□ When training a Naive Bayes classifier you can choose to ignore an attribute for a data instance, where the attribute's value is missing.

$\hfill \Box$ <br> The Naive Bayes classifier does not model correlations between features.

$\mathbf A$ Model Ansı

{11}------------------------------------------------

Layout: |||| 🔻

Consider the following definition of the logistic regression model: we try to find a function to predict the probability of a discrete label, by considering the class of linear functions mapped through a logistic function:

$h(\mathbf{x}) = \sigma(\mathbf{x}^\top \mathbf{w})$

where $\sigma(z) = \frac{1}{1+\exp{(-z)}}$

Suppose we denote the labels using $y$ and these labels can be either $0$ or $1.$ We define the average negative log likelihood as:

$J(\mathbf{w}) = -\frac{1}{N} \sum_{i=1}^{N} y_i \log(\sigma(\mathbf{x}_i^{\top} \mathbf{w})) + (1 - y_i) \log(1 - \sigma(\mathbf{x}_i^{\top} \mathbf{w})).$

We are interested in minimizing this average log likelihood.

Implement the following:

• 1. the function $compute_n11(w, X, Y)$ that returns the average negative log likelihood for a given $\mathbf{w}$ for a collection of objects (given as an numpy array, where is the number of features) and their labels Y (given as a NumPy array of length N ). w is given as a NumPy array of length D.

Hint: remember that the derivative of the logistic function is $\sigma'(z) = \sigma(z)(1-\sigma(z))$

For example inputs and corresponding outputs, see the two tests in the Test tab.

{12}------------------------------------------------

When class priors change what statement is true about the decision boundary of the Linear Discriminant classifier?

$\textit{Select}\ \textbf{one}\ \textit{option}$

The shape of the decision boundary cannot change, only the location can change.

$\Box$ Only the shape of the decision boundary can change, but not the location.

$\Box$ Both the shape and the location of the decision boundary cannot change.

$\Box$ Both the shape and the location of the decision boundary can change.

The decision boundary stays linear.

weight: 1.0

$A$ Model Answer

{13}------------------------------------------------

Which statement about PCA is false?

$\textit{Select}\ \textbf{one}\ \textit{option}$

$\hfill \Box$ <br> The larger the number of principal components for PCA, the smaller the reconstruction error.

☐ Principal components found by PCA are always orthogonal.

Reducing the dimensionality using PCA must not be done as preprocessing step before applying a classification model.

weight: 1.0

$A$ Model Ansv

☐ Two principal components are ideal for visualization purposes using PCA.

{14}------------------------------------------------

Given a 1-dimensional training data set with four feature vectors. Two points are from the first class and are located at coordinates 0 and 2. The two other points are from the second class and are at 3 and 7. To solve the classification task, we use the nearest mean classifier.

Leave-one-out

Give the leave-one-out estimate for the true error rate.

unight 10

A Model Arm

There are 4 situations Leave out 1. m1=2, m2=5, dec,boundary=3.5, error=0 Leave out 2: m1=0, m2=5, dec.boundary=2.5, error=0 Leave out 3: m1=1, m2=7, dec boundary=4, error=1 Leave out 4: m1=1; m2=3; dec boundary=2; error=0 or 0.5 or 1.0 So, in total an error of 0.25, 0.375 or 0.5

True distributions

weight 1.0

A Model Announ

Assume now that the true class distributions is uniform between 0 and 4 for the first class and uniform between 3 and 8 for the second class. Given equal prior class probabilities, determine the true error the nearest mean classifier will attain using the above training set.

Bayes error

verge 10

A Model Armour

Calculate the difference between the true error rate the nearest mean classifier achieves and the Bayes error for this problem.

The decision boundary of the Bayes classifier is at $d=4.$ That means that no error is made for class $y_1$ , but for class $y_2$ we get . The total error is therefore .

This error is therefore lower than that of the nearest mean.

{15}------------------------------------------------

Image /page/15/Figure/1 description: The image shows two probability density functions (pdfs) of two classes in a 1D feature space. The x-axis represents the feature space, and the y-axis represents the probability density. One pdf, labeled p(x|y1), is a triangle that starts at 1/2 on the y-axis when x is 0 and decreases linearly to 0 when x is 4. The other pdf, labeled p(x|y2), is a rectangle that starts at 1/4 on the y-axis when x is 0 and remains constant until x is 4.

Posterior probabilities

Assume that the class prior probabilities are $p(y_1)=2/3$ and $p(y_2)=1/3$ . Derive the posterior probability for class of .

weight: 1.0

$A$ Model Answer

weight: 1.0

weight: 1.0

$A$ Model Answe

$A$ Model An

For $x=2$ we see that $p(x|\omega_1)=1/2-x/8$ and . Now, using Bayes rule, we obtain .

Decision boundary of Bayes classifier Where is the decision boundary of the Bayes classifier?

The decision boundary will be where the two pdfs, multiplied by the class priors, cross. That is where $(\frac{1}{2} - x\frac{1}{8})\frac{2}{3} = \frac{1}{4}\frac{1}{3}$ so at

Bayes error

How large is the Bayes error?

We need to compute the area of the two class errors. The error for class 1 is $\frac{1}{2} \cdot 1 \cdot \frac{1}{8} = \frac{1}{16}$ . Error on class 2 is . The total error – taking class priors into account – is

Image /page/15/Picture/11 description: The image shows a light blue speech bubble with an information icon on the left and the text "Nov 10 '23 15:36" on the right.

{16}------------------------------------------------

Nested crossvalidation

Assume we are applying a nested crossvalidation to optimise the hyperparameters of a classification model. We use 5-fold for the inner crossvalidation, and 10-fold for the outer crossvalidation. In total our dataset has 100 objects.

Which of the following statements is correct?

Select one option

The validation fold has 18 samples, and the test fold has 10 samples.

$\Box$ The validation fold has 5 samples, and the test fold has 10 samples.

☐ The validation fold has 10 samples, and the test fold has 10 samples.

$\Box$ The validation fold has 10 samples, and the test fold has 18 samples.

Explanation

The outer crossvalidation determines the size of the test fold: $100/10=10.$ That means that for training 90 objects are available.

The inner crossvalidation determines the size of the validation fold: $90/5 = 18$ .

weight: 1.0

$\mathbf A$ Model Answe

{17}------------------------------------------------

Suppose I trained a linear logistic regression model with one feature. I denote this model as $\hat{p}(Y=1|x)$ . Suppose we also know that $\hat{p}(Y=1|1)$ = 0.3 and that What will the value for be?

Select one option

$\hat{p}(Y=1|0) > 0.3$

$\Box \; \hat{p}(Y=1|0) < 0.3$

$\Box \ \hat{p}(Y=1|0) = 0.3$

$\Box$ It's impossible to know based on the information given in the question.

Explanation

It must be 0.5.

weight: 1.0

{18}------------------------------------------------

Continuous Variables

Consider splitting on a continuous feature during the training of a decision tree classifier. Which of the following is always true?

weight: 1.0

$A$ Model An

Select one option

Scaling the feature values has no effect on the information gain.

☐ If the same feature was already used to split in a higher node in the tree (closer to the root), using it again can never improve the tree.

$\Box$ The number of thresholds for the split we need to consider is quadratic in the number of objects in the node.

□ Splitting on a continuous feature will lead to higher information gain than splitting on a discrete feature.

{19}------------------------------------------------

weight: 1.0

$A$ Model Answer

Crossvalidation

Assume we want to train a classifier on a dataset having $N$ samples. In expectation, the average 10-fold crossvalidation error is (ANSWER) the true error for this classifier trained on $N$ samples.

(Fill in the (ANSWER))

Select one option

larger than

☐ smaller than

$\Box$ an unbiased estimator of

$\Box$ (none of the above: we cannot make a claim like this)

Explanation

Because the classifier is trained with slightly less than $N$ samples each time in the crossvalidation, the error is, in expection, a bit worse than the true one.

{20}------------------------------------------------

ML Ethics

Which of the following statements best characterizes the difference between normative and descriptive ethics?

Select one option

$\Box$ Descriptive ethics offers psychological accounts of how we ought to act, whereas normative ethics offers sociological accounts of how we ought to act.

Descriptive ethics offers explanations of, and factual statements about the moral systems people subscribe to, - how we act and understand ourselves as moral beings, while normative ethics offers prescriptive accounts on how we ought to act and understand ourselves as moral beings.

☐ Descriptive ethics offers a characterization of how others behave morally, while normative ethics characterizes how I behave morally.

☐ Descriptive ethics offers a characterization of how we ought to behave while normative ethics characterizes the norms that we as a matter of fact obey as members of society.

weight: 1.0

$A$ Model Answer

{21}------------------------------------------------

Suppose we trained a linear regression model $h(x)$ on a dataset with a single feature $x$ , and we find parameters $w_0=-4$ and $w_1=2$ . Consider the following two predictions of this model: and Which statement is true?

weight: 1.0

{22}------------------------------------------------

ERM

Fill in the blank: In the empirical risk minimization framework, we minimise the empirical risk over all functions that are elements of the (ANSWER).

Select one option

$A$ Model Answer

weight: 1.0

Hypothesis class

$\Box$ Loss function

Maximum Likelihood

Gradient Descent

{23}------------------------------------------------

weight: 1.0

Machine Learning Tasks

Suppose you work for an athletics company that offers an app used by 100,000 runners which tracks 15 different measures that describe their activity profile (such as, the number of minutes they run, their average speed, etc.). We want to find out what different types of runners exist, using machine learning on this data. What type of machine learning task is this?

Select one option A Model Answer

{24}------------------------------------------------

K-nn

Which of the statements is true ?

$\text{Select one option}$

$\hfill \Box$ <br>K for KNN should be tuned by looking at the 2D plot of the decision boundary.

$\stackrel{\textcolor{red}{\blacksquare}}{=}$ If a very large k is used for KNN, the classification boundary becomes simpler.

$\hfill \Box$ <br> K-NN is insensitive to feature scaling.

$\Box$ K-NN requires an explicit training step.

$A$ Model Answer

weight: 1.0

{25}------------------------------------------------

Which of the following statements is TRUE about back-propagation.

Select one option

$\Box$ Backpropagation can be used during decision tree learning to propagate information from the leaves to the root of the tree

☐ Backpropagation tells us how to update the weights to improve the 0-1 loss (i.e. classification error)

$\Box$ Backpropagation helps efficiently calculate the gradient for the logistic regression model

Backpropagation requires both a "forward pass" and a "backward pass"

weight: 1.0

$A$ Model Ans

{26}------------------------------------------------

Description

For this assignment you're going to implement a part of the k-means clustering algorithm. To be able to update the centroids of the clusters, we need to know which points of the dataset belong to which cluster.

You should complete the implementation of assign_points(), which takes the following parameters:

• points ( <code>numpy.ndarray</code> ): the points that have to be assigned to a cluster. This is an $n\times D$ matrix, where $n$ is the number of points and $D$ is the dimensionality;
• centroids ( <code>numpy.ndarray</code> ): the centroids of the clusters. This is an $m \times D$ matrix, where $$ is the number of clusters and is the dimensionality.

The function returns a list that contains for each data point the index to its closest centroid. Use the Manhattan distance for the distance computation. The Manhattan Distance is calculated as follows:

$Manhattan(x, y) = \sum_{d \in D} |x_d - y_d|$

for two $D$ -dimensional points $x$ and $y$ .

If a data point has the same minimum Manhattan distance to multiple centroids, your function should assign the data point to the centroid with the highest index among those tied centroids.

Example

Input

<pre>points := np.array([[0, 0], [3, 3]])</pre>

centroids := np.array([[-1, 0], [5, 5], [2, 0]])

Output

A Python list [0, 2]

Specification Test

Layout: IIII ▾

Solution

Assignment Information

• Assignment Link: https://weblab.tudelft.nl/cse2510/2023-2024/assignment/119927/
• Pass Criteria: Pass All Sub-assignments (How many weighted sub-assignments should one pass to pass this folder?)
• Weight: 1.0 (of total 11.0)
• Minimum to Pass: 5.8 (minimum grade to pass this assignment)
• Deadline: No deadline (A personal deadline may be set, check your submission page)

Description

Welcome to the exam for CSE22A Machine Learning!

There are three types of questions: multiple choice questions, open questions and implementation questions.

The amount of points you can earn per question vary. Wrong answers result in 0 points.

For questions with images, you can enlarge the image by clicking on it. For your convenience we advise you to have paper, pen and a calculator ready during the exam.

If you want to consult the Python (3.8) or NumPy documentation for the implementation questions, they should be accessible here:

• https://weblab.tudelft.nl/docs/python/
• https://weblab.tudelft.nl/docs/numpy/

We wish you the best of luck!

Exam Structure

Statistics and aggregrated grades were last updated 7 months ago. Grade and Statistics updates are disabled.

---

Description

Consider the following pdf’s of two classes in a 1D feature space:

Solution

The solution for this assignment has been made available, but you need to unlock it first.

In case you have difficulties with this assignment, it may be more effective to ask a teaching assistant for help first.

By unlocking this answer, you understand that your submission grade may not be representative for your performance on this topic anymore. Consider this action as asking the course staff to provide the answer. While the course staff won't get notified, the timestamp of accessing the answer will be available to the course staff.

Do you want to proceed?

Test Suite

The solution for this assignment has been made available, but you need to unlock it first.

In case you have difficulties with this assignment, it may be more effective to ask a teaching assistant for help first.

By unlocking this answer, you understand that your submission grade may not be representative for your performance on this topic anymore. Consider this action as asking the course staff to provide the answer. While the course staff won't get notified, the timestamp of accessing the answer will be available to the course staff.

Do you want to proceed?

Library

The solution for this assignment has been made available, but you need to unlock it first.

In case you have difficulties with this assignment, it may be more effective to ask a teaching assistant for help first.

By unlocking this answer, you understand that your submission grade may not be representative for your performance on this topic anymore. Consider this action as asking the course staff to provide the answer. While the course staff won't get notified, the timestamp of accessing the answer will be available to the course staff.

Do you want to proceed?

---

Gradient Descent Properties

Which of these claims is true for gradient descent?

• Given enough steps, gradient descent converges to the global minimum
• Stochastic gradient descent uses fewer computations per step, so it will always converge faster than batch (non-stochastic) gradient descent
• Larger stepsizes lead to faster convergence
• For linear regression, we typically do not need gradient descent, because there is a closed-form (analytic) solution to the empirical risk minimization problem. (Correct)

---

Description

Assume we have a 5-dimensional dataset with 2 classes, and assume we want to fit a Quadratic Discriminant Analysis on this dataset. How many free parameters do we have to estimate? (Note: the number of free parameters of a 5-dimensional mean is 5.)

Solution

Select one option

• 15
• 18
• 27
• 35
• 37
• 40
• 41
• 43
• 72
• 73

---

Fairness - duty ethics

Which of the following statements best characterizes a duty ethics moral approach?

• The correct morality is ultimately in the eye of the beholder; do whatever feels right in terms of the consequences for you.
• Always act rationally in accordance with your disposition of character as developed over a lifetime of practice. You should aim to acquire the virtues that best fit the sort of life your life.
• What matters are the outcomes of your action. Always act in such a way that the outcomes of your actions maximize the happiness for most (or minimize the suffering for most). We can quantify the moral value of the outcome of our actions and compare different action on their expected moral value.
• Always act in such a way that you treat everyone as an end in themselves, with respect for their basic rights, and not merely as a means to your own ends. What matters are the reasons for which you act, the right reason for any action being that it is done for the sake of duty alone.

---

Assignment Details

Assignment Link: https://weblab.tudelft.nl/cse2510/2023-2024/assignment/119930/ Weight: 1.0 of total 42.5 Minimum to Pass: Not set Deadline: No deadline A personal deadline may be set, check your submission page

Type of Task

Based on the characteristics of a patient, I would like to predict the glucose level (a continuous value) in someone’s blood. To train this model, I use a lot of historical samples containing characteristics of previous patients and their blood glucose levels. What type of machine learning task is this? Select one option

• Regression
• Classification
• Clustering
• Dimensionality Reduction

Discussion with Course Staff

This is a private discussion, only visible to course staff and submitter of this work.

---

Description

In this exercise, we want to calculate the information gain of different binary splits to determine the best one. Here, we consider only one discrete feature.

We have n objects which we denote using a vector of n labels and a vector which contains the feature values.

y contains the labels (0 (negative) and 1 (positive)) corresponding to each object. x contains the value of the feature of each object.

E.g. if we have the following labels: [0,0,0,0,1,1,1,1] and feature values: [0,1,2,0,1,2,0,1], then if we consider the binary split 1 vs. not-1 we end up with the following sets of labels: [0,1,1], [0,0,0,1,1].

Calculate the information gain (in terms of entropy) for each binary split.

The formula for entropy is:

$I(p) = -\sum_{i} p_i\log_2 p_i$

The formula for overall (split-)entropy is:

$I_{m} = \sum_{j=1}^{s} \frac{N_j}{N} I(p_j)$

in which s is the number of nodes after the split.

Return the information gain resulting from the each split ($IG = I(p) - I_m$).

Implement the following:

• entropy(y) a function that computes the entropy
• split_entropy(y,x,split_value) a function that computes the split-entropy, for a particular binary split
• information_gain(y,x,split_value) a function that calculates the information gain for a particular binary split

Hint 1: to call a function from the Solution class you can use: Solution.function() Hint 2: Make sure you are also able to handle the situation in which all objects in a node belong to the same class (the entropy of such a node is 0).