--- jupytext: formats: md:myst text_representation: extension: .md format_name: myst kernelspec: display_name: Python 3 language: python name: python3 --- (assignment07_solution)= # Assignment #7 (demo). Unsupervised learning. Solution **
[mlcourse.ai](https://mlcourse.ai) – Open Machine Learning Course**

What is the minimum number of principal components required to cover the 90% of the variance of the original (scaled) data? ```{code-cell} ipython3 # В Your code here X_pca.shape ``` **Answer options:** - 56 - 65 **[+]** - 66 - 193 **Question 2:**
What percentage of the variance is covered by the first principal component? Round to the nearest percent. **Answer options:** - 45 - 51 **[+]** - 56 - 61 ```{code-cell} ipython3 # Your code here round(float(pca.explained_variance_ratio_ * 100)) ``` Visualize data in projection on the first two principal components. ```{code-cell} ipython3 # Your code here plt.scatter(X_pca[:, 0], X_pca[:, 1], c=y, s=20, cmap="viridis"); ``` **Question 3:**
If everything worked out correctly, you will see a number of clusters, almost perfectly separated from each other. What types of activity are included in these clusters?
**Answer options:** - 1 cluster: all 6 activities - 2 clusters: (walking, walking upstairs, walking downstairs ) and (sitting, standing, laying) **[+]** - 3 clusters: (walking), (walking upstairs, walking downstairs) and (sitting, standing, laying) - 6 clusters ------------------------------ Perform clustering with the `KMeans` method, training the model on data with reduced dimensionality (by PCA). In this case, we will give a clue to look for exactly 6 clusters, but in general case we will not know how many clusters we should be looking for. Options: - `n_clusters` = n_classes (number of unique labels of the target class) - `n_init` = 100 - `random_state` = RANDOM_STATE (for reproducibility of the result) Other parameters should have default values. ```{code-cell} ipython3 # Your code here kmeans = KMeans(n_clusters=n_classes, n_init=100, random_state=RANDOM_STATE) kmeans.fit(X_pca) cluster_labels = kmeans.labels_ ``` Visualize data in projection on the first two principal components. Color the dots according to the clusters obtained. ```{code-cell} ipython3 # Your code here plt.scatter(X_pca[:, 0], X_pca[:, 1], c=cluster_labels, s=20, cmap="viridis"); ``` Look at the correspondence between the cluster marks and the original class labels and what kinds of activities the `KMeans` algorithm is confused at. ```{code-cell} ipython3 tab = pd.crosstab(y, cluster_labels, margins=True) tab.index = [ "walking", "going up the stairs", "going down the stairs", "sitting", "standing", "laying", "all", ] tab.columns = ["cluster" + str(i + 1) for i in range(6)] + ["all"] tab ``` We see that for each class (i.e., each activity) there are several clusters. Let's look at the maximum percentage of objects in a class that are assigned to a single cluster. This will be a simple metric that characterizes how easily the class is separated from others when clustering. Example: if for class "walking downstairs" (with 1406 instances belonging to it), the distribution of clusters is: - cluster 1 - 900 - cluster 3 - 500 - cluster 6 - 6, then such a share will be 900/1406 \$ \approx \$ 0.64. **Question 4:**
Which activity is separated from the rest better than others based on the simple metric described above?
**Answer options:** - walking - standing - walking downstairs - all three options are incorrect **[+]** ```{code-cell} ipython3 pd.Series( tab.iloc[:-1, :-1].max(axis=1).values / tab.iloc[:-1, -1].values, index=tab.index[:-1], ) ``` It can be seen that kMeans does not distinguish activities very well. Use the elbow method to select the optimal number of clusters. Parameters of the algorithm and the data we use are the same as before, we change only `n_clusters`. ```{code-cell} ipython3 # Your code here inertia = [] for k in tqdm_notebook(range(1, n_classes + 1)): kmeans = KMeans(n_clusters=k, n_init=100, random_state=RANDOM_STATE).fit( X_pca ) inertia.append(np.sqrt(kmeans.inertia_)) ``` ```{code-cell} ipython3 plt.plot(range(1, 7), inertia, marker="s"); ``` We calculate \$ D(k) \$, as described in [this](https://medium.com/open-machine-learning-course/open-machine-learning-course-topic-7-unsupervised-learning-pca-and-clustering-db7879568417) article in the section "Choosing the number of clusters for K-means". ```{code-cell} ipython3 d = {} for k in range(2, 6): i = k - 1 d[k] = (inertia[i] - inertia[i + 1]) / (inertia[i - 1] - inertia[i]) ``` ```{code-cell} ipython3 d ``` **Question 5:**
How many clusters can we choose according to the elbow method?
**Answer options:** - 1 - 2 **[+]** - 3 - 4 ------------------------ Let's try another clustering algorithm, described in the article – agglomerative clustering. ```{code-cell} ipython3 ag = AgglomerativeClustering(n_clusters=n_classes, linkage="ward").fit(X_pca) ``` Calculate the Adjusted Rand Index (`sklearn.metrics`) for the resulting clustering and for ` KMeans` with the parameters from the 4th question. ```{code-cell} ipython3 # Your code here print("KMeans: ARI =", metrics.adjusted_rand_score(y, cluster_labels)) print("Agglomerative CLustering: ARI =", metrics.adjusted_rand_score(y, ag.labels_)) ``` **Question 6:**
Select all the correct statements.
**Answer options:** - According to ARI, KMeans handled clustering worse than Agglomerative Clustering **[+]** - For ARI, it does not matter which tags are assigned to the cluster, only the partitioning of instances into clusters matters **[+]** - In case of random partitioning into clusters, ARI will be close to zero **[+]** **Comment:** 1. Yes, the higher ARI, the better 2. Yes, if you renumber clusters differently, ARI will not change 3. True ------------------------------- You can notice that the task is not very well solved when we try to detect several clusters (> 2). Now, let's solve the classification problem, given that the data is labeled. For classification, use the support vector machine – class `sklearn.svm.LinearSVC`. In this course, we did study this algorithm separately, but it is well-known and you can read about it, for example [here](http://cs231n.github.io/linear-classify/#svmvssoftmax). Choose the `C` hyperparameter for` LinearSVC` using `GridSearchCV`. - Train the new `StandardScaler` on the training set (with all original features), apply scaling to the test set - In `GridSearchCV`, specify `cv` = 3. ```{code-cell} ipython3 # Your code here scaler = StandardScaler() X_train_scaled = scaler.fit_transform(X_train) X_test_scaled = scaler.transform(X_test) ``` ```{code-cell} ipython3 svc = LinearSVC(random_state=RANDOM_STATE) svc_params = {"C": [0.001, 0.01, 0.1, 1, 10]} ``` ```{code-cell} ipython3 %%time # Your code here best_svc = GridSearchCV(svc, svc_params, n_jobs=4, cv=3, verbose=1) best_svc.fit(X_train_scaled, y_train); ``` ```{code-cell} ipython3 best_svc.best_params_, best_svc.best_score_ ``` **Question 7**
Which value of the hyperparameter `C` was chosen the best on the basis of cross-validation?
**Answer options:** - 0.001 - 0.01 - 0.1 **[+]** - 1 - 10 ```{code-cell} ipython3 y_predicted = best_svc.predict(X_test_scaled) ``` ```{code-cell} ipython3 tab = pd.crosstab(y_test, y_predicted, margins=True) tab.index = [ "walking", "climbing up the stairs", "going down the stairs", "sitting", "standing", "laying", "all", ] tab.columns = [ "walking", "climbing up the stairs", "going down the stairs", "sitting", "standing", "laying", "all", ] tab ``` As you can see, the classification problem is solved quite well. **Question 8:**
Which activity type is worst detected by SVM in terms of precision? Recall?
**Answer options:** - precision – going up the stairs, recall – laying - precision – laying, recall – sitting - precision – walking, recall – walking - precision – standing, recall – sitting **[+]** **Comment:** The classifier solved the problem well, but not ideally. Finally, do the same thing as in Question 7, but add PCA. - Use `X_train_scaled` and` X_test_scaled` - Train the same PCA as before, on the scaled training set, apply scaling to the test set - Choose the hyperparameter `C` via cross-validation on the training set with PCA-transformation. You will notice how much faster it works now. **Question 9:**
What is the difference between the best quality (accuracy) for cross-validation in the case of all 561 initial characteristics and in the second case, when the principal component method was applied? Round to the nearest percent.
**Answer options:** - quality is the same - 2% - 4% **[+]** - 10% - 20% ```{code-cell} ipython3 # Your code here scaler = StandardScaler() X_train_scaled = scaler.fit_transform(X_train) X_test_scaled = scaler.transform(X_test) pca = PCA(n_components=0.9, random_state=RANDOM_STATE) X_train_pca = pca.fit_transform(X_train_scaled) X_test_pca = pca.transform(X_test_scaled) ``` ```{code-cell} ipython3 svc = LinearSVC(random_state=RANDOM_STATE) svc_params = {"C": [0.001, 0.01, 0.1, 1, 10]} ``` ```{code-cell} ipython3 %%time best_svc_pca = GridSearchCV(svc, svc_params, n_jobs=4, cv=3, verbose=1) best_svc_pca.fit(X_train_pca, y_train); ``` ```{code-cell} ipython3 best_svc_pca.best_params_, best_svc_pca.best_score_ ``` The result with PCA is worse by 4%, comparing accuracy on cross-validation. ```{code-cell} ipython3 round(100 * (best_svc_pca.best_score_ - best_svc.best_score_)) ``` **Question 10:**
Select all the correct statements: **Answer options:** - Principal component analysis in this case allowed to reduce the model training time, while the quality (mean cross-validation accuracy) suffered greatly, by more than 10% - PCA can be used to visualize data, but there are better methods for this task, for example, tSNE. However, PCA has lower computational complexity **[+]** - PCA builds linear combinations of initial features, and in some applications they might be poorly interpreted by humans **[+]** **Comment:** 1. The first statement is true, principal component analysis in this case allowed to significantly reduce the training time of the model, but the quality suffered not so much – by only 4% 2. For multidimensional data visualization it is better to use manifold learning methods, in particular, tSNE. At the same time, metrics assessing the quality of visualization have not really been invented yet, but tSNE is widely used precisely because in some cases it builds "good" pictures showing the data structure, as in the case of MNIST 3. Linear combinations of features, that PCA builds, are often poorly interpreted by humans, for example, 0.574 \* salary + 0.234 \* num_children