Question

Question 1 :

1. A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience

For a voice recognition learning problem determine the possible:

1. task T
2. performance measure P
3. and training experience E

Question 2 :

2. Explain what is the curse of dimensionality? What is the relation between machine learning problems and the curse of dimensionality?

Question 3 :

3. The following training dataset is “reading email dataset”.

This dataset has four features as follows: author, thread, length, and where to read the mail. According to the features the algorithm has to predict the user’s action whether to read or skip the mail.

Use Naïve Bayes classifier to predict the user’s action (skips or reads) when the author of the mail is known, the thread of the mail is follow up, the length of the mail is short, and where to read the email is home.

Author	Thread	Length	Where to read	User’s Action
Known	new	long	home	Skips
unknown	new	short	work	Reads
unknown	Follow up	long	work	Skips
Known	Follow up	Long	Home	Skips
Known	New	Short	Home	Reads
Known	Follow up	Long	Work	Skips
Unknown	New	short	work	skips
Unknown	New	short	Work	reads
Known	Follow up	Long	Home	Skips
known	New	Long	Work	skips
unknown	Follow up	short	home	Skips
Known	new	Long	work	Skips
Known	Follow up	Short	Home	Reads
Known	New	Short	Work	Reads
known	New	short	Home	Reads
Known	Follow up	short	Work	Reads
Known	New	Short	home	Reads
unknown	new	short	work	Reads

1. (35 points) Write a Python code to implement a naïve Bayesian classifier to predict the user’s action (skips or reads) when the author of the mail is known, the thread of the mail is follow up, the length of the mail is short, and where to read the email is home. (Do not use Scikit-Learn)
2. (35 points) Use Scikit-Learn to predict the user’s action (skips or reads) when the author of the mail is known, the thread of the mail is follow up, the length of the mail is short, and where to read the email is home.

Hint in authors feature you can use 0, 1 instead of unknown and known. In thread feature you can use 0, 1 instead of follow up and new. In length feature you can use 0, 1 instead of short and long. In where to read feature you can use 0, 1 instead of home, work. In the target you can use 0 instead of skips and 1 instead of reads.

Answer 1

1) For a voice recognition learning problem,

• Task: Convert input speech to to text
• Performance measures: Percentage of total number of speech commands converted to text correctly (with respect to the total number of speech commands tested)
• Experience: The text representation of given speech data (mappings of speech signals with the correct textual representation of the spoken command).

2) The curse of dimensionality can be summarised as: "As the dimensionality of the feature space increases, the number of configurations can grow exponentially, and hence, the number of configurations covered by an observation also decreases." We can explain it with help of the Hughes' phenomenon, which states that as the number of features for a given model increases, the classifier's performance keeps on increasing up to a particular number of features. After that, beyond the limit, training more will give degrade the performance of the classifier.

The Euclidean distance between two n-dimensional vectors with Cartesian coordinates p = (p1, p2, …, pn) and q = (q1, q2, …, qn) is computed using the distance formula:

• Notice that as the value of i increases, so do our dimensions required to calculate the distance.
• Consider this example:
  • When we have 10 observations with 1-dimension, we only have 10 features required to calculate distances.
  • However, when we increase the dimensionality to 2, we have to take care of 10x10 = 100 features to calculate the Euclidean distance.
  • When we up the number of dimensions to 3, we have to take care of 10x10x10 distance terms for calculating the Euclidean distance = 1000 features.

Thus, when we have more dimensions, the amount of calculations we need to perform goes up exponentially!

We can draw the following conclusions based on the above:

1. As distance between the various observations (one row of X, the dataset) increases, supervised machine learning becomes increasingly computationally intensive - predictions from the incoming data is likely to be based on other similar data the classifier has been previously trained on.
2. The number of unique sets of obersvations grows exponentially with the number of features.
3. Hence, detecting patterns within the data with the same model becomes exponentially harder, and the model's ability to generalise is hampered.
4. The variance of the data increases as the dimensionality increases, hence the model gets more exposed to noise which may mask useful features. Thus, again hampering generalization capability of the model.

(Please drop question 3 as a separate question).

If you liked this answer, consider giving it a thumbs up. Thank you so much!