Question

Data Analytics

Unsupervised learning is a type of machine learning that looks for previously undetected patterns in a data set with no pre-existing labels and with a minimum of human supervision, which is widely used in cybersecurity. Can you introduce three unsupervised methods to conduct anomaly detection? (2')

Answer 1

There are several methods which can be used to detect anomaly. Three unsupervised methods to find anomaly detection are:

Step1##

KNN- Global anomaly detection.

• It helps to detect global anomalies. Local anomalies are not included in these.
• Straightforward way of detecting anomalies.
• For every record, we have to fnd the k nearest neighbor.
• then anomaly score is calculated using these neighbors.
• either k^th- nearest neighbor dist. is used or average dist. of k- nearest neighbor.
• k^th-NN or k-NN respectively is used.
• The score value depends on dataset, normalization and no. of dimensions

step2##

LOC(Local outlier factor)

• It is a local anomaly detection method.
• There are 3 steps in it.
• find k nearest neighbor for each record x.
• local ddensity is found by computing the local reachability density by using k nearest neighbors.
• Then the LOF score is calculated by comparing the LRD and LRDs of k nearest neighbor.

step3#

LOop(Local outlier probability)

• It provide a anomaly peobability instead of score.
• Because in case of score, we are not clear that which anomaly score threashhold is a clear anomaly.
• It gives us better record of anomalies comparison.
• It assumes a half gaussiun distribution and uses the standard deviation which is called as probabilistic set distance, Used as local density estimation.
• by comparing the ratios of instnce to its neighbor result in local anoaly detection score.
• Then it is converted into probabiity by applying gaussian error function and normalization