1. Create a Python module that reads Linux_syslog_ip_parser.py that reads syslog file and parses the IPV4...

1. Create a Python module that reads Linux_syslog_ip_parser.py that reads syslog file and parses the IPV4 addresses and stores the result into a csv file. Your module should contain the following functions:

a. linux_logfile_reader(filename) function that reads the syslog file and returns all IPV4 addresses

b. count_ipfrequency(ip_list) function that accepts the list of IPV4 addresses retuned by linux_logfile_reader function and returns a dictionary list with ip:frequency pair.

c. Writeto_csv_file(ip_frequency) function that accepts a dictionary list containing ip:frequency pair returned from count_ipfrequeny function and stores the IpV4 addresses with its frequency value into a ‘ipv4.csv’ file.

i. Which internal IP address is the most frequency found in the ‘syslog’ file ( hint: internal ip addresses start with either 10, 172, or 192)

ii. Which external Ip address is the most frequently found in the ‘syslog’ file

Example From File.

83.149.9.216 - - [17/May/2015:10:05:03 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:43 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard3.png HTTP/1.1" 200 171717 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:47 +0000] "GET /presentations/logstash-monitorama-2013/plugin/highlight/highlight.js HTTP/1.1" 200 26185 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:12 +0000] "GET /presentations/logstash-monitorama-2013/plugin/zoom-js/zoom.js HTTP/1.1" 200 7697 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:07 +0000] "GET /presentations/logstash-monitorama-2013/plugin/notes/notes.js HTTP/1.1" 200 2892 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:34 +0000] "GET /presentations/logstash-monitorama-2013/images/sad-medic.png HTTP/1.1" 200 430406 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:57 +0000] "GET /presentations/logstash-monitorama-2013/css/fonts/Roboto-Bold.ttf HTTP/1.1" 200 38720 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:50 +0000] "GET /presentations/logstash-monitorama-2013/css/fonts/Roboto-Regular.ttf HTTP/1.1" 200 41820 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:24 +0000] "GET /presentations/logstash-monitorama-2013/images/frontend-response-codes.png HTTP/1.1" 200 52878 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:50 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard.png HTTP/1.1" 200 321631 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:46 +0000] "GET /presentations/logstash-monitorama-2013/images/Dreamhost_logo.svg HTTP/1.1" 200 2126 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:11 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-dashboard2.png HTTP/1.1" 200 394967 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:19 +0000] "GET /presentations/logstash-monitorama-2013/images/apache-icon.gif HTTP/1.1" 200 8095 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:33 +0000] "GET /presentations/logstash-monitorama-2013/images/nagios-sms5.png HTTP/1.1" 200 78075 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:00 +0000] "GET /presentations/logstash-monitorama-2013/images/redis.png HTTP/1.1" 200 25230 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:25 +0000] "GET /presentations/logstash-monitorama-2013/images/elasticsearch.png HTTP/1.1" 200 8026 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:59 +0000] "GET /presentations/logstash-monitorama-2013/images/logstashbook.png HTTP/1.1" 200 54662 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:30 +0000] "GET /presentations/logstash-monitorama-2013/images/github-contributions.png HTTP/1.1" 200 34245 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:53 +0000] "GET /presentations/logstash-monitorama-2013/css/print/paper.css HTTP/1.1" 200 4254 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:24 +0000] "GET /presentations/logstash-monitorama-2013/images/1983_delorean_dmc-12-pic-38289.jpeg HTTP/1.1" 200 220562 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:54 +0000] "GET /presentations/logstash-monitorama-2013/images/simple-inputs-filters-outputs.jpg HTTP/1.1" 200 1168622 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:33 +0000] "GET /presentations/logstash-monitorama-2013/images/tiered-outputs-to-inputs.jpg HTTP/1.1" 200 1079983 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
83.149.9.216 - - [17/May/2015:10:05:56 +0000] "GET /favicon.ico HTTP/1.1" 200 3638 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36"
24.236.252.67 - - [17/May/2015:10:05:40 +0000] "GET /favicon.ico HTTP/1.1" 200 3638 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0"

Solutions

Expert Solution

The following is the complete code:-

import re

def linux_logfile_reader(path):
    ip_list = []
    with open(path, 'r') as f:
        for line in f.readlines():
            ip_list.append(
                re.findall("\d+\.\d+\.\d+\.\d+", line)[0]
            )
    f.close()
    return ip_list

def count_ipfrequency(ip_list):
    count = {} 
    for i in ip_list: 
        count[i] = count.get(i, 0) + 1
    return count

def Writeto_csv_file(ip_frequency_pair):
    with open("ipv4.csv", "w") as f:
        for key in ip_frequency_pair.keys():
            f.write("%s,%s\n"%(key, ip_frequency_pair[key]))
            
    f.close()

ip_list = linux_logfile_reader('sys_log.txt')
print(ip_list)

ip_frequency_pair = count_ipfrequency(ip_list)
print(ip_frequency_pair)

Writeto_csv_file(ip_frequency_pair)

1) linux_logfile_reader

This function is dependent on the re package. We open the path of the text file that has been passed into the function. We read the file line by line and append all the values that match the pattern digits.digits.digits.digits. Once we iterate through each line in the syslogs we return the list with the IP strings.

2) count_ipfrequency

This function takes the IP list we created in the last function and then converts it into a dictionary with IP addresses as the key and the occurrence of that IP as the value.

3) Writeto_csv_file

This function opens the file ipv4.csv but using the write method mentioned by w. We iterate through the dictionary and store the key and its count in the csv.

In order to answer the i and ii questions, you can do that by running the code on the actual data and then analysing the csv.

I hope this answered your queries. If any further doubts do let me know in the comments I will clear them out by updating the answer. Regards.

venereology answered 3 months ago

Next > < Previous

Related Solutions

Create a Python program that: Reads the content of a file (Vehlist.txt) The file contains matching...

Create a Python program that: Reads the content of a file (Vehlist.txt) The file contains matching pairs of vehicle models and their respective makes Separate out the individual make and model on each line of the file Add the vehicle make to one list, and the vehicle model to another list; such that they are in the same relative position in each list Prompt the user to enter a vehicle model Search the list containing the vehicle models for a...

Create a program that parses a CSV file of product data and prints the items with...

Create a program that parses a CSV file of product data and prints the items with a price that is less than or equal to that input by the user. • Your program should take two arguments: an input file to process and a price limit. • Print only the names of each item to stdout that have a price less than or equal to the given limit. • If the given file does not exist or cannot be opened,...

This is a python file Reads information from a text file into a list of sublists....

This is a python file Reads information from a text file into a list of sublists. Be sure to ask the user to enter the file name and end the program if the file doesn’t exist. Text file format will be as shown, where each item is separated by a comma and a space: ID, firstName, lastName, birthDate, hireDate, salary Store the information into a list of sublists called empRoster. EmpRoster will be a list of sublists, where each sublist...

1) Write a python program that opens a file, reads all of the lines into a list...

1) Write a python program that opens a file, reads all of the lines into a list of strings, and closes the file. Use the Readlines() method. Test your programing using the names.txt file provided. 2) Convert the program into a function called loadFile, that receives the file name as a parameter and returns a list of strings. 3) Write a main routine that calls loadFIle three times to load the three data files given into three lists. Then choose a...

Module 1 Program Write a complete Java program in a file called Module1Program.java that reads all...

Module 1 Program Write a complete Java program in a file called Module1Program.java that reads all the lyrics from a file named lyrics.txt that is to be found in the same directory as the running program. The program should read the lyrics for each line and treat each word as a token. If the line contains a double (an integer is also treated as a double) it should use the first double it finds in line as the timestamp for...

Problem: Write a Python module (a text file containing valid Python code) named p5.py. This file...

Problem: Write a Python module (a text file containing valid Python code) named p5.py. This file must satisfy the following. Define a function named rinsert. This function will accept two arguments, the first a list of items to be sorted and the second an integer value in the range 0 to the length of the list, minus 1. This function shall insert the element corresponding to the second parameter into the presumably sorted list from position 0 to one less...

Implement in Python a script that does the following: 1) reads input from a supposed file...

Implement in Python a script that does the following: 1) reads input from a supposed file called firstnames_2.txt. 2) processes the input and writes and saves the output to a file. NOTE: Please make sure that the names are written in the outfile with one name on each line no comma ( , ) after the name in the output

Using Python 3 Write a program that reads the 4letterwords.txt file and outputs a file called...

Using Python 3 Write a program that reads the 4letterwords.txt file and outputs a file called 4letterwords.out that has every word found in 4letterwords.txt but each word on one line. Any leading and trailing blank spaces must be removed in the output file.

Question: Write a program in python that reads in the file climate_data_2017_numeric.csv and prompts the user...

Question: Write a program in python that reads in the file climate_data_2017_numeric.csv and prompts the user to enter the name of a field (other than Date), and then outputs the highest and lowest values recorded in that field for the month of August. The file climate_data_2017_numeric.csv contains the following fields: Date Minimum temperature (C) Maximum temperature (C) Rainfall (mm) Speed of maximum wind gust (km/h) 9am Temperature (C) 9am relative humidity (%) 3pm Temperature (C) 3pm relative humidity (%) Expected...

This is C++ Create a program that reads an HTML file and converts it to plain...

This is C++ Create a program that reads an HTML file and converts it to plain text. Console: HTML Converter Grocery List * Eggs * Milk * Butter Specifications: The HTML file named groceries.html contains these HTML tags: <h1>Grocery List</h1> <ul> <li>Eggs</li> <li>Milk</li> <li>Butter</li> </ul> When the program starts, it should read the contents of the file, remove the HTML tags, remove any spaces to the left of the tags, add asterisks (*) before the list items, and display the...

Subjects