Question

The objective of this homework assignment is to demonstrate proficiency with reading files, and using string methods to slice strings, working with dictionaries and using-step wise development to complete your program.

Python is an excellent tool for reading text files and parsing (i.e. filtering) the data. Your assignment is to write a Python program in four steps that reads a Linux authentication log file, identifies the user names used in failed password attempts and counts the times each user name is used.

The purpose of step-wise development is to take an otherwise complex problem and break it down into separate manageable steps. Too often students and novice developers struggle because they try and attack a problem all at once. This leads to multiple problems in the code that prove too difficult to overcome. By breaking the problem up into pieces, the overall solution is much easier to obtain.

You need to complete this assignment in four separate steps and will be submitting four separate programs as described below.

Note that the first three steps use file auth.log.1b, because the full log file, auth.log.1, can take a long time to run in IDLE, because of the print statements. But, auth.log.1b does not have all of the data you'll need for Step 4. As a result, you need to use auth.log.1 in Step 4.

Step 1 (50 pts):

The objective of this program is simply to open a file, read each line and display it. Your program should:

• Open auth.log.1b. (See note below).
• Read each line.
• Display each line.
• Note that your program could take over 15 seconds to complete.

Save this program as step_1.py.

Notes:

Download auth.log.1b by right clicking on it and doing a ‘Save as’ to the same directory in which you have your program. Although auth.log.1b is a plain text file, the ‘.1b’ file extension is non-standard and will not be recognized as a particular file type. You can view it, if you like, in any text editor such as NotePad, TextEdit or IDLE, but you may have to select ‘All file types’ to open it.

Potential gotchas:

If your program produces a FileNotFound error, it is most likely because auth.log.1b is not in the same folder as your program or the name got changed somehow. There may be a hidden file extension, like ‘.txt’.

Check the 'garbage in, garbage out' rule. Windows hides known file type extensions by default. What you're seeing might not be what you're getting, in terms of the actual exact file names. To make absolutely sure that the name of auth.log.1b is correct on your system do the following:

• go into the Control Panel->Appearance and Personalization->File Explorer Options -> select the View tab
• uncheck the box for 'Hide Extensions for known file types'.
• Click Apply
• Click Save
• The real name of auth.log.1b should be clearly visible in your folder. The '.py' extension on your filenames should also become visible.
• See these screenshots:
  • Unhiding File Extensions and Visible File Extensions

Step 2 (15 pts):

The objective of this step is to recognize which lines indicate an attack. Start this step by making a copy of your Step 1 program.

Your program should do the following:

• As each line is read, check to see if “Failed password” is in the line by using the ‘in’ operator.
• If “Failed password” is in the line, display the line.
• Otherwise do not.

Save this program as step_2.py.

Potential gotchas

• If your program is not finding any lines, make sure you have the case right. “Failed password” and “failed password” are different.
• Also, be mindful of spelling.

Step 3 (25 pts):

This step is perhaps the most challenging. The objective here is to slice the user name out of the lines that include “Failed password”. Begin by making a copy of step_2.py.

Strategy:

You will slice the user name, which varies in length and start position within the line, by finding a pattern that always appears immediately before it and another pattern that always appears immediately after it. Use these offsets to compute the starting and ending values of the user name slice.

Your program should do the following:

• For just the lines that include “Failed password”:
  • Use the string find() method to get the offset of “invalid user ” from the start of the line.
  • Determine the starting point of the user name slice by adding the length of “invalid user “ to the offset provided by find(). This will be the index of the first character in the user name.
  • Use the string find() method to get the offset of “ from “. This will serve as the end point of the user name slice.
  • Slice the user name from the line using the starting and ending points and store the result in a variable.
  • Instead of displaying the whole line, just display the slice. You should see ‘root’ displayed quite a bit.

Save your program as step_3.py.

Potential gotchas:

• find() returns -1 if it doesn’t find a pattern in the search. If you are unsuccessful in creating a slice, most likely you have a problem with find(). If this is the case, then try this:
• Use a print statement to display the full line and another print statement to show the result from each find().
• If ‘-1’ is displayed for either of find() results, double check the spelling of the pattern you are using for find().
• If you are getting a positive number from find, then count the number of characters from the beginning of the line to see what find() is finding. “invalid user ” should start somewhere around offset 55 and “ from “ somewhere around offset 72.
• If you are getting reasonable values from find, but still are not able to slice the user name, then you probably are not setting up the slice properly.

Step 4 (10 pts):

The objective of this step is to use a dictionary to count the number of times each user name appears in attack attempts. Begin by making a copy of step_3.py.

Strategy:

Up until this point, you've been working with an abbreviated log file, auth.log.1b, which only has data for attacks from user root. For this step, you'll need the full log file: auth.log.1. Download it just as you did with auth.log.1b and change your code to open auth.log.1 instead of auth.log.1b. Note that your program could take considerably longer to run using auth.log.1.

Use a dictionary to count the number of times each user name appears in the file. The items in the dictionary will consist of a user name as the key and a count as the value.

Your program should do the following:

• Create an empty dictionary at the top of the program right after the header comments.
• After you have sliced a name, use the in operator to see if it is already in the dictionary
• If it is not in the dictionary, add it. Do so by using the user name as the key and set the value to 1.
• If it is in the dictionary, use the user name as a key to get the current value. Add 1 to the current value and store it back in the dictionary.
• After all of the lines of the file have been read, use a for loop to display each key and value. The key will be a user name and the value will be the number of times the user name appears in the dictionary.

Save your file as hwk6.py

Potential gotchas:

• If you are not seeing lots of names and values displayed, check to see what is in the dictionary by simply printing it by itself.
• If the dictionary is empty or there is just one item, then look to see where you are creating the dictionary.
• If you are doing so inside the loop, then you are basically erasing it every time the loop runs.
• Otherwise, check for variable name spelling. You could be stepping on your dictionary by using the same name for a different variable.

Submit all four of your program files for full credit. Be sure to include header comments in each file!

Answer 1

//Only if you encounter error like file not found then check for extension of file and add the name with extension in open() Good Luck!

step_1.py

fileread = open('auth.log.1b', 'r')      #reading from file
line_in_file = fileread.readlines()      
for line in line_in_file: 
    print(line.strip())                  #display file

step_2.py

fileread = open('auth.log.1b', 'r')      #reading from file
line_in_file = fileread.readlines()      
for line in line_in_file:
    if("Failed password" in line.strip()): 
        print(line.strip())                  #display file

step_3.py

fileread = open('auth.log.1b', 'r')      #reading from file
line_in_file = fileread.readlines()      
for line in line_in_file:
    if("Failed password" in line.strip()): 
        start=line.find("invalid user")+len("invalid user");
        end=line.find("from")
        print(line[start:end])

hwk6.py

dictionary={}
fileread = open('auth.log.1', 'r')      #reading from file
line_in_file = fileread.readlines()      
for line in line_in_file:
    if("Failed password" in line.strip()): 
        start=line.find("invalid user")+len("invalid user");    
        #start offset  --if it takes white space at start then change line to 
        #start=line.find("invalid user")+len("invalid user")+1 
        end=line.find("from")                       #end offset
        username=line[start:end]
        if(dictionary.get(username)):               #check if username is in dict
            value=dictionary.get(username)
            dictionary[username]=value+1;
        else:
            dictionary[username]=1;
for dictkey,dictvalue in dictionary.items():
    print("{} {}".format(dictkey,dictvalue));       #print dict key value