Question

17. Write a function in Python which removes a set of common words from a long piece of text. Be sure to strip out any punctuation. The common words to be removed are:

​​a, an, as, at, by, for, in, is, it, of, that, this, to, was, will, the

These are typical words which are considered to have low semantic value.

Process each paragraph provided below individually. Your end result for each paragraph should be a string or list containing the processed paragraph with the common words and punctuation removed. It is not required to put the text samples into a file (you can simply copy and paste into a string variable inside your Python script).

For the text samples to process, use the following (taken from the official Python tutorial):

If you do much work on computers, eventually you find that there’s some task you’d like to automate.

For example, you may wish to perform a search-and-replace over a large number of text files, or rename and rearrange a bunch of photo files in a complicated way. Perhaps you’d like to write a small custom database, or a specialized GUI application, or a simple game.

If you’re a professional software developer, you may have to work with several C/C++/Java libraries but find the usual write/compile/test/re-compile cycle is too slow. Perhaps you’re writing a test suite for such a library and find writing the testing code a tedious task. Or maybe you’ve written a program that could use an extension language, and you don’t want to design and implement a whole new language for your application.

You could write a Unix shell script or Windows batch files for some of these tasks, but shell scripts are best at moving around files and changing text data, not well-suited for GUI applications or games. You could write a C/C++/Java program, but it can take a lot of development time to get even a first-draft program. Python is simpler to use, available on Windows, Mac OS X, and Unix operating systems, and will help you get the job done more quickly.

Python is simple to use, but it is a real programming language, offering much more structure and support for large programs than shell scripts or batch files can offer. On the other hand, Python also offers much more error checking than C, and, being a very-high-level language, it has high-level data types built in, such as flexible arrays and dictionaries. Because of its more general data types Python is applicable to a much larger problem domain than Awk or even Perl, yet many things are at least as easy in Python as in those languages.

Python allows you to split your program into modules that can be reused in other Python programs. It comes with a large collection of standard modules that you can use as the basis of your programs — or as examples to start learning to program in Python. Some of these modules provide things like file I/O, system calls, sockets, and even interfaces to graphical user interface toolkits like Tk.

Python is an interpreted language, which can save you considerable time during program development because no compilation and linking is necessary. The interpreter can be used interactively, which makes it easy to experiment with features of the language, to write throw-away programs, or to test functions during bottom-up program development. It is also a handy desk calculator.

Answer 1

Below is a screen shot of the python program to check indentation. Comments are given on every line explaining the code.

Below is the output of the program:

Below is the code to copy:
#CODE STARTS HERE----------------

def remove(text_string):
   #list of punctuations to remove
   punc = '''!()-[]{};:'"\,<>./?@#$%^&*_~'''
   punc_removed = "" #Used to store the string without punctuations
   for letter in text_string: #Loop every character in the string
      if letter not in punc: #If characters is not in punc, add it to new string
         punc_removed += letter

   #List of words to be removed
   stop_words = ["a", "an", "as", "at", "by", "for", "in", "is",
              "it", "of", "that", 'this', "to", "was", "will", "the"]
   word_removed = "" #Used to store the string after removing words
   for word in punc_removed.split(): #Loop through every word
      if word not in stop_words:
         word_removed += " "+word #Add words separated by space
   print(word_removed) #Print result

#Sample text
text = "If you do much work on computers, eventually you find that there’s some task you’d like to automate." \
      "For example, you may wish to perform a search-and-replace over a large number of text files, or rename " \
      "and rearrange a bunch of photo files in a complicated way. Perhaps you’d like to write a small custom " \
      "database, or a specialized GUI application, or a simple game." \
      "If you’re a professional software developer, you may have to work with several C/C++/Java libraries " \
      "but find the usual write/compile/test/re-compile cycle is too slow. Perhaps you’re writing a test suite " \
      "for such a library and find writing the testing code a tedious task. Or maybe you’ve written a program that " \
      "could use an extension language, and you don’t want to design and implement a whole new language for your application." \
      "You could write a Unix shell script or Windows batch files for some of these tasks, but shell scripts are " \
      "best at moving around files and changing text data, not well-suited for GUI applications or games. You " \
      "could write a C/C++/Java program, but it can take a lot of development time to get even a first-draft " \
      "program. Python is simpler to use, available on Windows, Mac OS X, and Unix operating systems, " \
      "and will help you get the job done more quickly.Python is simple to use, but it is a real programming language," \
      " offering much more structure and support for large programs than shell scripts or batch files can offer. " \
      "On the other hand, Python also offers much more error checking than C, and, being a very-high-level language," \
      " it has high-level data types built in, such as flexible arrays and dictionaries. Because of its more general" \
      " data types Python is applicable to a much larger problem domain than Awk or even Perl, yet many things are at" \
      " least as easy in Python as in those languages.Python allows you to split your program into modules that can be " \
      "reused in other Python programs. It comes with a large collection of standard modules that you can use as the" \
      " basis of your programs — or as examples to start learning to program in Python. Some of these modules provide " \
      "things like file I/O, system calls, sockets, and even interfaces to graphical user interface toolkits like Tk." \
      "Python is an interpreted language, which can save you considerable time during program development because no" \
      " compilation and linking is necessary. The interpreter can be used interactively, which makes it easy to experiment" \
      " with features of the language, to write throw-away programs, or to test functions during bottom-up program development." \
      " It is also a handy desk calculator."
remove(text) #function call
#CODE ENDS HERE------------------