Question

In: Computer Science

a. What is the main cue for splitting English documents into sentences? When and how might...

a. What is the main cue for splitting English documents into sentences? When and how might it go wrong? ( python coding required.)

b. What is the main cue in English for splitting sentences into tokens? When and how might it go wrong? (python coding required.)

Solutions

Expert Solution

a.) Below is the python program to split the text file into sentence but this may raise a problem is there is no \n new line character found than the file will be shown as it is we can also use "." but it may also arise the same problem so to avoid this we must have to read and print the data of the file.

# suppose we have a file called filenew.txt, lets see:- 
 
f=open("filenew.txt").read() # reads file 

sentence=f.split("\n") # split into sentences or we can use sentence=f.split(".")

print(sentence) # print each sentence that will be splited after the new line character \n

b.) This may arrise a situation if there will is no space in the file or the file contains only commas to seprate the words so we can print out the file data to check how we can split it into the tokens.

# suppose we have a file called filenew.txt, lets see:- 
 
f=open("filenew.txt").read() # reads file 

words=f.split(" ") # split into words(tokens) and characters 

print(words) 

Related Solutions

In one or two sentences describe the corrosion failure mode. What is one visual cue to...
In one or two sentences describe the corrosion failure mode. What is one visual cue to indicate corrosion? Does corrosion always necessitate part replacement?
Determine what clinical problem or issue an organization might be facing to implement cue-based feeding on...
Determine what clinical problem or issue an organization might be facing to implement cue-based feeding on infants. Discuss two implications for nursing.
write a few English sentences describing how you would change the Linked-Chain implementation of the List...
write a few English sentences describing how you would change the Linked-Chain implementation of the List ADT to support removal of a node from the middle retaining order. Write PSEUDOCODE to insert a node at position 2 in a doubly-linked list (assume position follows classic indexing from 0 to item_count - 1)
In a few sentences, describe a time when you might be able to utilize ESXTOP. Describe...
In a few sentences, describe a time when you might be able to utilize ESXTOP. Describe how you would use it?
Question 4 What type of organisational standards might exist for creating business documents? Give at least...
Question 4 What type of organisational standards might exist for creating business documents? Give at least five examples.
Write at least 5 sentences on the following question: What are the main challenges in today's...
Write at least 5 sentences on the following question: What are the main challenges in today's society that limit nuclear families from thriving?
-How does strength correlate with other properties of hardened concrete? -What is the relationship of splitting...
-How does strength correlate with other properties of hardened concrete? -What is the relationship of splitting tensile strength of concrete with its compressive strength? -What is the significance of splitting tensile test experiment? -Why plywood strips are used in splitting tensile test?
What negative impacts might there be from the hosting of the Olympic Games? Full sentences please....
What negative impacts might there be from the hosting of the Olympic Games? Full sentences please. I need this answered ASAP. Thanks in advance.
What are the main ways of organizing a business? What kind of business organization might be...
What are the main ways of organizing a business? What kind of business organization might be suitable for a plumber? Explain. Why might someone choose to organize as an LLC instead of a sole proprietorship? In legal terms, what is the relation between a corporation and those individuals who found the corporation? In what ways does the structure of a corporation protect its owners from absorbing ethical responsibility for the company’s actions? How can corporations raise money?
What does it mean when a documents or information is"privileged"? A. The tax professional who prepared...
What does it mean when a documents or information is"privileged"? A. The tax professional who prepared the document or information is the only person who may authorize its dislosure B. the documents or information is not relevant to the determination of the taxpayer 's tax liability. C. The IRS and other authorities may obtain and review the document or information ,but they may not use it against the client. D. the document or information is protected from discovery and use...
ADVERTISEMENT
ADVERTISEMENT
ADVERTISEMENT