Question

UNSURE HOW TO PARSE: I am trying to parse a website with a CSV :

import csv
import requests as r
from bs4 import BeautifulSoup


urltoget = 'a.ttu.edu/2019c/isqs6339/http://drd.bhw1/index.php'
filepath = 'C:\\Users\\Zuliat\\Desktop\\test123.csv'


res = r.get(urltoget)
res.content

if res.status_code == 200:
print (' request is good')
else:
print (' bad request, retrieved code' + str(res.status_code))
  
print (res.content)

soup = BeautifulSoup(res.content,'lxml')
print(soup.title)

They are responding with a message Line 1

Answer 1

You should use a HTML parser when parsing a HTML page.. I am not able to open the page link which you have given.. However, I have written a sample for some of my student earlier, i hope it helps you..

import requests
from bs4 import BeautifulSoup

# Get the HTML contents of URL using requests module
page = requests.get('https://www.tdcj.texas.gov/death_row/dr_executions_by_year.html')

# Create a BS4 Object using the html content
# mention parser as HTML parser.
soup = BeautifulSoup(page.text, 'html.parser')

# Now Soup object has a select method, in which you can mention
# the CSS selector, with which it selects the list of elements
# like, Below i am finding elements with class "overflow", then 
# inside that, i am finding all the table rows
rows = soup.select('.overflow table tr')[1:-1]

# We are trying to find the number of executions on each year from
# the webpage, So we create a dictionary, where we will store the 
# number of executions on each year

executions = {}

for link in rows:
        # on each table rows, th element contains the year
        # while second last td element contains the number of executions
        x = link.find('th')
        y = link.findAll('td')[-2]
        executions[int(x.text)] = int(y.text)

for y in sorted(executions):
        print(y, ':', executions[y])

**************************************************

Thanks for your question. We try our best to help you with detailed answers, But in any case, if you need any modification or have a query/issue with respect to above answer, Please ask that in the comment section. We will surely try to address your query ASAP and resolve the issue.

Please consider providing a thumbs up to this question if it helps you. by Doing that, You will help other students, who are facing similar issue.