import requests from bs4 import BeautifulSoup url = "https://website.com" # send a request to the website response = requests.get(url) # parse the response as HTML soup = BeautifulSoup(response.text, "html.parser") # find all of the content in the body of the HTML document content = soup.find("body") # print the content print(content.prettify())
This code sends a request to the website using the
requests library, then parses the response as HTML using the
BeautifulSoup library. The
find method is used to search the HTML document for a specific tag (in this case, the
body tag), and the
prettify method is used to format the content for easy reading.
Keep in mind that web scraping can have legal consequences, so it is important to make sure that you have the website owner’s permission before scraping their site. Additionally, some websites may block scraper IPs, so it is always a good idea to use a proxy service to hide your IP address.