Pandas makes it easy to scrape a table (<table> tag) on a web page. After obtaining it as a DataFrame, it is of course possible to do various processing and save it as an Excel file or csv file.

In this article you’ll learn how to extract a table from any webpage. Sometimes there are multiple tables on a webpage, so you can select the table you need.

Using the Python programming language, it is possible to “scrape” data from the web. Web Scraping with Python: Collecting More Data from the Modern Web — Book on Amazon. Jose Portilla's Data Science and ML Bootcamp — Course on Udemy. Easiest way to get started with Data Science. Covers Pandas, Matplotlib, Seaborn, Scikit-learn, and a lot of other useful topics. In this article, you’ll see how to perform a quick, efficient scraping of these elements with two main different approaches: using only the Pandas library and using the traditional scraping library BeautifulSoup. As an example, I scraped the Premier L e ague classification table. This is good because it’s a common table that can be found on.

Web Scraping With Pandas

Related course:Data Analysis with Python Pandas

Pandas web scraping

Install modules

Web Scraping With Pandas Using

Web Scraping With Python Pandas

It needs the modules lxml, html5lib, beautifulsoup4. You can install it with pip.

pands.read_html()

You can use the function read_html(url) to get webpage contents.

The table we’ll get is from Wikipedia. We get version history table from Wikipedia Python page:

This outputs:

Because there is one table on the page. If you change the url, the output will differ.
To output the table:

You can access columns like this:

Pandas Web Scraping

Once you get it with DataFrame, it’s easy to post-process. If the table has many columns, you can select the columns you want. See code below:

Then you can write it to Excel or do other things:

Python Web Scraping Modules

Related course:Data Analysis with Python Pandas