admin管理员组文章数量:1400211
When I use pd.read_html() to load a table into a pandas dataframe and then run print(dataframe) the output starts with "Empty DataFrame".
Also, I'm having trouble accessing the elements of the dataframe.
Here is the html:
<HTML>
<HEAD>
<TITLE>System Information 10.8.129.1</TITLE>
</HEAD>
<BODY>
<br><H4>Port Statistics</H4>
<table BORDER COLS=5 WIDTH="80%">
<tr><th>Slot/Port</th><th>Intf.</th><th>TX Frm.</th><th>TX Oct.</th><th>RX Frm.</th><th>RX Oct.</th><th>RX BC</th><th>RX MC</th><th>CRC Align.</th><th>Unders.</th><th>Overs.</th><th>Frag.</th><th>Jabbers</th><th>Total Coll.</th><th>Late Coll.</th><tr><th>1/1</th><th>1</th><th>66967821</th><th>3429650783</th><th>96815811</th><th>2328791105</th><th>39571627</th><th>3960333</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
<tr><th>1/2</th><th>2</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
<tr><th>2/1</th><th>9</th><th>17533</th><th>1899146</th><th>13646</th><th>1821221</th><th>416</th><th>34</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
<tr><th>2/2</th><th>10</th><th>12941919</th><th>1909511968</th><th>3896084</th><th>1687222693</th><th>415948</th><th>78</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
</table>
</BODY>
</HTML>
I try to read extract the dataframe by running:
import os
import pandas as pd
url='C:\\WebFiles\switch_stats.html'
dataframe = pd.read_html(url)
print(dataframe)
The output is:
[Empty DataFrame
Columns: [(Slot/Port, 1/1, 1/2, 2/1, 2/2), (Intf., 1, 2, 9, 10), (TX Frm., 66967821, 0, 17533, 12941919), (TX Oct., 3429650783, 0, 1899146, 1909511968), (RX Frm., 96815811, 0, 13646, 3896084), (RX Oct., 2328791105, 0, 1821221, 1687222693), (RX BC, 39571627, 0, 416, 415948), (RX MC, 3960333, 0, 34, 78), (CRC Align., 0, 0, 0, 0), (Unders., 0, 0, 0, 0), (Overs., 0, 0, 0, 0), (Frag., 0, 0, 0, 0), (Jabbers, 0, 0, 0, 0), (Total Coll., 0, 0, 0, 0), (Late Coll., 0, 0, 0, 0)]
Index: []]
When I use pd.read_html() to load a table into a pandas dataframe and then run print(dataframe) the output starts with "Empty DataFrame".
Also, I'm having trouble accessing the elements of the dataframe.
Here is the html:
<HTML>
<HEAD>
<TITLE>System Information 10.8.129.1</TITLE>
</HEAD>
<BODY>
<br><H4>Port Statistics</H4>
<table BORDER COLS=5 WIDTH="80%">
<tr><th>Slot/Port</th><th>Intf.</th><th>TX Frm.</th><th>TX Oct.</th><th>RX Frm.</th><th>RX Oct.</th><th>RX BC</th><th>RX MC</th><th>CRC Align.</th><th>Unders.</th><th>Overs.</th><th>Frag.</th><th>Jabbers</th><th>Total Coll.</th><th>Late Coll.</th><tr><th>1/1</th><th>1</th><th>66967821</th><th>3429650783</th><th>96815811</th><th>2328791105</th><th>39571627</th><th>3960333</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
<tr><th>1/2</th><th>2</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
<tr><th>2/1</th><th>9</th><th>17533</th><th>1899146</th><th>13646</th><th>1821221</th><th>416</th><th>34</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
<tr><th>2/2</th><th>10</th><th>12941919</th><th>1909511968</th><th>3896084</th><th>1687222693</th><th>415948</th><th>78</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th><th>0</th></tr>
</table>
</BODY>
</HTML>
I try to read extract the dataframe by running:
import os
import pandas as pd
url='C:\\WebFiles\switch_stats.html'
dataframe = pd.read_html(url)
print(dataframe)
The output is:
[Empty DataFrame
Columns: [(Slot/Port, 1/1, 1/2, 2/1, 2/2), (Intf., 1, 2, 9, 10), (TX Frm., 66967821, 0, 17533, 12941919), (TX Oct., 3429650783, 0, 1899146, 1909511968), (RX Frm., 96815811, 0, 13646, 3896084), (RX Oct., 2328791105, 0, 1821221, 1687222693), (RX BC, 39571627, 0, 416, 415948), (RX MC, 3960333, 0, 34, 78), (CRC Align., 0, 0, 0, 0), (Unders., 0, 0, 0, 0), (Overs., 0, 0, 0, 0), (Frag., 0, 0, 0, 0), (Jabbers, 0, 0, 0, 0), (Total Coll., 0, 0, 0, 0), (Late Coll., 0, 0, 0, 0)]
Index: []]
- Can you share the URL of the page whose table you are trying to scrape? – Ifeanyi Idiaye Commented Mar 24 at 17:56
1 Answer
Reset to default 1The issue is that each Table Row (<tr>) contains only Table Header elements (th), even though only the first row should logically have such elements. That is to say: your table should have Table Data Cell elements (td) after the first row.
By default, pd.read_html looks for th elements to determine what it should treat as the header. Hence, you end up with an empty dataframe with MultiIndex column labels. To avoid this, simply set header=0:
Minimal Reproducible Example
import pandas as pd
from io import StringIO
# proper `td` elements from row 1 onwards
correct_table = """
<table>
<tr><th>col1</th><th>col2</th></tr>
<tr><td>1</th><td>A</td></tr>
<tr><td>2</th><td>B</td></tr>
</table>
"""
df1 = pd.read_html(StringIO(correct_table))[0]
# only `th` elements
incorrect_table = """
<table>
<tr><th>col1</th><th>col2</th></tr>
<tr><th>1</th><th>A</th></tr>
<tr><th>2</th><th>B</th></tr>
</table>
"""
df2 = pd.read_html(StringIO(incorrect_table), header=0)[0]
df1.equals(df2)
# True
Output:
col1 col2
0 1 A
1 2 B
Above, StringIO(...) mimics your url file read. Without header=0, you get:
pd.read_html(StringIO(incorrect_table))[0].columns
MultiIndex([('col1', '1', '2'),
('col2', 'A', 'B')],
)
本文标签：
版权声明：本文标题：python - Reading network switch html table into pandas dataframe results with "Empty DataFrame" - Stack Overfl 内容由网友自发贡献，该文观点仅代表作者本人， 转载请联系作者并注明出处：http://www.betaflare.com/web/1744238000a2596648.html， 本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容，一经查实，本站将立刻删除。
发表评论