今天写爬虫的时候看bs4的官方文档,说使用lxml作为解析器比使用内置解析器速度快很多,于是写了个函数测试了一下,但不知道为什么,测试跑出来的结果刚好相反,暂时未找到原因,先做下记录后续继续研究。 图 1

测试代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from bs4 import BeautifulSoup
import time
from wswp.WebScraping import download
def time_test():
FIELDS = ('area', 'population', 'iso', 'country', 'capital', 'continent', 'tld', 'currency_code', 'currency_name', 'phone', 'postal_code_format', 'postal_code_regex', 'languages', 'neighbours')
NUM_ITERATIONS = 1000 # number of times to test each scraper
html = download('http://example.webscraping.com/view/Aland-Islands-2')
start = time.time()
for name in ['html.parser', 'lxml']:
for i in range(NUM_ITERATIONS):
result = {}
soup = BeautifulSoup(html, name)
for field in FIELDS:
result[field]=soup.find('tr', id='places_%s__row' % field).find('td', class_='w2p_fw').text
assert(result['area'] == '1,580 square kilometres')
end = time.time()
print('%s the time is: %.2f seconds' % (name, (end - start)))
time_test()