Web scraping: Missing href attribute - Need to simulate mouse clicks for web scraping?
P粉550823577
P粉550823577 2024-04-04 10:32:06
0
1
3440

For a fun web scraping project, I want to collect NHL data from ttps://www.nhl.com/stats/teams.

There is a clickable Excel export tag and I can find it usingseleniumandbs4.

Unfortunately, things end here: I can't seem to access the data since there is nohrefattribute.

I got what I wanted by simulating a mouse click usingpynput, but I want to know:

What could I have done differently? If it feels awkward.

-> Labels with export icons can be found here:

a class="styles__ExportIcon-sc-16o6kz0-0 dIDMgQ"

-> This is my code

`import pynput from pynput.mouse import Button, Controller import time from bs4 import BeautifulSoup from selenium import webdriver driver = webdriver.Chrome(executable_path = 'somepath\chromedriver.exe') URL = 'https://www.nhl.com/stats/teams' driver.get(URL) html = driver.page_source # DOM with JavaScript execution complete soup = BeautifulSoup(html) body = soup.find('body') print(body.prettify()) mouse = Controller() time.sleep(5) # Sleep for 5 seconds until page is loaded mouse.position = (1204, 669) # thats where the icon is on my screen mouse.click(Button.left, 1) # executes download`

P粉550823577
P粉550823577

Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template
About us Disclaimer Sitemap
php.cn:Public welfare online PHP training,Help PHP learners grow quickly!