Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python Scraping Error
#1
I wanted to get .text of <a>, but it simply doesn't work.

Here is my code:
from bs4 import BeautifulSoup import requests generatedLink = "MyLink" page = requests.get(generatedLink) contents = page.text soup = BeautifulSoup(contents, "html.parser") name = soup.find('a',class_=["yt-simple-endpoint", "style-scope", "ytd-video-renderer"]) print(name)
And it returns "None"
Output:
<a id="video-title" class="yt-simple-endpoint style-scope ytd-video-renderer" aria-label="TURNIR 1 VS 1 U LOLU FINALEE!! od korisnika KaLuu Vrijeme streaminga: prije 3 dana 3 sata i 49 minuta 644 pregleda" href="/watch?v=5N4X4hjkzOw" title="TURNIR 1 VS 1 U LOLU FINALEE!!"> TURNIR 1 VS 1 U LOLU FINALEE!! </a>
Ik that something is rly wrong here, but if you can, help me please. Thank you!
Reply
#2
Look at what soup source return.
If you see <a> tag in browser dos not always mean that you get that in source Requests get.
This is because of JavaScript,what you see in browser inspect is what has been rendered in DOM.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  I am scraping a web page but got an Error Sarmad54 3 3,117 Mar-02-2023, 08:20 PM
Last Post: Sarmad54
  error in code web scraping alexisbrunaux 5 5,901 Aug-19-2020, 02:31 AM
Last Post: alexisbrunaux
  error zomato scraping data syxzetenz 3 4,980 Jun-23-2020, 08:53 PM
Last Post: Gribouillis
  Web scraping error jithin123 0 3,728 Mar-22-2020, 08:13 PM
Last Post: jithin123
  Web Scraping Error : Not getting expected result adminravi 4 3,949 Oct-08-2019, 09:53 AM
Last Post: snippsat
  Scraping data saving to DB error with Cursor cubangt 3 4,376 May-20-2019, 08:30 PM
Last Post: Yoriz
  Error while scraping item price from online store mgtheboss 2 6,751 Jan-12-2018, 06:42 PM
Last Post: snippsat
  Error while scraping links with beautiful soup mgtheboss 4 11,079 Dec-22-2017, 12:41 PM
Last Post: mgtheboss

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020
This forum uses Lukasz Tkacz MyBB addons.
Forum use Krzysztof "Supryk" Supryczynski addons.