Mar-07-2020, 02:56 PM
Hello Everyone.
I am trying to scrape a website using python. I want to scrape text inside a span, which is inside a div. but the span is thrown automatically there, after every 5 minutes. so I want the script to run forever, and scrap every time the value of span, when it appears.
My code is working fine locally when I have the HTML file in my local machine. but when I try to scrape from the website, it is not working.
Where could be the error?
More details here. Details here
I am trying to scrape a website using python. I want to scrape text inside a span, which is inside a div. but the span is thrown automatically there, after every 5 minutes. so I want the script to run forever, and scrap every time the value of span, when it appears.
My code is working fine locally when I have the HTML file in my local machine. but when I try to scrape from the website, it is not working.
Where could be the error?
More details here. Details here
from urllib.request import urlopen from bs4 import BeautifulSoup from bs4 import BeautifulSoup as soup # HTML data structure from urllib.request import urlopen as uReq # Web client import re import time import pyperclip while True: page_url = "https://www.example.com/" uClient = uReq(page_url) page_soup = soup(uClient.read(), "html.parser") div = page_soup.find('div',{'class':'content'}) if div: span = div.find('span') if not span: continue numbers = span.get_text() match = re.search('\d{5,}', numbers) card = match.group(0) print(card) time.sleep(2)