Python Forum
How to get html information from a tab of my default browser opened with webbrowser?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to get html information from a tab of my default browser opened with webbrowser?
#1
Question 
I have tried to figure out how can I get html information (such as h1 tags) from a new tab in my default browser that was opened with the following program:

 import tkinter as tk #from tkinter import filedialog import webbrowser from bs4 import BeautifulSoup root = tk.Tk() #create a GUI element root.geometry('500x400') #resolution root.title("Bulkdozer") #Name of this program root.attributes('-topmost', True) #keep the program's window top-most def open_chrome_profile(): soup = BeautifulSoup(webbrowser.open_new_tab('https://opensea.io/asset/create'), 'html.parser') #open a new tab using user's default browser content = soup.find('h1', {"class": "Blockreact__Block-sc-1xf18x6-0 Textreact__Text-sc-1w94ul3-0 dBFmez llcnwK"}) print(content) #####BUTTON ZONE####### open_browser = tk.Button(root, width=20, text="Open OpenSea Tab", command=open_chrome_profile) #executes the function when clicked open_browser.grid(row=22, column=1) #####BUTTON ZONE END####### root.mainloop()
The program above does open the corresponding tab, but also throws the following error at the line of soup variable:

Output:
TypeError: object of type 'bool' has no len()
And it doesn't print the content variable.

The reason for developing this in such a particular way is because I'm trying to make a program that will take control of user's default browser to make some repetitive tasks, and such automated process must be done using the default browser because it will have the Metamask extension that has been previously installed by the user.

May I know any ways of solving this problem?
Reply
#2
webbrowser.open_new_tab() returns boolean value. It does not return html source of the page you open.

You can use requests package or if the webpage uses JavaScript you should use tools like Selenium. webbrowser module from standard library is useless for a task such this.
Alternatively, or I would say as a better approach, you can explore the available third-party package to deal with this specific site/API
https://pypi.org/search/?q=opensea
noahverner1995 likes this post
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
(Jan-14-2022, 09:09 AM)buran Wrote: webbrowser.open_new_tab() returns boolean value. It does not return html source of the page you open.

You can use requests package or if the webpage uses JavaScript you should use tools like Selenium. webbrowser module from standard library is useless for a task such this.
Alternatively, or I would say as a better approach, you can explore the available third-party package to deal with this specific site/API
https://pypi.org/search/?q=opensea

Thank for the feedback, I think I managed to deal with it, here:

import tkinter as tk #from tkinter import filedialog #import webbrowser from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.service import Service root = tk.Tk() root.geometry('500x400') #resolution root.title("Bulkdozer") #Name of this program root.attributes('-topmost', True) #keep the program's window top-most opt = Options() #selenium options opt.add_argument("--user-data-dir="+r"C:\Users\ResetStoreX\AppData\Local\Google\Chrome\User Data") #PATH profile opt.add_argument('--profile-directory=Default') #Profile to use s = Service('C:/Users/ResetStoreX/AppData/Local/Programs/Python/Python39/Scripts/chromedriver.exe') def open_chrome_profile(): #webbrowser.open_new_tab('https://opensea.io/asset/create') #open a new tab using user's default browser driver = webdriver.Chrome(service=s, options=opt) driver.get('https://opensea.io/asset/create') #####BUTTON ZONE####### open_browser = tk.Button(root, width=20, text="Open OpenSea Tab", command=open_chrome_profile) #executes the function when clicked open_browser.grid(row=22, column=1) #####BUTTON ZONE END####### root.mainloop()
I removed the Default word from the profile PATH.

And I passed the absolute path of the ChromeDriver within the Service() function.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  .xlsm file can't be opened after writing data to one worksheet mhyatt 4 724 Oct-11-2025, 11:25 PM
Last Post: Pedroski55
  does not save in other path than opened files before icode 3 5,106 Jun-23-2023, 07:25 PM
Last Post: snippsat
  Webbrowser.open() causes errors ChrisOfBristol 1 1,810 Apr-09-2023, 08:34 PM
Last Post: deanhystad
  Tkinterweb (Browser Module) Appending/Adding Additional HTML to a HTML Table Row AaronCatolico1 0 2,873 Dec-25-2022, 06:28 PM
Last Post: AaronCatolico1
  reading html and edit chekcbox to html jacklee26 5 5,601 Jul-01-2021, 10:31 AM
Last Post: snippsat
  Rmarkdown opened by python code - errors Rav013 0 3,088 Apr-27-2021, 03:13 PM
Last Post: Rav013
  How to get a URL from python 'webbrowser'? dheeraj 0 3,090 Apr-05-2021, 03:55 PM
Last Post: dheeraj
  Webbrowser jbrick97 3 108,439 Sep-27-2020, 03:58 AM
Last Post: ndc85430
  Making WebBrowser In PySide2 Harshil 0 2,590 Sep-16-2020, 05:03 PM
Last Post: Harshil
  webbrowser and menu not working. sik 1 2,835 Oct-31-2019, 03:39 AM
Last Post: newbieAuggie2019

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020
This forum uses Lukasz Tkacz MyBB addons.
Forum use Krzysztof "Supryk" Supryczynski addons.