| Copyright | (c) 2025 Tushar Adhatrao |
|---|---|
| License | MIT |
| Maintainer | Tushar Adhatrao <tusharadhatrao@gmail.com> |
| Stability | experimental |
| Safe Haskell | None |
| Language | Haskell2010 |
Langchain.Tool.WebScraper
Description
WebScraper is a tool that scrapes text content from a given URL. It fetches the HTML content of the page, extracts the body text, removes scripts, and strips classidstyle attributes from the HTML tags. It is designed to be used with the Langchain framework for building language models and applications.
Synopsis
- data WebScraper = WebScraper
- data WebPageInfo = WebPageInfo {
- pageTitle :: Maybe Text
- pageContent :: Text
- fetchAndScrape :: Text -> IO (Either String WebPageInfo)
Documentation
data WebScraper Source #
Represents a web scraper tool that extracts content from web pages
Constructors
| WebScraper |
Instances
| Show WebScraper Source # | |||||||||
Defined in Langchain.Tool.WebScraper Methods showsPrec :: Int -> WebScraper -> ShowS # show :: WebScraper -> String # showList :: [WebScraper] -> ShowS # | |||||||||
| Tool WebScraper Source # | Implement the Tool typeclass for WebScraper | ||||||||
Defined in Langchain.Tool.WebScraper Associated Types
Methods toolName :: WebScraper -> Text Source # toolDescription :: WebScraper -> Text Source # runTool :: WebScraper -> Input WebScraper -> IO (Output WebScraper) Source # | |||||||||
| type Input WebScraper Source # | |||||||||
Defined in Langchain.Tool.WebScraper | |||||||||
| type Output WebScraper Source # | |||||||||
Defined in Langchain.Tool.WebScraper | |||||||||
data WebPageInfo Source #
Stores the extracted webpage information
Constructors
| WebPageInfo | |
Fields
| |
Instances
fetchAndScrape :: Text -> IO (Either String WebPageInfo) Source #
Fetch HTML content from a URL and extract webpage information