ZenRowsUniversalScraper

ZenRows is an enterprise-grade web scraping tool that provides advanced web data extraction capabilities at scale. For more information about ZenRows and its Universal Scraper API, visit the official documentation.

This document provides a quick overview for getting started with ZenRowsUniversalScraper tool. For detailed documentation of all ZenRowsUniversalScraper features and configurations head to the API reference.

Overview

Integration details

Class	Package	JS support	Package latest
ZenRowsUniversalScraper	langchain-zenrows	❌

Tool features

Feature	Support
JavaScript Rendering	✅
Anti-Bot Bypass	✅
Geo-Targeting	✅
Multiple Output Formats	✅
CSS Extraction	✅
Screenshot Capture	✅
Session Management	✅
Premium Proxies	✅

Setup

Install the required dependencies.

pip install langchain-zenrows

Credentials

You'll need a ZenRows API key to use this tool. You can sign up for free at ZenRows.

import os

# Set your ZenRows API key
os.environ["ZENROWS_API_KEY"] = "<YOUR_ZENROWS_API_KEY>"

Instantiation

Here's how to instantiate an instance of the ZenRowsUniversalScraper tool.

import os

from langchain_zenrows import ZenRowsUniversalScraper

# Set your ZenRows API key
os.environ["ZENROWS_API_KEY"] = "<YOUR_ZENROWS_API_KEY>"

zenrows_scraper_tool = ZenRowsUniversalScraper()

You can also pass the ZenRows API key when initializing the ZenRowsUniversalScraper tool.

from langchain_zenrows import ZenRowsUniversalScraper

zenrows_scraper_tool = ZenRowsUniversalScraper(zenrows_api_key="your-api-key")

Invocation

Basic Usage

The tool accepts a URL and various optional parameters to customize the scraping behavior:

import os

from langchain_zenrows import ZenRowsUniversalScraper

# Set your ZenRows API key
os.environ["ZENROWS_API_KEY"] = "<YOUR_ZENROWS_API_KEY>"

# Initialize the tool
zenrows_scraper_tool = ZenRowsUniversalScraper()

# Scrape a simple webpage
result = zenrows_scraper_tool.invoke({"url": "https://httpbin.io/html"})
print(result)

Advanced Usage with Parameters

import os

from langchain_zenrows import ZenRowsUniversalScraper

# Set your ZenRows API key
os.environ["ZENROWS_API_KEY"] = "<YOUR_ZENROWS_API_KEY>"

zenrows_scraper_tool = ZenRowsUniversalScraper()

# Scrape with JavaScript rendering and premium proxies
result = zenrows_scraper_tool.invoke(
 {
 "url": "https://www.scrapingcourse.com/ecommerce/",
 "js_render": True,
 "premium_proxy": True,
 "proxy_country": "us",
 "response_type": "markdown",
 "wait": 2000,
 }
)

print(result)

Use within an agent

import os

from langchain_openai import ChatOpenAI # or your preferred LLM
from langchain_zenrows import ZenRowsUniversalScraper
from langgraph.prebuilt import create_react_agent

# Set your ZenRows and OpenAI API keys
os.environ["ZENROWS_API_KEY"] = "<YOUR_ZENROWS_API_KEY>"
os.environ["OPENAI_API_KEY"] = "<YOUR_OPEN_AI_API_KEY>"


# Initialize components
llm = ChatOpenAI(model="gpt-4o-mini")
zenrows_scraper_tool = ZenRowsUniversalScraper()

# Create agent
agent = create_react_agent(llm, [zenrows_scraper_tool])

# Use the agent
result = agent.invoke(
 {
 "messages": "Scrape https://news.ycombinator.com/ and list the top 3 stories with title, points, comments, username, and time."
 }
)

print("Agent Response:")
for message in result["messages"]:
 print(f"{message.content}")

API Reference:create_react_agent

API reference

For detailed documentation of all ZenRowsUniversalScraper features and configurations head to the ZenRowsUniversalScraper API reference.

For comprehensive information about the underlying API parameters and capabilities, see the ZenRows Universal API documentation.

Related

• Tool conceptual guide
• Tool how-to guides