Loaders v1.3
Overview
Loaders are used to load data from sources and parse it into Artifacts. Each loader can be used to load a single "source" with load() or multiple sources with load_collection().
File
The following Loaders load a file using a FileManagerDriver and loads the resulting data into an Artifact for the respective file type.
As a convenience, File Loaders also have a save() method that can save an Artifact to a destination.
from griptape.drivers.file_manager.local import LocalFileManagerDriver from griptape.loaders import TextLoader loader = TextLoader(file_manager_driver=LocalFileManagerDriver()) data = loader.load("tests/resources/test.txt") data.value = data.value.replace("foo", "bar") loader.save("tests/resources/test.txt", data)
Text
Loads text files into TextArtifacts:
import urllib.request from pathlib import Path from griptape.loaders import TextLoader TextLoader().load("tests/resources/test.txt") urllib.request.urlretrieve("https://example-files.online-convert.com/document/txt/example.txt", "example.txt") TextLoader().load(Path("example.txt")) TextLoader().load_collection(["tests/resources/test.txt", Path("example.txt")])
Info
This driver requires the loaders-pdf extra.
Loads PDF files into ListArtifacts, where each element is a TextArtifact containing a page of the PDF:
import urllib.request from pathlib import Path from griptape.loaders import PdfLoader urllib.request.urlretrieve("https://arxiv.org/pdf/1706.03762.pdf", "attention.pdf") # Load a single PDF file PdfLoader().load("attention.pdf") # You can also pass a Path object PdfLoader().load(Path("attention.pdf")) urllib.request.urlretrieve("https://arxiv.org/pdf/1706.03762.pdf", "CoT.pdf") # Load multiple PDF files PdfLoader().load_collection([Path("attention.pdf"), Path("CoT.pdf")])
CSV
Loads CSV files into ListArtifacts, where each element is a TextArtifact containing a row of the CSV:
from pathlib import Path from griptape.loaders import CsvLoader # Load a single CSV file CsvLoader().load("tests/resources/cities.csv") # You can also pass a Path object CsvLoader().load(Path("tests/resources/cities.csv")) # Load multiple CSV files CsvLoader().load_collection([Path("tests/resources/cities.csv"), "tests/resources/addresses.csv"])
Image
Info
This driver requires the loaders-image extra.
Loads images into ImageArtifacts:
from pathlib import Path from griptape.loaders import ImageLoader # Load an image from disk ImageLoader().load("tests/resources/mountain.png") # You can also pass a Path object ImageLoader().load(Path("tests/resources/mountain.png"))
By default, the Image Loader will load images in their native format, but not all models work on all formats. To normalize the format of Artifacts returned by the Loader, set the format field.
from pathlib import Path from griptape.loaders import ImageLoader # Load a single image in BMP format ImageLoader(format="bmp").load("tests/resources/mountain.png") # You can also pass a Path object ImageLoader(format="bmp").load(Path("tests/resources/mountain.png")) # Load multiple images in BMP format ImageLoader().load_collection([Path("tests/resources/mountain.png"), "tests/resources/cow.png"])
Audio
Loads audio files into AudioArtifacts:
The Loader will load audio in its native format and populates the resulting Artifact's format field by making a best-effort guess of the underlying audio format using the filetype package.
from pathlib import Path from griptape.loaders import AudioLoader # Load an image from disk AudioLoader().load("tests/resources/sentences.wav") # You can also pass a Path object AudioLoader().load(Path("tests/resources/sentences.wav"))
JSON
Loads JSON files into JsonArtifacts:
from pathlib import Path from griptape.loaders import JsonLoader # Load an image from disk JsonLoader().load("tests/resources/test.json") # You can also pass a Path object JsonLoader().load(Path("tests/resources/test.json"))
Web
Info
This driver requires the loaders-web extra.
Scrapes web pages using a WebScraperDriver and loads the resulting text into TextArtifacts.
from griptape.loaders import WebLoader WebLoader().load("https://www.griptape.ai") WebLoader().load_collection(["https://www.griptape.ai", "https://docs.griptape.ai"])
SQL
Loads data from a SQL database using a SQLDriver and loads the resulting data into ListArtifacts, where each element is a TextArtifact containing a row of the SQL query.
from griptape.drivers.sql.sql_driver import SqlDriver from griptape.loaders import SqlLoader SqlLoader(sql_driver=SqlDriver(engine_url="sqlite:///:memory:")).load("SELECT 'foo', 'bar'") SqlLoader(sql_driver=SqlDriver(engine_url="sqlite:///:memory:")).load_collection( ["SELECT 'foo', 'bar';", "SELECT 'fizz', 'buzz';"] )
Info
This driver requires the loaders-email extra.
Loads data from an imap email server into a ListArtifacts, where each element is a TextArtifact containing an email.
from griptape.loaders import EmailLoader loader = EmailLoader(imap_url="an.email.server.hostname", username="username", password="password") loader.load(EmailLoader.EmailQuery(label="INBOX")) loader.load_collection([EmailLoader.EmailQuery(label="INBOX"), EmailLoader.EmailQuery(label="SENT")])