ComponentsDocument loadersDocument loaders📄️ acreomacreom is a dev-first knowledge base with tasks running on local markdown files.📄️ Airbyte CDKAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte GongAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte HubspotAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte JSONAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte SalesforceAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte ShopifyAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte StripeAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte TypeformAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airbyte Zendesk SupportAirbyte is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases.📄️ Airtable* Get your API key here.📄️ Alibaba Cloud MaxComputeAlibaba Cloud MaxCompute (previously known as ODPS) is a general purpose, fully managed, multi-tenancy data processing platform for large-scale data warehousing. MaxCompute supports various data importing solutions and distributed computing models, enabling users to effectively query massive datasets, reduce production costs, and ensure data security.📄️ Apify DatasetApify Dataset is a scaleable append-only storage with sequential access built for storing structured web scraping results, such as a list of products or Google SERPs, and then export them to various formats like JSON, CSV, or Excel. Datasets are mainly used to save results of Apify Actors—serverless cloud programs for varius web scraping, crawling, and data extraction use cases.📄️ ArcGISThis notebook demonstrates the use of the langchain.document_loaders.ArcGISLoader class.📄️ ArxivarXiv is an open-access archive for 2 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.📄️ AssemblyAI Audio TranscriptsThe AssemblyAIAudioTranscriptLoader allows to transcribe audio files with the AssemblyAI API and loads the transcribed text into documents.📄️ Async ChromiumChromium is one of the browsers supported by Playwright, a library used to control browser automation.📄️ AsyncHtmlAsyncHtmlLoader loads raw HTML from a list of URLs concurrently.📄️ AWS S3 DirectoryAmazon Simple Storage Service (Amazon S3) is an object storage service📄️ AWS S3 FileAmazon Simple Storage Service (Amazon S3) is an object storage service.📄️ AZLyricsAZLyrics is a large, legal, every day growing collection of lyrics.📄️ Azure Blob Storage ContainerAzure Blob Storage is Microsoft's object storage solution for the cloud. Blob Storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn't adhere to a particular data model or definition, such as text or binary data.📄️ Azure Blob Storage FileAzure Files offers fully managed file shares in the cloud that are accessible via the industry standard Server Message Block (SMB) protocol, Network File System (NFS) protocol, and Azure Files REST API.📄️ Azure Document IntelligenceAzure Document Intelligence (formerly known as Azure Forms Recognizer) is machine-learning📄️ BibTeXBibTeX is a file format and reference management system commonly used in conjunction with LaTeX typesetting. It serves as a way to organize and store bibliographic information for academic and research documents.📄️ BiliBiliBilibili is one of the most beloved long-form video sites in China.📄️ BlackboardBlackboard Learn (previously the Blackboard Learning Management System) is a web-based virtual learning environment and learning management system developed by Blackboard Inc. The software features course management, customizable open architecture, and scalable design that allows integration with student information systems and authentication protocols. It may be installed on local servers, hosted by Blackboard ASP Solutions, or provided as Software as a Service hosted on Amazon Web Services. Its main purposes are stated to include the addition of online elements to courses traditionally delivered face-to-face and development of completely online courses with few or no face-to-face meetings📄️ BlockchainOverview📄️ Brave SearchBrave Search is a search engine developed by Brave Software.📄️ BrowserlessBrowserless is a service that allows you to run headless Chrome instances in the cloud. It's a great way to run browser-based automation at scale without having to worry about managing your own infrastructure.📄️ ChatGPT DataChatGPT is an artificial intelligence (AI) chatbot developed by OpenAI.📄️ College ConfidentialCollege Confidential gives information on 3,800+ colleges and universities.📄️ Concurrent LoaderWorks just like the GenericLoader but concurrently for those who choose to optimize their workflow.📄️ ConfluenceConfluence is a wiki collaboration platform that saves and organizes all of the project-related material. Confluence is a knowledge base that primarily handles content management activities.📄️ CoNLL-UCoNLL-U is revised version of the CoNLL-X format. Annotations are encoded in plain text files (UTF-8, normalized to NFC, using only the LF character as line break, including an LF character at the end of file) with three types of lines:📄️ Copy PasteThis notebook covers how to load a document object from something you just want to copy and paste. In this case, you don't even need to use a DocumentLoader, but rather can just construct the Document directly.📄️ CSVA comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. Each line of the file is a data record. Each record consists of one or more fields, separated by commas.📄️ Cube Semantic LayerThis notebook demonstrates the process of retrieving Cube's data model metadata in a format suitable for passing to LLMs as embeddings, thereby enhancing contextual information.📄️ Datadog LogsDatadog is a monitoring and analytics platform for cloud-scale applications.📄️ DiffbotUnlike traditional web scraping tools, Diffbot doesn't require any rules to read the content on a page.📄️ DiscordDiscord is a VoIP and instant messaging social platform. Users have the ability to communicate with voice calls, video calls, text messaging, media and files in private chats or as part of communities called "servers". A server is a collection of persistent chat rooms and voice channels which can be accessed via invite links.📄️ DocugamiThis notebook covers how to load documents from Docugami. It provides the advantages of using this system over alternative data loaders.📄️ DropboxDrobpox is a file hosting service that brings everything-traditional files, cloud content, and web shortcuts together in one place.📄️ DuckDBDuckDB is an in-process SQL OLAP database management system.📄️ EmailThis notebook shows how to load email (.eml) or Microsoft Outlook (.msg) files.📄️ Embaasembaas is a fully managed NLP API service that offers features like embedding generation, document text extraction, document to embeddings and more. You can choose a variety of pre-trained models.📄️ EPubEPUB is an e-book file format that uses the ".epub" file extension. The term is short for electronic publication and is sometimes styled ePub. EPUB is supported by many e-readers, and compatible software is available for most smartphones, tablets, and computers.📄️ EtherscanEtherscan  is the leading blockchain explorer, search, API and analytics platform for Ethereum,📄️ EverNoteEverNote is intended for archiving and creating notes in which photos, audio and saved web content can be embedded. Notes are stored in virtual "notebooks" and can be tagged, annotated, edited, searched, and exported.🗃️ example_data1 items📄️ Microsoft ExcelThe UnstructuredExcelLoader is used to load Microsoft Excel files. The loader works with both .xlsx and .xls files. The page content will be the raw text of the Excel file. If you use the loader in "elements" mode, an HTML representation of the Excel file will be available in the document metadata under the textashtml key.📄️ Facebook ChatMessenger) is an American proprietary instant messaging app and platform developed by Meta Platforms. Originally developed as Facebook Chat in 2008, the company revamped its messaging service in 2010.📄️ FaunaFauna is a Document Database.📄️ FigmaFigma is a collaborative web application for interface design.📄️ GeopandasGeopandas is an open source project to make working with geospatial data in python easier.📄️ GitGit is a distributed version control system that tracks changes in any set of computer files, usually used for coordinating work among programmers collaboratively developing source code during software development.📄️ GitBookGitBook is a modern documentation platform where teams can document everything from products to internal knowledge bases and APIs.📄️ GitHubThis notebooks shows how you can load issues and pull requests (PRs) for a given repository on GitHub. We will use the LangChain Python repository as an example.📄️ Google BigQueryGoogle BigQuery is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.📄️ Google Cloud Storage DirectoryGoogle Cloud Storage is a managed service for storing unstructured data.📄️ Google Cloud Storage FileGoogle Cloud Storage is a managed service for storing unstructured data.📄️ Google DriveGoogle Drive is a file storage and synchronization service developed by Google.📄️ GrobidGROBID is a machine learning library for extracting, parsing, and re-structuring raw documents.📄️ GutenbergProject Gutenberg is an online library of free eBooks.📄️ Hacker NewsHacker News (sometimes abbreviated as HN) is a social news website focusing on computer science and entrepreneurship. It is run by the investment fund and startup incubator Y Combinator. In general, content that can be submitted is defined as "anything that gratifies one's intellectual curiosity."📄️ Huawei OBS DirectoryThe following code demonstrates how to load objects from the Huawei OBS (Object Storage Service) as documents.📄️ Huawei OBS FileThe following code demonstrates how to load an object from the Huawei OBS (Object Storage Service) as document.📄️ HuggingFace datasetThe Hugging Face Hub is home to over 5,000 datasets in more than 100 languages that can be used for a broad range of tasks across NLP, Computer Vision, and Audio. They used for a diverse range of tasks such as translation,📄️ iFixitiFixit is the largest, open repair community on the web. The site contains nearly 100k repair manuals, 200k Questions & Answers on 42k devices, and all the data is licensed under CC-BY-NC-SA 3.0.📄️ ImagesThis covers how to load images such as JPG or PNG into a document format that we can use downstream.📄️ Image captionsBy default, the loader utilizes the pre-trained Salesforce BLIP image captioning model.📄️ IMSDbIMSDb is the Internet Movie Script Database.📄️ IuguIugu is a Brazilian services and software as a service (SaaS) company. It offers payment-processing software and application programming interfaces for e-commerce websites and mobile applications.📄️ JoplinJoplin is an open source note-taking app. Capture your thoughts and securely access them from any device.📄️ Jupyter NotebookJupyter Notebook (formerly IPython Notebook) is a web-based interactive computational environment for creating notebook documents.📄️ LarkSuite (FeiShu)LarkSuite is an enterprise collaboration platform developed by ByteDance.📄️ MastodonMastodon is a federated social media and social networking service.📄️ MediaWiki DumpMediaWiki XML Dumps contain the content of a wiki (wiki pages with all their revisions), without the site-related data. A XML dump does not create a full backup of the wiki database, the dump does not contain user accounts, images, edit logs, etc.📄️ Merge Documents LoaderMerge the documents returned from a set of specified data loaders.📄️ mhtmlMHTML is a is used both for emails but also for archived webpages. MHTML, sometimes referred as MHT, stands for MIME HTML is a single file in which entire webpage is archived. When one saves a webpage as MHTML format, this file extension will contain HTML code, images, audio files, flash animation etc.📄️ Microsoft OneDriveMicrosoft OneDrive (formerly SkyDrive) is a file hosting service operated by Microsoft.📄️ Microsoft PowerPointMicrosoft PowerPoint is a presentation program by Microsoft.📄️ Microsoft SharePointMicrosoft SharePoint is a website-based collaboration system that uses workflow applications, “list” databases, and other web parts and security features to empower business teams to work together developed by Microsoft.📄️ Microsoft WordMicrosoft Word is a word processor developed by Microsoft.📄️ Modern TreasuryModern Treasury simplifies complex payment operations. It is a unified platform to power products and processes that move money.📄️ MongoDBMongoDB is a NoSQL , document-oriented database that supports JSON-like documents with a dynamic schema.📄️ News URLThis covers how to load HTML news articles from a list of URLs into a document format that we can use downstream.📄️ Notion DB 1/2Notion is a collaboration platform with modified Markdown support that integrates kanban boards, tasks, wikis and databases. It is an all-in-one workspace for notetaking, knowledge and data management, and project and task management.📄️ Notion DB 2/2Notion is a collaboration platform with modified Markdown support that integrates kanban boards, tasks, wikis and databases. It is an all-in-one workspace for notetaking, knowledge and data management, and project and task management.📄️ NucliaNuclia automatically indexes your unstructured data from any internal and external source, providing optimized search results and generative answers. It can handle video and audio transcription, image content extraction, and document parsing.📄️ ObsidianObsidian is a powerful and extensible knowledge base📄️ Open Document Format (ODT)The Open Document Format for Office Applications (ODF), also known as OpenDocument, is an open file format for word processing documents, spreadsheets, presentations and graphics and using ZIP-compressed XML files. It was developed with the aim of providing an open, XML-based file format specification for office applications.📄️ Open City DataSocrata provides an API for city open data.📄️ Org-modeA Org Mode document is a document editing, formatting, and organizing mode, designed for notes, planning, and authoring within the free software text editor Emacs.📄️ Pandas DataFrameThis notebook goes over how to load data from a pandas DataFrame.📄️ Amazon TextractAmazon Textract is a machine learning (ML) service that automatically extracts text, handwriting, and data from scanned documents. It goes beyond simple optical character recognition (OCR) to identify, understand, and extract data from forms and tables. Today, many companies manually extract data from scanned documents such as PDFs, images, tables, and forms, or through simple OCR software that requires manual configuration (which often must be updated when the form changes). To overcome these manual and expensive processes, Textract uses ML to read and process any type of document, accurately extracting text, handwriting, tables, and other data with no manual effort. You can quickly automate document processing and act on the information extracted, whether you’re automating loans processing or extracting information from invoices and receipts. Textract can extract the data in minutes instead of hours or days.📄️ Polars DataFrameThis notebook goes over how to load data from a polars DataFrame.📄️ PsychicThis notebook covers how to load documents from Psychic. See here for more details.📄️ PubMedPubMed® by The National Center for Biotechnology Information, National Library of Medicine comprises more than 35 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full text content from PubMed Central and publisher web sites.📄️ PySparkThis notebook goes over how to load data from a PySpark DataFrame.📄️ ReadTheDocs DocumentationRead the Docs is an open-sourced free software documentation hosting platform. It generates documentation written with the Sphinx documentation generator.📄️ Recursive URLWe may want to process load all URLs under a root directory.📄️ RedditReddit is an American social news aggregation, content rating, and discussion website.📄️ RoamROAM is a note-taking tool for networked thought, designed to create a personal knowledge base.📄️ RocksetRockset is a real-time analytics database which enables queries on massive, semi-structured data without operational burden. With Rockset, ingested data is queryable within one second and analytical queries against that data typically execute in milliseconds. Rockset is compute optimized, making it suitable for serving high concurrency applications in the sub-100TB range (or larger than 100s of TBs with rollups).📄️ RSS FeedsThis covers how to load HTML news articles from a list of RSS feed URLs into a document format that we can use downstream.📄️ RSTA reStructured Text (RST) file is a file format for textual data used primarily in the Python programming language community for technical documentation.📄️ SitemapExtends from the WebBaseLoader, SitemapLoader loads a sitemap from a given URL, and then scrape and load all pages in the sitemap, returning each page as a Document.📄️ SlackSlack is an instant messaging program.📄️ SnowflakeThis notebooks goes over how to load documents from Snowflake📄️ Source CodeThis notebook covers how to load source code files using a special approach with language parsing: each top-level function and class in the code is loaded into separate documents. Any remaining code top-level code outside the already loaded functions and classes will be loaded into a seperate document.📄️ SpreedlySpreedly is a service that allows you to securely store credit cards and use them to transact against any number of payment gateways and third party APIs. It does this by simultaneously providing a card tokenization/vault service as well as a gateway and receiver integration service. Payment methods tokenized by Spreedly are stored at Spreedly, allowing you to independently store a card and then pass that card to different end points based on your business requirements.📄️ StripeStripe is an Irish-American financial services and software as a service (SaaS) company. It offers payment-processing software and application programming interfaces for e-commerce websites and mobile applications.📄️ SubtitleThe SubRip file format is described on the Matroska multimedia container format website as "perhaps the most basic of all subtitle formats." SubRip (SubRip Text) files are named with the extension .srt, and contain formatted lines of plain text in groups separated by a blank line. Subtitles are numbered sequentially, starting at 1. The timecode format used is hoursseconds,milliseconds with time units fixed to two zero-padded digits and fractions fixed to three zero-padded digits (0000,000). The fractional separator used is the comma, since the program was written in France.📄️ TelegramTelegram Messenger is a globally accessible freemium, cross-platform, encrypted, cloud-based and centralized instant messaging service. The application also provides optional end-to-end encrypted chats and video calling, VoIP, file sharing and several other features.📄️ Tencent COS DirectoryThis covers how to load document objects from a Tencent COS Directory.📄️ Tencent COS FileThis covers how to load document object from a Tencent COS File.📄️ TensorFlow DatasetsTensorFlow Datasets is a collection of datasets ready to use, with TensorFlow or other Python ML frameworks, such as Jax. All datasets are exposed as tf.data.Datasets, enabling easy-to-use and high-performance input pipelines. To get started see the guide and the list of datasets.📄️ 2Markdown2markdown service transforms website content into structured markdown files.📄️ TOMLTOML is a file format for configuration files. It is intended to be easy to read and write, and is designed to map unambiguously to a dictionary. Its specification is open-source. TOML is implemented in many programming languages. The name TOML is an acronym for "Tom's Obvious, Minimal Language" referring to its creator, Tom Preston-Werner.📄️ TrelloTrello is a web-based project management and collaboration tool that allows individuals and teams to organize and track their tasks and projects. It provides a visual interface known as a "board" where users can create lists and cards to represent their tasks and activities.📄️ TSVA tab-separated values (TSV) file is a simple, text-based file format for storing tabular data.[3] Records are separated by newlines, and values within a record are separated by tab characters.📄️ TwitterTwitter is an online social media and social networking service.📄️ Unstructured FileThis notebook covers how to use Unstructured package to load files of many types. Unstructured currently supports loading of text files, powerpoints, html, pdfs, images, and more.📄️ URLThis covers how to load HTML documents from a list of URLs into a document format that we can use downstream.📄️ WeatherOpenWeatherMap is an open source weather service provider📄️ WebBaseLoaderThis covers how to use WebBaseLoader to load all text from HTML webpages into a document format that we can use downstream. For more custom logic for loading webpages look at some child class examples such as IMSDbLoader, AZLyricsLoader, and CollegeConfidentialLoader📄️ WhatsApp ChatWhatsApp (also called WhatsApp Messenger) is a freeware, cross-platform, centralized instant messaging (IM) and voice-over-IP (VoIP) service. It allows users to send text and voice messages, make voice and video calls, and share images, documents, user locations, and other content.📄️ WikipediaWikipedia is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system called MediaWiki. Wikipedia is the largest and most-read reference work in history.📄️ XMLThe UnstructuredXMLLoader is used to load XML files. The loader works with .xml files. The page content will be the text extracted from the XML tags.📄️ Xorbits Pandas DataFrameThis notebook goes over how to load data from a xorbits.pandas DataFrame.📄️ YouTube audioBuilding chat or QA applications on YouTube videos is a topic of high interest.📄️ YouTube transcriptsYouTube is an online video sharing and social media platform created by Google.PreviousvLLM ChatNextacreom