TL/DR;

PotpieAI introduces Web Access, enabling AI Agents to fetch real-time data from the internet, enhancing context-awareness and accuracy in responses. This feature leverages Firecrawl for structured web data extraction and tool calling to dynamically interact with specialized tools like GitHub issue retrieval. Developers can provide live links or let the AI autonomously fetch relevant data, streamlining workflows and improving AI-driven decision-making.

At Potpie, we continuously push the boundaries of our AI Agents' capabilities by integrating advanced features that enhance functionality and improve the user experience for developers.

‍With that goal in mind, we’re excited to introduce Web Access, a new feature that allows our AI Agents to retrieve real-time information from the internet, making them even more context-aware and precise in generating responses

This upgrade enables AI Agents to go beyond static, pre-trained knowledge by dynamically fetching and incorporating the latest data from external sources. Whether it’s retrieving documentation updates, analyzing GitHub issues, or referencing online technical resources, Web Access ensures that the responses remain accurate, relevant, and up to date, providing developers with the most informed guidance possible.

What is Web Access feature?

The "Web Access" feature empowers AI Agents to reference the internet while considering the context of the defined repository, enabling them to generate more accurate, relevant, and informed responses.

AI Agents can now reference the internet while maintaining awareness of the defined repository context, enabling them to generate precise, informed responses.

Core Capabilities:

• Live Link Referencing: Users can provide a live link within their prompt, allowing the AI Agent to retrieve and align external information with the repository’s context. This ensures that responses are both accurate and relevant.

• Autonomous Web Lookup: The AI Agent intelligently recognizes when a query requires external information and autonomously retrieves the necessary data—eliminating the need for users to manually supply links.

• Context-Aware Insights: Retrieved information is seamlessly integrated with the repository’s existing knowledge, ensuring that responses remain actionable and aligned with the codebase.

This feature enhances the AI Agent’s ability to provide well-informed, real-time insights while reducing manual effort.

Several AI tools on the market offer web access features, with ChatGPT being one of the most well-known. ChatGPT introduced web browsing as an experimental feature in GPT-4 Turbo, which became available to ChatGPT Plus users in September 2023. 

‍This feature allows users to include a live link in their prompt or ask ChatGPT to search for specific information and provide results. Unlike ChatGPT, competitors like Google’s Gemini, Claude, and Deepseek do not yet support live link referencing, meaning they can't fetch data from provided links in the same way.

Some other popular tools that support this feature:

• Perplexity - Perplexity’s web search feature allows users to input URLs, enabling the AI to retrieve and analyze live content. This ensures the generation of accurate, up-to-date responses based on real-time data.

• You.com - An AI-powered search engine that integrates real-time web search with conversational capabilities, delivering responses enriched with the latest internet data.

• Grok (xAI) - Grok, developed by xAI, integrates with the internet to provide real-time information. Users can input prompts with live links, allowing Grok to access and analyze the content for accurate, up-to-date responses.

How it works?

The Web Access feature is powered by a robust backend architecture leveraging Firecrawl. This open-source tool is instrumental in converting web content into LLM-ready data formats, such as clean Markdown or structured JSON, making web data readily accessible for AI processing.

Firecrawl provides a seamless way to extract data from the web, enabling our AI Agent to:

1. Link Extraction: Identify and extract URLs from user prompts.

2. Real-time Web Browsing: Access and browse specified web pages.

3. Data Scrapping and Structuring: Scrape and structure the required data for optimal processing.

4. Seamless Integration: Integrate retrieved information into the response generation process.

By incorporating Firecrawl, we ensure that our Web Access feature delivers accurate, well-structured, and context-aware responses, enhancing the AI Agent's ability to provide desired response

The Web Access feature operates on the concept of function calling or tool calling, enabling it to interact dynamically with specialized tools to generate accurate and context-aware responses. This structured approach allows AI Agents to break down complex queries and execute actions efficiently.

How Tool Calling Works:

Defining Tools
A predefined set of tools is provided to the LLM (Large Language Model), allowing it to interact with different functionalities based on user queries. At Potpie, we have a growing toolkit, including:

• Webpage Content Extractor – Extracts the full text content from a webpage, including metadata, ensuring structured data retrieval. This tool helps analyze and process web content by fetching the page’s text while preserving relevant details.

• GitHub Tool – Retrieves GitHub issues and PR details, including diffs, for repository analysis. Requires valid GitHub credentials and returns structured metadata and content.

To know more about tools, check out our docs.

2. LLM Decision-Making
   When a user submits a query, the LLM processes the input and decides which tool(s) should be used to retrieve or generate the required information. Instead of relying solely on pre-trained knowledge, the AI dynamically selects the best tool based on the prompt’s context.

3. Executing the Tools
   Once the LLM identifies the appropriate tools, the agent framework executes them, fetching data, running computations, or extracting structured insights from external or internal sources. 

4. Final Output Structuring
   After the tools generate their respective outputs, the LLM refines, manipulates, and structures the response to match the expected format. This step ensures that the final output is not only accurate but also easy to understand, whether it's a natural language explanation, a structured dataset, or a formatted code snippet.

By leveraging tool calling, Potpie’s AI Agents can think, execute, and refine—making them highly powerful, and capable of solving complex, real-world problems with precision.

How to use it

As we discussed earlier in this blog, there are two main ways to use the web access feature: providing a live link to an external resource and including specific details about the external source in the prompt.

Let’s take a look at both these ways with the help of an example.

1. Providing a Live Link of an External Resource

The user can directly include a live URL in their prompt. The AI Agent will extract the link, access the webpage, and retrieve relevant information from the content. This method is useful when the user wants to reference a specific webpage ensuring precise and contextually relevant responses.

‍Here’s an example:

Here, we are using the Debugging with Knowledge Graph Agent to resolve an issue. We provided the agent with an issue link and instructed it to generate the exact code needed to fix the problem.

2. Providing Specific Details About an External Source

If the user does not provide a direct link but instead asks the AI Agent to look up a specific thing in the GitHub codebase, the Web Access feature is intelligent enough to recognize this request. The AI Agent will autonomously search for relevant information online, extract key insights, and use them to generate a well-informed response. This makes the process seamless, removing the need for manual link insertion while still ensuring access to the latest, most relevant data.

Here’s an example:

Here, we are again using the Debugging with Knowledge Graph Agent. However, instead of a live link, we provided an issue number and instructed the agent to resolve it. The agent searches for the issue on the live GitHub Issues page of the specified repository, retrieves its context, and generates an optimal solution.

To know more about this feature, checkout the demo:

https://x.com/aditikothari_/status/1890113806384374195

Checkout our Quickstart guide and start building with Potpie: https://docs.potpie.ai/quickstart

Got stuck anywhere? Join our Discord Server - https://discord.gg/NxUQCcz6