OpenAI Powers Responses API with Computer Agents

OpenAI Equips Responses API with Computer Environment, Turning Models into Persistent Agents

Key Facts

What: OpenAI launched updates to the Responses API enabling developers to run secure, stateful agents inside OpenAI-hosted containers with a full shell tool, file system access, and persistent state.
Core Components: New Shell Tool for terminal access, container_auto option for managed compute, built-in file search with in-house vector storage, and web search tool.
Model Performance: Computer-Using Agent (CUA) achieves 38.1% on OSWorld benchmark; described as one of the first agentic models capable of directly interacting with software environments.
Availability: Available now via the Responses API; open-source Agents SDK released on top of the new system.
Safety Focus: Updated system card addresses misuse prevention, unintended actions, and adversarial risks; human oversight strongly recommended for non-browser tasks.

Lead

OpenAI has taken a major step from conversational AI toward autonomous agents by equipping its newly introduced Responses API with a hosted computer environment, complete with a full shell tool and persistent container runtime. The update allows developers to build stateful agents that maintain files, tools, and memory across interactions, effectively transforming GPT models from stateless assistants into persistent system processes. Announced via OpenAI’s official channels, the release includes an open-source Agents SDK, enhanced file search capabilities using in-house vector storage, and a Computer Use tool that enables direct interaction with digital environments.

Building an Agent Runtime

According to OpenAI’s announcement, the company built an agent runtime using the Responses API, a new Shell Tool, and hosted containers. The system lets developers opt into container_auto, which automatically provisions an OpenAI-hosted secure container for each agent session. This marks OpenAI’s entry into managed compute territory, moving beyond simple API calls into environments where models can execute shell commands, read and write files, and maintain long-term state.

The Responses API serves as the foundation. It supports multiple built-in tools including:

Web search (powered by the same model behind ChatGPT Search, providing real-time information with citations)
File search with native vector storage, eliminating the need for developers to handle chunking and embedding pipelines themselves
The new Shell Tool / Computer Use capability

This combination allows agents to operate with a complete terminal shell inside a sandboxed container. The architecture turns the model from a “forgetful assistant” into a persistent process that retains context, files, and tool outputs across multiple turns, according to coverage in VentureBeat.

Technical Details and Benchmarks

OpenAI’s Computer-Using Agent (CUA) model is positioned as one of the first agentic AI systems capable of directly manipulating software environments. Early benchmark results show it achieving 38.1% success on the OSWorld benchmark, a test designed to evaluate realistic computer use tasks across operating systems.

The company acknowledges significant limitations. Performance on OSWorld indicates the model is not yet highly reliable for general OS automation. OpenAI’s updated system card emphasizes that human oversight remains strongly recommended, particularly for tasks outside of controlled browser environments. The release includes detailed safety analysis covering API-specific risks, misuse prevention, unintended actions, and potential adversarial attacks.

The hosted container approach provides security boundaries while offering developers a realistic Linux-like shell environment. Agents can install packages, manipulate files, run scripts, and interact with the filesystem in ways previously difficult to achieve reliably through API-only interfaces. OpenAI hosts and manages the underlying compute, simplifying infrastructure for developers while maintaining control over security and resource allocation.

Competitive Context and Industry Implications

This release positions OpenAI directly against other agent frameworks and computer-use initiatives from competitors. Anthropic has explored similar computer-use capabilities, while several startups focus on agent orchestration and persistent runtimes. Microsoft’s Azure AI Foundry has already announced integration of the Responses API and Computer-Using Agent, highlighting the enterprise interest in these capabilities.

The open-source Agents SDK built atop the Responses API is expected to accelerate adoption. Developers on platforms like Reddit have described the Computer Use tool as “pivotal” for building unorthodox AI integrations. The native file search with in-house vector storage removes a major pain point in retrieval-augmented generation (RAG) implementations, potentially simplifying agent memory systems.

OpenAI executives emphasized the foundational nature of these tools. One quote from the announcement states: “The world is so complex, there are so many industries and use cases… and so we’re super excited to provide those foundations, those building blocks for developers to build the best agents for their use case, their needs.”

Impact on Developers and the Industry

For developers, the Responses API with computer environment significantly lowers the barrier to building sophisticated agents. Previously, creating persistent, stateful agents required managing external compute, container orchestration, and complex state persistence layers. OpenAI now offers this as a managed service integrated directly with its frontier models.

The Shell Tool enables agents to perform actions such as:

Installing and running software packages
Processing and transforming files
Executing multi-step workflows that require filesystem state
Interacting with command-line tools and scripts
Maintaining project directories across conversations

Enterprise users gain the ability to build domain-specific agents that can work with proprietary codebases, analyze datasets, generate reports, and automate complex digital workflows while operating within OpenAI’s security and compliance boundaries.

The release also signals OpenAI’s strategic shift toward agentic AI. Rather than focusing solely on improving reasoning within chat interfaces, the company is investing in infrastructure that lets models act directly in digital environments. This mirrors broader industry movement toward AI agents that can plan, execute, and adapt over long horizons.

Safety and Responsible Development

OpenAI has published an updated system card detailing safety work specific to the Responses API and computer-use features. The document addresses new challenges that arise when models can take direct actions in software environments, including risks of misuse, unintended consequences, and adversarial manipulation.

The company stresses that the current capabilities, while groundbreaking, still require careful implementation. The 38.1% OSWorld score underscores that these systems are best used in scenarios with appropriate guardrails and human supervision. OpenAI recommends starting with lower-risk applications and gradually expanding agent autonomy as reliability improves.

What’s Next

OpenAI has framed this release as providing foundational building blocks rather than complete agent solutions. The company expects the developer community to build specialized agents on top of the Responses API and Agents SDK for diverse industry use cases.

Future improvements are likely to focus on increasing reliability on computer-use benchmarks, expanding the range of supported environments, and enhancing safety mechanisms for autonomous operation. Integration with other OpenAI tools and services, as well as deeper enterprise features through Azure, will probably follow.

The open-source nature of the Agents SDK should encourage rapid experimentation and the emergence of community best practices for agent design, observability, and evaluation. GitHub repositories already appearing with guides to the Responses API, tools, and SDK suggest strong early interest from the developer community.

As agentic capabilities mature, the industry may see a shift from prompt engineering toward environment design and agent governance. OpenAI’s move into managed compute with the Responses API represents a significant infrastructure bet on the future of autonomous AI systems.

From model to agent: Equipping the Responses API with a computer environment

Sources

Original Source

Related Topics

Comments