Exclusive: Honor Debuts a New AI Agent That Can Read and Understand Your Screen

It selected a restaurant, however then could not full the method because the spot it selected required a bank card to verify a reservation, at which level the person needed to take over. You will be versatile in your question—in one other instance, asking it to guide a “extremely rated” restaurant meant it might have a look at opinions with excessive scores, although the agent does not do any extra analysis than that. It isn’t cross-referencing OpenTable opinions with information from different elements of the online, particularly since all of this information is processed on system and is not despatched to the cloud.

This sort of agentic synthetic intelligence is the current buzzword in the tech sphere. My colleague Will Knight recently tested an AI assistant that would browse the online and carry out duties on-line. Google late final 12 months unveiled its Gemini 2 AI model educated to take actions in your behalf. It additionally renews the thought of a generative person interface for smartphones—at MWC 2024, we noticed a number of firms engaged on methods to interact with apps without using apps at all, as a substitute leaning on AI assistants to generate a person interface as you issued a command.

Honor’s strategy feels considerably like what Rabbit—of the notorious Rabbit R1—is doing with Teach Mode, the place you practice its assistant manually to finish a activity. There is no have to entry an app’s Software Programming Interface (API), which is the normal approach apps or companies talk with one another. The agent memorizes the method, permitting you to then subject the command and have it execute the duty.

However Honor says its self-reliant AI execution mannequin is not educated to comply with strict steps—it is able to multimodal display context recognition to carry out duties autonomously. As an alternative of getting to coach the assistant to study each single a part of the OpenTable app, it’s able to understanding the semantic components of the person interface and can follow-through with a multi-step course of to execute your request. Honor highlighted that this course of was less expensive: “Not like rivals equivalent to Apple, Samsung, and Google, which depend on exterior APIs—leading to larger operational prices—Honor’s AI Agent independently manages a variety of duties.”

{Photograph}: Julian Chokkattu