The area of ad-hoc automatically generated UI fascinates me and I see it as key for modern applications born in an AI world. We will certainly see applications that use non-visual modalities, for example voice, but screens will stay with us for some time. I do not think it is likely that UI will keep being universally pre-programmed inside every application.
Introduction
Universal UI for AI vs. AI-first UI
I expect two approaches:
That will be, for example, Copilot or Excel. A universal way to interact with AI will exist, and my specific applications will live inside it. Chat is useful for advice and overviews, but financial planning and budgets often belong in Excel.
Some applications will be so different and complex that consuming them from a universal UI such as Copilot or PowerPoint will not be suitable. For musicians, for example, the product might need humming, playback and arrangement controls rather than a chat box.
Three paths to creating visual GenUI and how they compare
I see three paths emerging for generated visual user interfaces (GenUI):
Three paths
- 1Ad-hoc generated static artifacts
The output is an image or a similar artifact.
- 2Ad-hoc generated declarative UI specifications
AI composes predefined components.
- 3Fully generated interactive UI through ad-hoc imperative code generation
AI creates a small custom application.
Let's look at them more closely and compare their advantages and disadvantages.
This became widespread already in 2023. It is typically used to generate charts and diagrams from tabular data: Code Interpreter writes Python with Matplotlib or Seaborn, produces an image, pulls it from the sandbox and shows it to the user.
- Very safe: the sandbox typically cannot access the internet and the user receives only an image.
- Easy to store in history and recall later.
- Zero interactivity: the chart cannot be manipulated, filtered or zoomed.
- Visual limitations and weaker adaptation to the application's design.
This variant separates the display code from the way we say what should be displayed. AI generates within a declarative language and chooses from predefined components. Examples include Adaptive Cards and A2UI.
- Safe: AI cannot generate arbitrary code.
- Interactive: the user can typically click on things.
- Design is controlled by the client.
- Limited to the component set.
- Harder to implement and maintain.
What if we give AI a free hand to generate any code and show that code to the user in an iframe? This enables custom mini-applications. New standards such as MCP-Apps push this further and are starting to appear in tools such as VS Code Insiders.
- Maximum flexibility.
- Full interactivity: literally mini-applications.
- Security risks require sandboxing and other controls.
- Design can be inconsistent without strong instructions.
Summary
Static artifacts
Safe and easy to store, but not interactive.
Declarative specifications
Safer and interactive, but limited by the component set.
Imperative code
Maximum flexibility and mini-applications, but with security and design risks.
Final question
So, do your applications already have #GenUI capabilities? Which path are you planning to use?