Multimodal UX: voice, text, image and video in the same flow
Designing experiences where the user combines voice, text, image and video input in one natural conversation, without switching tools for each input type.
The problem we solve.
Until recently, multimodal input was a dream: your product understood text, or image, or voice, but rarely all three at once and naturally. Modern multimodal models (Claude, GPT-4o, Gemini) make it possible - but the UX for those flows is still largely to be invented.
We design experiences where the user can take a photo, comment on it by voice and refine by text in the same interaction, with an agent that keeps the thread and updates the answer at every step.
What is included
- Design of flows combining ≥2 input modalities
- Immediate feedback patterns for each modality
- Error handling specific to each channel (poorly recorded audio, blurry image)
- Microcopy and guidance that help users adopt the new modalities
- User tests on the specific friction of each modality
Our process for Multimodal UX.
Mapping
We identify which tasks are best done in each modality and where the user gains by switching between them.
Design
Navigable prototypes with real voice, image and text. Testing on the target audience's devices.
Calibration
Iteration with real users: in multimodal, the human factor weighs more than in traditional interfaces.
Typical use cases
Customer service capturing a product image + a voice description. Health apps with a photo of a symptom + a spoken comment. Field assistants (technicians, sales reps) who need to capture context quickly without stopping to type. Education with oral explanation + illustration generated on the fly. The more natural it is to switch modality, the more each of these cases unlocks.
Other services within AI-native experiences
AI product strategy
How to integrate AI into your existing product or build a new one with AI at the centre.
View service →Functional AI prototyping
Functional prototypes connected to real LLMs to validate use cases in days, not months.
View service →AI agent design
Designing the agent experience: how they present themselves, escalate to humans and manage uncertainty.
View service →Context engineering
Designing the context that feeds the models: prompts, tools, memory, RAG and usage policies.
View service →Generative UI
Interfaces built on the fly based on user intent and the available context.
View service →