Product Design, UI/UX Design, Secondary Research, User Interviews, Art Direction, Design Workshops
Captions started off as the one-to-go video captions editor for beginners. Powered by AI, the appcan automatically split the frames to catch blank spots and remove all the filling words like "Mmh", "Euh", etc. Just press the button and ta-da, it's done!
When they first reached me the goal was to redesign the app and enrich the product withdifferent basic video editor functionalities such as split, trim, pic in pic, transitions, etc. Weobviously also built on captions functionalities and enabled more options such as highlightingwords, editing style, matching stickers to words, and so on.
Soon the company started surfing on the wave of AI's new generation language models andpivoting towards Artificial Intelligence as the center of their offering. From voice assistants to text-to-image generators, AI opens a universe of possibilities and through the power of thistechnology, Captions decided to turn speech recognition to the next level.
Lip dub was the first feature we designed, a powerful one where you can basically change whatyou're saying. Use cases examples:
User engagement and response were so positive that we decided to bring it to the next leveland create Captions' very own script-writing assistant. Imagine, you just describe what youwant to talk about and the app will write the entire script for you.
The first iteration was a simple ideas generator that can give daily ideas for users to generatecontent faster. It can scan the user project content, see what they talk about, and suggest ideasrelated to their interest. We imagined an experience where the user needs at least 5 projects to unlock the idea generator. That would encourage users to engage with the app in order to unlock the feature. In this experience, the user gets one idea (or a few) at a time and can shuffle, like, or control how random it is. It was already a step forward in injecting more AI power into the product, but why not push it even further?
What if the AI could take that idea and literally write that script for your next TikTok video or techreview you want to record? Meet Geniio, your script-writer assistant.
The Flows:
We tested the feature and got some feedback from users that some aspects of the AI were notclear. We also put priority on the scrip writer AI and moved the ideas generator to the next phase.
People record videos for different purposes and the format is very different from making an ad fortheir local hair saloon or an influencer talking about tech so we introduced a template section inthe editor
We made it clearer that they could describe exactly what they want like they would describe it to a real human, so we added explicit hint text and real examples.
We wanted to teach the users that filling in the inputs enables them to 'Generate' the scriptoutput. Once on the output screen (the script), the user can easily 'Edit' and go back to editormode. In fact, the input from users and the output from the AI are like a conversation.
We set a first-time-use event (scroll after 5 seconds) to trigger a bottom drawer that encourages
In the higher level of navigation, we placed the 'Version history' to enable users to access previousversions. From the script mode, the user can 'Record', 'Shuffle', or 'Delete'.
From that script, the user should be able to record it and go to teleprompter mode.
While in another app, the user should be able to see his script and record a video.
That feature is the result of a continuous conversation with the client to follow his vision andoptimised from user feedback. From ideas-only, ideas-to-script, to editor-script mode, it took afew tries to get the flow and the layout right while keeping the design smooth and the vibe right.
