Alibaba experiments with basic captions for deaf live-streaming e-commerce sellers
- Kaola, a cross-border e-commerce platform, is one of the first in China to feature live shopping videos hosted by deaf streamers
- Real-time machine translation of sign language remains a difficult challenge
Alibaba is the parent company of the South China Morning Post.
Live-streaming shopping sessions are typically full of chatty sales pitches. But in a nearly two-hour session on Thursday, two women with hearing impairments co-hosted a relatively quiet live stream on Kaola, an online platform for overseas brands selling to Chinese shoppers.
The guest hosts were also joined by a Kaola live-streamer who spoke, and together they showed off products ranging from mixed nuts to ready-to-eat French fries. For viewers, captions appeared above the hosts’ heads summarising what was being expressed in sign language.
Some local media dubbed it a “sign language recognition system”, but it did not appear to have been performing sophisticated real-time translation. Rather, the captions showed words and short phrases such as “tastes good” or “sweet” around thirty seconds after they were signed.
The words for the captions first appeared as pinyin, the common romanisation system for Mandarin Chinese often used for typing, before they were rendered into Chinese characters. Alibaba did not immediately respond to questions about the technology.
