Koraiio

Personal movie trailers, without collecting personal data

Every viewer sees the same trailer today. This system builds a trailer around the action each viewer prefers, on their own device, so their tastes never have to be shared.

Year
2020
Proof
Published in IEEE Access
Runs on
Viewer devices of different power
Architecture with an HLS server side holding M3U8 playlists, segment representations and thumbnail containers, connected over the network to an HLS client side with a persistent broadcast connection, thumbnail containers analyzer and HLS player that requests the trailer segments
Fig. 1 The server keeps the film; the viewer's device decides what goes in the trailer.

The challenge

Personal trailers made on a server need a profile of each viewer and a lot of computing for every film.

What we built

  1. The viewer's device scans the film's preview images for the kinds of action they like.
  2. It asks for just those scenes over a single connection.
  3. It strings them together into a personal trailer.

Results

  • Tested on 25 broadcast films, westerns and sports.
  • Designed to run on devices of different power and to adapt to network conditions.

Where this applies

  • Streaming catalogs and film archives
  • Recommendation previews that respect privacy

Under the hood

  • A fine-tuned Inception-v3 action model (UCF-101) labels thumbnails; segments are requested over a persistent HTTP connection from a local HLS server.
  • Paper - Mujtaba and Ryu, IEEE Access, vol. 8, 2020 (open access).

Inception-v3UCF-101HLSIISPersistent HTTP

A thumbnail container is split into recognized-action thumbnails and detailed thumbnails, mapped to segment entry and exit times, then decoded into the video sequence
Fig. 2 How recognized moments in the preview images become a sequence of scenes.
Sequence diagram between server and client over one connection - request thumbnail containers, send, acknowledge, request selected video chunks, send, acknowledge, close
Fig. 3 One connection carries the previews and then only the selected scenes.

Have a perception problem that has to run on real hardware?

Tell us what you need to detect, where it has to run and what it runs on. You'll get a reply from the founder.

Email gmujtaba@koraiio.com

Email
gmujtaba@koraiio.com
Founder
Dr. Ghulam Mujtaba
Product
DroSonic
Based in
Denver, Colorado