Tencent Effect Studio

an application designed for designers with non-tech background to build augmented reality experiences

Product Manager, Tencent, Shanghai

May 2022

Tencent Effect Studio is a pc-based software with atomic component design and visual scripting system to help third-party designers without tech background to build AR experience.

Background

With strong user demand for updating the AR filter list and the constraint on internal designer capacity, we seek third-party designers and developers to elevate the production. Thus, a tool designed for them to build AR lenses became necessary to meet the goal.

Visual Design Interface

I examined some features and found that most properties remain in common but only a few vary. The diagram above shows some examples.

To better balance user learnability and extensibility for the future, after researching and analyzing existing tools including Spark AR Studio, Lens Studio, and Effect Creator, I finally decided to apply ESC design principle to structure the application.

on wikipedia:

Entity Component System (ECS) is a software architectural pattern mostly used in video game development for the representation of game world objects. An ECS comprises entities composed from components of data, with systems which operate on entities' components.

The interface consists of five main panels:

1. Assets: where users organize their source materials for the AR lens, including images, gifs, textures, etc.

2. Component Panel: where users fill assets into the field, combine multiple components and edit the properties

3. Entity Panel: where users create and configure layers

4. Viewport: the viewport changes into 2D/3D view according to different types of entities. Users can see the effect as they build it.

5. Simulator: representing a device's screen, where users can preview lenses by turning on web cam on their web cam.

Scripting Interface v1.0
Paragraph

The interface introduced above manages the rendering and decides what an AR lens looks like at a moment. However, an AR lens for a short video takes a period of time, which always involves triggers and logic to make the whole experience more interactive and interesting.

3-paragraph is a typical framework in short-video platforms, where users trigger the transition with a hand gesture or facial expression. The transition only plays once, and eventually settles down in paragraph 3.

Thus, I designed the version in a paragraph form in version 1.

Scripting Interface v2.0
Node-based

However, with more complicated flow paths designed, they revealed the weakness of scripting v1.0.

The system can't control the item directly, but only through 'paragraph'. A quiz game with scores like the video below requires designers to enumerate the whole tree structure.

I started seeking a solution of direct control.

The design principles can be abstracted as follows:

1、 System Design: event condition action (ECA)

The event part specifies the signal that triggers the invocation of the rule
The condition part(optional) is a logical test that, if satisfied, causes the action to be carried out
The action part consists of updates or invocations on the local data

Multiple events can co-exist in the system. A few examples of combinations are listed as follows.

2. Node Design: a node consists of ports, variables, and configurations

Different knots are marked with different shapes to indicate how a node connects with one another.

The node is colored in different ways to indicate its category.

The 'Panda Sticker' example mentioned above is structured as below.

Ecosystem

The Tencent Effect Studio is only a tool to build AR lenses. To maintain the relationship with third-party designers and developers, we formed an ecosystem with our users with data feedback and cash reward.

Outcomes

500+

Lenses

Over 500 lenses have been submitted through Tencent Effect Studio

200+

Creators

Over 200 creators have registered on the platform

50%+

Contribution

Third-party designers and developers consist over 50% of total lenses contribution

Types

The platform has expanded to video templates, stickers and video clips, in addition to AR lenses