(By Dr Daniel Ng, Edmund Teo, Jway Jin Jun, Kow Yong Sheng, Sun Yikang, Tan Juan Boon)

To support the fight against COVID-19, AI Singapore has developed a social distancing application, FinePose, to encourage individuals to maintain physical distance from each other. While there are other solutions out there that are based on object detection, this initiative is unique as it is an offshoot from AI Singapore’s foray into human pose estimation. A new product for human pose will also be launched within the next few months.

The AI Behind It

There are two main components to obtain the distance between individuals. These include human pose estimation and distance projection heuristics.

We used an open source human pose estimation model known as PoseNet to identify key human skeletal points. This allows the application to identify where individuals are located within the video feed. The coordinates of the various skeletal points will then be used to determine the distance between individuals.

To measure the distance between individuals, we have to convert the keypoints in 2D coordinates to keypoints in 3D world coordinates. To achieve this, it is possible to estimate the depth (Z) from the XY coordinates. As such, the following heuristics are applied:

Using the similar triangle rule, we are able to compute Z.

Where:

Z = depth or distance of scene point from camera

f = focal length of camera

y = y position of image point

Y = y position of scene point

A reference or “ground truth length” is required to obtain the depth if specific calibration for every camera is to be avoided. After numerous experiments, it was decided that the optimal reference length would be the average height of a human torso (height from human hip to center of face). Width was not used as this value has high variance due to the different body angles of an individual while facing the camera.

Deployment on the Edge

To ensure privacy, the video feed is neither recorded nor stored. The inference is conducted in real-time with all the AI models deployed on the edge. This is unlike most video analytics solutions which usually process the videos on the cloud. One drawback of edge deployment is the use of non-ideal compute resources which may cause poor inference performance. Thus, FinePose was developed to be as lightweight as possible. This allows for flexible deployment across device types. You may access the FinePose mobile application here.

With the increasing number of COVID-19 infections worldwide, we hope that FinePose will be able to contribute to the fight against COVID-19. We believe that the collective actions of everyone will help us overcome this outbreak together.

Note: As this application reflects an approximate measure of physical distance, it should not be used as a substitute for healthcare guidelines.

You may contact the team at products@aisingapore.org.