Real-Time 3D Skeleton Reconstruction: A Comprehensive Approach from Multiple Views

Overview of the method, consisting of three main steps. The input data consists of multi-view images representing the same scene with characters for whom we want to reconstruct the 3D skeleton. (1) The first step involves extracting 2D skeletons, which need to be (2) associated with each other to create groups of skeletons corresponding to the same character seen in different views. Once we have groups of 2D skeletons belonging to the same character, we can (3), using the obtained homologous points, reconstruct the 3D skeleton through triangulation. To expedite the process, the skeleton association step can be replaced by (2') a tracking step that leverages the already labeled skeletons from the previous frame. The reconstructed 3D skeletons are then used in the images of the next frame, either in the case of (2') tracking, where they are reprojected in 2D onto each view as additional labeled skeletons, or in the case of (2) scene update, where they are used to ensure temporal coherence in skeleton labeling.