Browse > Article
http://dx.doi.org/10.3745/KTCCS.2022.11.10.373

Deep Learning Based Group Synchronization for Networked Immersive Interactions  

Lee, Joong-Jae ((재)실감교류인체감응솔루션연구단)
Publication Information
KIPS Transactions on Computer and Communication Systems / v.11, no.10, 2022 , pp. 373-380 More about this Journal
Abstract
This paper presents a deep learning based group synchronization that supports networked immersive interactions between remote users. The goal of group synchronization is to enable all participants to synchronously interact with others for increasing user presence Most previous methods focus on NTP-based clock synchronization to enhance time accuracy. Moving average filters are used to control media playout time on the synchronization server. As an example, the exponentially weighted moving average(EWMA) would be able to track and estimate accurate playout time if the changes in input data are not significant. However it needs more time to be stable for any given change over time due to codec and system loads or fluctuations in network status. To tackle this problem, this work proposes the Deep Group Synchronization(DeepGroupSync), a group synchronization based on deep learning that models important features from the data. This model consists of two Gated Recurrent Unit(GRU) layers and one fully-connected layer, which predicts an optimal playout time by utilizing the sequential playout delays. The experiments are conducted with an existing method that uses the EWMA and the proposed method that uses the DeepGroupSync. The results show that the proposed method are more robust against unpredictable or rapid network condition changes than the existing method.
Keywords
Group Synchronization; Deep Learning; Immersive Interaction;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 K. B. Lee and J. Eldson, "Standard for a precision clock synchronization protocol for networked measurement and control systems," IEEE Standard 1588, 2004.
2 Google [Internet], https://developers.google.com/glass/, visited in October 2022.
3 J. D. N. Dionisio, W. G. Burns III, and R. Gilbert., "3D virtual worlds and the metaverse: Current status and future possibilities," ACM Computing Surveys (CSUR), Vol.45, No.3, 2013, pp.1-38.
4 D. L. Mills, "Network time protocol version 4 reference and implementation guide," Electrical and Computer Engineering Technical Report 06-06-1, 2006.
5 Z. Huang, K. Nahrstedt, and R. Steinmetz, "Evolution of temporal multimedia synchronization principles: A historical viewpoint," ACM Transactions on Multimedia Computing Communcations and Applications, Vol.9, Nol.1s, pp.1-23, 2013.
6 SteamVR [Internet], https://steamcommunity.com/steamvr, visited in October 2022.
7 Microsoft [Internet], https://www.microsoft.com/en-us/mesh visited in October 2002.
8 R. Steinmetz and M. Thomas, "Multimedia synchronization techniques: Experiences based on different system structures," ACM SIGCOMM Computer Communication Review, Vol.22, No.3, pp.90-91, 1992.   DOI
9 P. J. Huber, "Robust estimation of a location parameter," Breakthroughs in statistics (pp.492-518), Springer, New York, NY, 1992.
10 S. N. Givigi, K. M. Cabral, and P. T. Jardine, "Group synchronization in coordination tasks via network control methods," IFAC-PapersOnLine, Vol.53, No.2, pp. 10182-10187, 2020.   DOI
11 Meta [Internet], https://www.meta.com/en/, visited in October 2022.
12 H. M. M. M. Lwin, Y. Ishibashi, and K.T. Mya, "Human perception of group synchronization error in remote learning dependencies of voice and video contents in one-way communication," International Journal of Communications, Network and System Sciences, Vol.15, No.3, pp.31-42, 2022.   DOI
13 T. Y. Lee, E. Lee, J. Surh, J. J. Lee, and B. J. You, "Balanced clock skew compensation for immersive networked interactions based on inter media synchronization levels," In 2018 10th Computer Science and Electronic Engineering (CEEC), pp.119-124, 2018.
14 P. Cisar, S. Bosnjak, and S. M. Cisar, "EWMA algorithm in network practice," International Journal of Computers Communications & Control, Vol.5, No.2, pp.160-170, 2010.   DOI
15 K. Cho, B. Van Merrienboer, D. Bahdanau, and Y. Bengio, "On the properties of neural machine translation: Encoder-decoder approaches," arXiv preprint arXiv:1409.1259, 2014.