Video Analysis System for Action and Emotion Detection by Object with Hierarchical Clustering based Re-ID (계층적 군집화 기반 Re-ID를 활용한 객체별 행동 및 표정 검출용 영상 분석 시스템)
-
- Journal of Intelligence and Information Systems
- /
- v.28 no.1
- /
- pp.89-106
- /
- 2022
Recently, the amount of video data collected from smartphones, CCTVs, black boxes, and high-definition cameras has increased rapidly. According to the increasing video data, the requirements for analysis and utilization are increasing. Due to the lack of skilled manpower to analyze videos in many industries, machine learning and artificial intelligence are actively used to assist manpower. In this situation, the demand for various computer vision technologies such as object detection and tracking, action detection, emotion detection, and Re-ID also increased rapidly. However, the object detection and tracking technology has many difficulties that degrade performance, such as re-appearance after the object's departure from the video recording location, and occlusion. Accordingly, action and emotion detection models based on object detection and tracking models also have difficulties in extracting data for each object. In addition, deep learning architectures consist of various models suffer from performance degradation due to bottlenects and lack of optimization. In this study, we propose an video analysis system consists of YOLOv5 based DeepSORT object tracking model, SlowFast based action recognition model, Torchreid based Re-ID model, and AWS Rekognition which is emotion recognition service. Proposed model uses single-linkage hierarchical clustering based Re-ID and some processing method which maximize hardware throughput. It has higher accuracy than the performance of the re-identification model using simple metrics, near real-time processing performance, and prevents tracking failure due to object departure and re-emergence, occlusion, etc. By continuously linking the action and facial emotion detection results of each object to the same object, it is possible to efficiently analyze videos. The re-identification model extracts a feature vector from the bounding box of object image detected by the object tracking model for each frame, and applies the single-linkage hierarchical clustering from the past frame using the extracted feature vectors to identify the same object that failed to track. Through the above process, it is possible to re-track the same object that has failed to tracking in the case of re-appearance or occlusion after leaving the video location. As a result, action and facial emotion detection results of the newly recognized object due to the tracking fails can be linked to those of the object that appeared in the past. On the other hand, as a way to improve processing performance, we introduce Bounding Box Queue by Object and Feature Queue method that can reduce RAM memory requirements while maximizing GPU memory throughput. Also we introduce the IoF(Intersection over Face) algorithm that allows facial emotion recognized through AWS Rekognition to be linked with object tracking information. The academic significance of this study is that the two-stage re-identification model can have real-time performance even in a high-cost environment that performs action and facial emotion detection according to processing techniques without reducing the accuracy by using simple metrics to achieve real-time performance. The practical implication of this study is that in various industrial fields that require action and facial emotion detection but have many difficulties due to the fails in object tracking can analyze videos effectively through proposed model. Proposed model which has high accuracy of retrace and processing performance can be used in various fields such as intelligent monitoring, observation services and behavioral or psychological analysis services where the integration of tracking information and extracted metadata creates greate industrial and business value. In the future, in order to measure the object tracking performance more precisely, there is a need to conduct an experiment using the MOT Challenge dataset, which is data used by many international conferences. We will investigate the problem that the IoF algorithm cannot solve to develop an additional complementary algorithm. In addition, we plan to conduct additional research to apply this model to various fields' dataset related to intelligent video analysis.
Buyongjeong, a pavilion in the Rear Garden of Changdeok Palace, was appointed as Treasure No. 1763 on March 2, 2012, by the South Korea government since it shows significant symmetry and proportion on its unique planar shape, spatial configuration, building decoration, and so forth. However, the designation of Treasure selection was mainly evaluated by concrete science, in that the selection has not clearly articulated how and why Buoungjeong was constructed as a present unique form. Therefore, this study aims to clarify the identity of Buyongjeong at the time of construction by considering its historical, ideological, philosophical background and building intention. Summary are as follows: First, Construction backgrounds and characters of Buyongjeong: Right after the enthronement, King Jeongjo had founded Kyujanggak(奎章閣), and sponsored civil ministers who were elected by the national examination, as a part of political reform. In addition, he established his own political system by respecting "Kaksin(閣臣)", Kyujanggak's officials as much as "Kain(家人)", internal family members. King Jeongjo's aggressive political reform finally enabled King's lieges to visit King's Rear Garden. In the reign of King Jeongjo's 16th year(1792), Naekaksangjohoe(內閣賞釣會) based on "Kaksin" was officially launched and the Rear Garden visitation became a regular meeting. The Rear Garden visitation consisted of "Sanghwajoeoyeon(賞花釣魚宴)" - enjoying flowers and fishing, and activities of "Nanjeongsugye". Afterward, it eventually became a huge national event since high rank government officials participated the event. King Jeongjo shared the cultural activities with government officials together to Buyongjeong as a place to fulfill his royal politics. Second, The geographical location and spatial characteristics of Buyongjeong: On the enthronement of King Jeongjo(1776), he renovated Taeksujae. Above all, aligning and linking Gaeyuwa - Taeksujae - a cicular island - Eosumun - Kyujangkak along with the construction axis is an evidence for King Jeongjo to determine how the current Kyujangkak zone was prepared and designed to fulfill King Jeonjo's political ideals. In 17th year(1793) of the reign of King Jeongjo, Taeksujae, originally a square shaped pavilion, was modified and expanded with ranks to provide a place to get along with the King and officials. The northern part of Buyongjeong, placed on pond, was designed for the King's place and constructed one rank higher than others. Discernment on windows and doors were made with "Ajasal" - a special pattern for the King. The western and eastern parts were for government officials. The center part was prepared for a place where government officials were granted an audience with the King, who was located in the nortern part of Buyongjeong. Government officials from the western and eastern parts of Buyongjeong, could enter the central part of the Buyongjeong from the southern part by detouring the corner of Buyongjeong. After all, Buyongjeong is a specially designed garden building, which was constructed to be a royal palace utilizing its minimal space. Third, Cultural Values of Buyongjeong: The Buyongjeong area exhibits a trait that it had been continuously developed and it had reflected complex King's private garden cultures from King Sejo, Injo, Hyunjong, Sukjong, Jeongjo and so forth. In particular, King Jeongjo had succeded physical, social and imaginary environments established by former kings and invited their government officials for his royal politics. As a central place for his royal politics, King Jeongjo completed Buyongjeong. Therefore, the value of Buyongjeong, as a garden building reflecting permanency of the Joseon Dynasty, can be highly evaluated. In addition, as it reflects Confucianism in the pavilion - represented by distinguishing hierarchical ranks, it is a unique example to exhibit its distinctiveness in a royal garden.
The World Wide Web is a very large distributed digital information space. From its origins in 1991, the web has grown to encompass diverse information resources as personal home pasges, online digital libraries and virtual museums. Some estimates suggest that the web currently includes over 500 billion pages in the deep web. The ability to search and retrieve information from the web efficiently and effectively is an enabling technology for realizing its full potential. With powerful workstations and parallel processing technology, efficiency is not a bottleneck. In fact, some existing search tools sift through gigabyte.syze precompiled web indexes in a fraction of a second. But retrieval effectiveness is a different matter. Current search tools retrieve too many documents, of which only a small fraction are relevant to the user query. Furthermore, the most relevant documents do not nessarily appear at the top of the query output order. Also, current search tools can not retrieve the documents related with retrieved document from gigantic amount of documents. The most important problem for lots of current searching systems is to increase the quality of search. It means to provide related documents or decrease the number of unrelated documents as low as possible in the results of search. For this problem, CiteSeer proposed the ACI (Autonomous Citation Indexing) of the articles on the World Wide Web. A "citation index" indexes the links between articles that researchers make when they cite other articles. Citation indexes are very useful for a number of purposes, including literature search and analysis of the academic literature. For details of this work, references contained in academic articles are used to give credit to previous work in the literature and provide a link between the "citing" and "cited" articles. A citation index indexes the citations that an article makes, linking the articleswith the cited works. Citation indexes were originally designed mainly for information retrieval. The citation links allow navigating the literature in unique ways. Papers can be located independent of language, and words in thetitle, keywords or document. A citation index allows navigation backward in time (the list of cited articles) and forwardin time (which subsequent articles cite the current article?) But CiteSeer can not indexes the links between articles that researchers doesn't make. Because it indexes the links between articles that only researchers make when they cite other articles. Also, CiteSeer is not easy to scalability. Because CiteSeer can not indexes the links between articles that researchers doesn't make. All these problems make us orient for designing more effective search system. This paper shows a method that extracts subject and predicate per each sentence in documents. A document will be changed into the tabular form that extracted predicate checked value of possible subject and object. We make a hierarchical graph of a document using the table and then integrate graphs of documents. The graph of entire documents calculates the area of document as compared with integrated documents. We mark relation among the documents as compared with the area of documents. Also it proposes a method for structural integration of documents that retrieves documents from the graph. It makes that the user can find information easier. We compared the performance of the proposed approaches with lucene search engine using the formulas for ranking. As a result, the F.measure is about 60% and it is better as about 15%.
This study aims to explore the direction for Korea's effective response to Target 3 (30by30), which can be said to be the core of the Kunming-Montreal Global Biodiversity Framework (K-M GBF) of the Convention on Biological Diversity (CBD), to find the direction of systematic OECM (Other Effective area-based Conservation Measures) discovery at the national level through a survey of global conceptual review and expert perception of OECM. This study examined ① the use of Korean terms related to OECM, ② derivation of determining criteria reflecting global standards, ③ deriving types of potential OECM candidates in Korea, and ④ considerations for OECM identification and reporting to explore the direction for identifying systematic, national-level OECM that complies with global standards and reflects the Korean context. First, there was consensus for using Korean terminology that reflects the concept of OECM rather than simple translations, and it was determined that "nature coexistence area" was the most preferred term (12 people) and had the same context as CBD 2050 Vision of "a world of living in harmony with nature." This study suggests utilizing four criteria (1. No protected areas, 2. Geographic boundaries, 3. Governance/management, and 4. Biodiversity value) that reflect OECM's core characteristics in the first-stage selection process, carrying out the consensus-building process (stage 2) with the relevant agencies, and adding two criteria (3-1 Effectiveness and sustainability of governance and management and 4-1 Long-term conservation) and performing the in-depth diagnosis in stage 3 (full assessment for reporting). The 28 types examined in this study were generally compatible with OECMs (4.45-6.21/7 points, mean 5.24). In particular, the "Conservation Properties (6.21 points)" and "Conservation Agreements (6.07 points)", which are controlled by National Nature Trust, are shown to be the most in line with the OECM concept. They were followed by "Buffer zone of World Natural Heritage (5.77 points)", "Temple Forest (5.73 points)", "Green-belt (Restricted development zones, 5.63 points)", "DMZ (5.60 points)", and "Buffer zone of biosphere reserve (5.50 point)" to have high potential. In the case of "Uninhabited Islands under Absolute Conservation", the response that they conformed to the protected areas (5.83/7 points) was higher than the OECM compatibility (5.52/7 points), it is determined that in the future, it would be preferable to promote the listing of absolute unprotected islands in the Korea Database on Protected Areas (KDPA) along with their surrounding waters (1 km). Based on the results of a global OECM standard review and expert perception survey, 10 items were suggested as considerations when identifying OECM in the Korean context. In the future, continuous research is needed to identify the potential OECMs through site-level assessment regarding these considerations and establish an effective in-situ conservation system at the national level by linking existing protected area systems and identified OECMs.
The wall shear stress in the vicinity of end-to end anastomoses under steady flow conditions was measured using a flush-mounted hot-film anemometer(FMHFA) probe. The experimental measurements were in good agreement with numerical results except in flow with low Reynolds numbers. The wall shear stress increased proximal to the anastomosis in flow from the Penrose tubing (simulating an artery) to the PTFE: graft. In flow from the PTFE graft to the Penrose tubing, low wall shear stress was observed distal to the anastomosis. Abnormal distributions of wall shear stress in the vicinity of the anastomosis, resulting from the compliance mismatch between the graft and the host artery, might be an important factor of ANFH formation and the graft failure. The present study suggests a correlation between regions of the low wall shear stress and the development of anastomotic neointimal fibrous hyperplasia(ANPH) in end-to-end anastomoses. 30523 T00401030523 ^x Air pressure decay(APD) rate and ultrafiltration rate(UFR) tests were performed on new and saline rinsed dialyzers as well as those roused in patients several times. C-DAK 4000 (Cordis Dow) and CF IS-11 (Baxter Travenol) reused dialyzers obtained from the dialysis clinic were used in the present study. The new dialyzers exhibited a relatively flat APD, whereas saline rinsed and reused dialyzers showed considerable amount of decay. C-DAH dialyzers had a larger APD(11.70
The wall shear stress in the vicinity of end-to end anastomoses under steady flow conditions was measured using a flush-mounted hot-film anemometer(FMHFA) probe. The experimental measurements were in good agreement with numerical results except in flow with low Reynolds numbers. The wall shear stress increased proximal to the anastomosis in flow from the Penrose tubing (simulating an artery) to the PTFE: graft. In flow from the PTFE graft to the Penrose tubing, low wall shear stress was observed distal to the anastomosis. Abnormal distributions of wall shear stress in the vicinity of the anastomosis, resulting from the compliance mismatch between the graft and the host artery, might be an important factor of ANFH formation and the graft failure. The present study suggests a correlation between regions of the low wall shear stress and the development of anastomotic neointimal fibrous hyperplasia(ANPH) in end-to-end anastomoses. 30523 T00401030523 ^x Air pressure decay(APD) rate and ultrafiltration rate(UFR) tests were performed on new and saline rinsed dialyzers as well as those roused in patients several times. C-DAK 4000 (Cordis Dow) and CF IS-11 (Baxter Travenol) reused dialyzers obtained from the dialysis clinic were used in the present study. The new dialyzers exhibited a relatively flat APD, whereas saline rinsed and reused dialyzers showed considerable amount of decay. C-DAH dialyzers had a larger APD(11.70