• Journal of Internet Computing and Services
• /
• v.21 no.1
• /
• pp.33-43
• /
• 2020

Retrieving a 3D model from a 3D database and augmenting the retrieved model in the Augmented Reality system simultaneously became an issue in developing the plausible AR environments in a convenient fashion. It is considered that the sketch-based 3D object retrieval is an intuitive way for searching 3D objects based on human-drawn sketches as query. In this paper, we propose a novel deep learning based approach of retrieving a sketch-based 3D object as for an Augmented Reality Model. For this work, we introduce a new method which uses Sketch CNN, Wasserstein CNN and Wasserstein center loss for retrieving a sketch-based 3D object. Especially, Wasserstein center loss is used for learning the center of each object category and reducing the Wasserstein distance between center and features of the same category. The proposed 3D object retrieval and augmentation consist of three major steps as follows. Firstly, Wasserstein CNN extracts 2D images taken from various directions of 3D object using CNN, and extracts features of 3D data by computing the Wasserstein barycenters of features of each image. Secondly, the features of the sketch are extracted using a separate Sketch CNN. Finally, we adopt sketch-based object matching method to localize the natural marker of the images to register a 3D virtual object in AR system. Using the detected marker, the retrieved 3D virtual object is augmented in AR system automatically. By the experiments, we prove that the proposed method is efficiency for retrieving and augmenting objects.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.443-447
• /
• 2023

검색 증강형 언어 모델은 입력과 연관된 문서들을 검색하고 텍스트 생성 과정에 통합하여 언어 모델의 생성 능력을 강화한다. 본 논문에서는 사전 학습된 대규모 언어 모델의 추가적인 학습 없이 In-Context 검색 증강으로 한국어 언어 모델의 생성 능력을 강화하고 기존 언어 모델 대비 성능이 증가함을 보인다. 특히 다양한 크기의 사전 학습된 언어 모델을 활용하여 검색 증강 결과를 보여 모든 규모의 사전 학습 모델에서 Perplexity가 크게 개선된 결과를 확인하였다. 또한 오픈 도메인 질의응답(Open-Domain Question Answering) 과업에서도 EM-19, F1-27.8 향상된 결과를 보여 In-Context 검색 증강형 언어 모델의 성능을 입증한다.

• Journal of Korea Game Society
• /
• v.11 no.3
• /
• pp.63-72
• /
• 2011

In this paper, we propose a novel database creation and retrieval method of feature descriptors to support real-time marker-less tracking in the augmented reality environments. Each feature descriptor is encoded by integer and multi-level database is created in order to retrieve a feature descriptor efficiently. The retrieval of a feature descriptor is performed as follows: Firstly, candidate feature descriptors are searched by traversing the multi-level database. Secondly, the euclidean distance between input feature descriptor and each candidate one is compared. The shortest one is retrieved. The proposed method is 16 ms faster than previous KD-Tree method for each feature descriptor.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.7
• /
• pp.2407-2424
• /
• 2022

Recently, Transformer has made great progress in video retrieval tasks due to its high representation capability. For the structure of a Transformer, the cascaded self-attention modules are capable of capturing long-distance feature dependencies. However, the local feature details are likely to have deteriorated. In addition, increasing the depth of the structure is likely to produce learning bias in the learned features. In this paper, an improved Transformer structure named TransDCS (Transformer with Dynamic Convolution and Shortcut) is proposed. A Multi-head Conv-Self-Attention module is introduced to model the local dependencies and improve the efficiency of local features extraction. Meanwhile, the augmented shortcuts module based on a dual identity matrix is applied to enhance the conduction of input features, and mitigate the learning bias. The proposed model is tested on MSRVTT, LSMDC and Activity-Net benchmarks, and it surpasses all previous solutions for the video-text retrieval task. For example, on the LSMDC benchmark, a gain of about 2.3% MdR and 6.1% MnR is obtained over recently proposed multimodal-based methods.

• Annual Conference on Human and Language Technology
• /
• 2022.10a
• /
• pp.105-108
• /
• 2022

오픈 도메인 질의 응답은 사전학습 언어모델의 파라미터에 저장되는 정보만을 사용하여 답하는 질의 응답 방식과 달리 대량의 문서 등에서 질의에 대한 정답을 찾는 문제이다. 최근 등장한 Dense Retrieval은 BERT 등의 모델을 사용해 질의와 문서들의 벡터 연산으로 질의와 문서간의 유사도를 판별하여 문서를 검색한다. 이러한 Dense Retrieval을 활용하는 방안 중 RAG는 Dense Retrieval을 이용한 외부 지식과 인코더-디코더 모델에 내재된 지식을 결합하여 성능을 향상시킨다. 본 논문에서는 RAG를 한국어 오픈 도메인 질의 응답 데이터에 적용하여 베이스라인에 비해 일부 향상된 성능을 보임을 확인하였다.

• Annual Conference on Human and Language Technology
• /
• 2022.10a
• /
• pp.141-145
• /
• 2022

다중 문서 기반 대화 시스템에서 응답 시스템은 올바른 답변을 생성하기 위해서 여러 개의 문서 중 질문과 가장 관련 있는 문서를 검색하는 것부터 시작해야 한다. DialDoc 2022 Shared Task[1]를 비롯한 최근의 연구들은 대화 시스템의 문서 검색 과정을 위해 Dense Passage Retrieval(DPR)[2] 모델을 사용하고 있으며 검색기의 성능 개선을 위해 Re-ranking과 Hard negative sampling 같은 방법들이 연구되고 있다. 본 논문에서는 문서에 기반하는 대화 데이터의 양이 적거나 제한될 경우, 주어진 데이터를 효율적으로 활용해 보고자 검색기를 생성 모델을 이용하여 문서의 엔티티를 기반으로 질문을 생성하고 기존 데이터에 증강하는 방법을 제시했으며 실험의 결과로 MRR metric의 경우 0.96 ~ 1.56의 성능 향상을, R@1 metric의 경우 1.2 ~ 1.57의 성능 향상을 확인하였다.

• Journal of Information Technology Services
• /
• v.10 no.4
• /
• pp.269-280
• /
• 2011

Mobile Augmented Reality (mobile AR), as one of the most prospective mobile applications, intends to provide richer experiences by annotating tags or virtual objects over the scene observed through camera embedded in a handheld device like smartphone or pad. In this paper, we analyzed the current status of the art of mobile AR and proposed a novel Points of Interest (POIs) data model based on ontology to provide context-aware information retrievals on lots of POIs data. Proposed ontology was expanded from the standard POIs data model of W3C POIs Working Group and established using OWL (Web Ontology Language) and Protege. We also proposed a context-aware mobile AR platform which can resolve three distinguished issues in current platforms : interoperability problem of POI tags, POIs data retrieval issue, and context-aware service issue.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.24 no.2
• /
• pp.87-92
• /
• 2024

In this paper, a RAG system was implemented using an existing Large Language Model (LLM) and Langchain library to develop a dental consultation chatbot. For this purpose, we collected contents from the webpage bulletin boards of domestic dental university hospitals and constructed consultation data with the advice and supervision of dental specialists. In order to divide the input consultation data into appropriate sizes, the chunk size and the size of the overlapping text in each chunk were set to 1001 and 100, respectively. As a result of the simulation, the Retrieval Augmented LLM searched for and output the consultation content that was most similar to the user input. It was confirmed that the accessibility of dental consultation and the accuracy of consultation content could be improved through the built chatbot.

• Spatial Information Research
• /
• v.20 no.2
• /
• pp.59-70
• /
• 2012

Advances in science and technology, especially the increasing popularity of smart phone has opened up infinite possibilities for the information retrieval. And the augmented reality technology which has started getting attention to various fields has allowed us to get an information in an innovative way by providing additional information to a real world before our eyes. The purpose of this study is to suggest and to construct mobile application service which give history-tourism information efficiently based on augmented reality technology. As a result of study, we suggest that it is needed to find the view point that the most accurate information can be provided to the users according to each contents such as maps, pictures, drawings etc. and to provide the function to the users which shows the view points information efficiently. The purpose of this study is to investigate the possibility whether the technology studied in private sector by using augmented reality based on smart phone can also be applied in public sector for the service of history tourism information. We expect that this study can be the basis for providing the more advanced augmented reality service in the near future.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2020.01a
• /
• pp.167-168
• /
• 2020

본 논문에서 제안하는 모바일 어플리케이션은 증강현실 기술을 적용한 정보공유 플랫폼으로 일상 혹은 특별한 여정에서 찍은 사진에 대한 확장 정보를 제공한다. 이 정보들을 통해 자신의 위치를 기준으로 지역을 구체적으로 알아볼 수 있으며 개인의 지역명소를 찾아 공유한다. 누구나 친숙하게 활용할 수 있도록 개발된 모바일 어플리케이션은 지역경제 및 전통시장 활성화에 대한 솔루션으로 제공이 가능하다.