• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.448-453
• /
• 2023

대규모 언어 모델 (Large Language Model, LLM)을 인간의 선호도 관점에서 평가하는 것은 기존의 벤치마크 평가와는 다른 도전적인 과제이다. 이를 위해, 기존 연구들은 강력한 LLM을 평가자로 사용하여 접근하였지만, 높은 비용 문제가 부각되었다. 또한, 평가자로서 LLM이 사용하는 주관적인 점수 기준은 모호하여 평가 결과의 신뢰성을 저해하며, 단일 모델에 의한 평가 결과는 편향될 가능성이 있다. 본 논문에서는 엄격한 기준을 활용하여 편향되지 않은 평가를 수행할 수 있는 평가 프레임워크 및 평가자 모델 'FubaoLM'을 제안한다. 우리의 평가 프레임워크는 심층적인 평가 기준을 통해 다수의 강력한 한국어 LLM을 활용하여 연쇄적 사고(Chain-of-Thought) 기반 평가를 수행한다. 이러한 평가 결과를 다수결로 통합하여 편향되지 않은 평가 결과를 도출하며, 지시 조정 (instruction tuning)을 통해 FubaoLM은 다수의 LLM으로 부터 평가 지식을 증류받는다. 더 나아가 본 논문에서는 전문가 기반 평가 데이터셋을 구축하여 FubaoLM 효과성을 입증한다. 우리의 실험에서 앙상블된 FubaoLM은 GPT-3.5 대비 16% 에서 23% 향상된 절대 평가 성능을 가지며, 이항 평가에서 인간과 유사한 선호도 평가 결과를 도출한다. 이를 통해 FubaoLM은 비교적 적은 비용으로도 높은 신뢰성을 유지하며, 편향되지 않은 평가를 수행할 수 있음을 보인다.

• Tuberculosis and Respiratory Diseases
• /
• v.62 no.3
• /
• pp.184-191
• /
• 2007

Background: For the diagnosis of pleural tuberculosis, polymerase chain reaction (PCR) of pleural effusion specimens has shown very low sensitivity, which might be due to the small number of bacilli in the samples. The purpose of this investigation is to determine whether the sensitivity of PCR testing can be improved when increasing the amount of pleural effusion specimens. Methods: We prospectively analyzed pleural effusion specimens obtained from 53 patients for whom the exclusion of the possibility of tuberculous pleural effusion was necessary. We performed Mycobacterium tuberculosis PCR testing using the Cobas Amplicor MTB test (Roche Diagnostic Systems) with three different amounts (10ml, 25ml, and 50ml) of pleural effusion specimen in each patient. Pleural tuberculosis was defined as having one of the following: culture-positive pleural fluid sample, histopathologic finding consistent with tuberculosis on pleural biopsy, culture-positive sputum specimen, and/or positive response to anti-tuberculous medication without other possible causes of pleural effusion. Results: Of the 53 patients, 26 received the diagnosis of pleural tuberculosis. The sensitivities of AFB smearing, Mycobacterium tuberculosis culture of pleural effusion specimen, pleural biopsy, and measurement of ADA were 3.8%, 15.4%, 84.6%, and 88.5%, respectively. The results of PCR testing were positive for 3 (11.5%), 4 (15.4%), and 3 (11.5%) of the 26 patients when using 10ml, 25ml, and 50ml of pleural effusion specimens, respectively. These results did not show a statistically significant difference in the sensitivity of PCR testing when increasing the amount of pleural effusion samples (p>0.05, symmetry exact test). Conclusion: For specimens such as pleural effusion, in which the bacillary load is very low, the clinical utility of PCR testing seems highly limited with the kits designed for the diagnosis of pulmonary tuberculosis. An increased amount of pleural effusion sample does not improve the sensitivity of PCR testing.

• Laboratory Medicine Online
• /
• v.1 no.1
• /
• pp.26-34
• /
• 2011

Background: Laboratory diagnosis of new influenza A (H1N1) is crucial for managing patients and establishing control and prevention measures. We compared the diagnostic accuracies of the real time RT-PCR (rRT-PCR) test recommended for the confirmation of the new flu and the viral culture method used conventionally for viral disease with that of the rapid antigen test (RAT). Methods: We performed RAT, R-mix culture, and real-time PCR by using 861 respiratory samples collected from December 2009 to January 2010 and evaluated the abilities of these methods to detect new influenza A. The relationship among the positive rates of RAT, grades of culture, and the cycle threshold (Ct) values of rRT-PCR was also evaluated. Results: Of the 861 patients, 308 (35.8%) were diagnosed with new influenza A. The sensitivities, specificities, positive predictive values, and negative predictive values of the tests were respectively as follows: 59.7%, 99.5%, 98.4%, and 81.6% for RAT; 93.2%, 100%, 100%, and 96.3% for R-mix culture; and 95.8%, 100%, 100%, and 97.7% for rRT-PCR. Samples with weak positive grade in culture and those with Ct values of 30-37 in rRT-PCR showed positivities as low as 25.3% and 2.3% in RAT, respectively. The hospitalization rate and death rate of the confirmed patients were 3.2% and 0.3%, respectively, and gastrointestinal symptoms were observed in 7.2% of the patients. Conclusions: R-mix culture and rRT-PCR tests showed excellent reliability in the diagnosis of new influenza A and could be very useful, especially for samples with low viral load.

• Biomedical Science Letters
• /
• v.4 no.1
• /
• pp.43-56
• /
• 1998

A multiplex PCR method was designed by employing primers specific for the eaeA gene, conserved sequences of Shiga-like toxins (SLT-I.II), and the 60-MDa plasmid of enterohemorrhagic E. coli (EHEC) O157:H7 strain. A set of six synthetic oligonucleotide primers derived from sequences of the SLT-I.II, eaeA, and 60-MDa plasmid genes of E. coli O157:H7 were used in a multiplex PCR amplification procedure to detect these genes in the same enteric pathogens. In two enterohemorrhagic E. coli O157:H7 (ATCC 35150, ATCC 43894) reference strains, PCR products of 317bps (eaeA), 228bps (SLT-I.II), and 167bps (60-MDa plasmid) were successfully amplified simultaneously in a single reaction. However, the specific PCR products were not amplified in control strains of other enteric bacteria. The sensitivity of the multiplex PCR assay for detection of the SLT-I.II, eaeA, and 60-MDa plasmid genes of E. coli O157:H7 was found to be 2.5$\times$10$^{6}$ of bacteria in diarrheal stool to amplify all three bands. The multiplex PCR technology will allow large-scale screening of many clinical specimens or contaminated foods, and will be a very useful method for the detection of a wide range of microorganisms present in the environment, including EHEC O157:H7 in various types of specimens. The multiplex PCR assay has the potential to be used as a specific and rapid method for clinical diagnosis of disease caused by EHEC O157:H7.

• Korean Journal of Microbiology
• /
• v.39 no.1
• /
• pp.36-39
• /
• 2003

The rapid and reliable 16S rDNA multiplex polymerase chain reaction (PCR) assay was established to characterize bacterial etiologies of middle ear effusion. These etiologies included Haemophilus influenzae, Moraxella catarrhalis and Streptococcus pneumonia, which were detected in middle-ear effusion (MEE) samples taken from patient with otitis media. A total of 39 MEE samples were aspirated from 26 patients. DNA was extracted from MEE samples, and PCR was done with DNA extracts by using the common primers, which is localized at C4 region in the 16S rDNA gene of all bacterial species, and species-specific primers: (i) Haemophilus-specific primer, (ii) Moraxella- specific primer, and (iii) Streptococcus-specific primer. Among 39 samples tested, 24 (61.5%) were positive for H. influenzae, 10 (25.6%) were positive for M. catarrhalis, 3(7.7%) were positive for S. pneumonia, and 11 (28%) were negative for 165 rDNA multiplex PCR reaction. Nine samples (28.6%) exhibited a mixed infection and were positive for both H. infuenzae and M. catarrhalis. We suggested that 16S rDNA multiplex PCR is a useful method to identify rapidly for rapid identification of the pathogenic bacteria and characterization of bacterial etiologies of middle ear effusion.

• Journal of fish pathology
• /
• v.20 no.2
• /
• pp.109-117
• /
• 2007

This study was performed to identity non hemolytic streptococcus from cultured flounder (Paralichthys olivaceus) with Streptococcosis in the Jeju island. The result of BIOLOGTM test was Streptococcus uberis that simility of 0.5 and 98% identified in MicroLogTM system (Release 4.05). Carbohydrate utility pattern was dextrin, N-acetyl-D-glucosamine, arbutin, maltose, maltotriose, D-cellobiose, D-fructose, D-mannose, α-D-glucose, D-mannitol, β-methyl D-glucoside, salicin, sucrose, D-trehalose, pruvatic acid methyl ester, mono-methyl succinate, glycerol. In addition hemolysis test for S. parauberis and were S. iniae hemolysis in BAP (Blood agar plate). Antibiotic test for S. parauberis were Ampicillin, Amoxicillin and Fluoroquinolone sensitivity. Mutiplex PCR assay were detected S. pauberis (718 bp), S. iniae (870 bp) L. garviae (1,100 bp). Dectected S. parauberis (718 bp) were result of 16S rRNA sequence identified with S. parauberis (Gene bank accession number X89967). All isolated S. parauberis that with bouned by one group. The result were S. pauberis that γ-hemolytic chain form cocci and negative reaction of catalase, Multiplex PCR assay were 718 bp amplicon size.

• Water for future
• /
• v.29 no.6
• /
• pp.179-188
• /
• 1996

The chronical sequences of daily precipitation are of great practical importance in the planning and operational processes of water resources system. A sequence of days with alternate dry day and wet day can be generated by two state Markov chain model that establish the subsequent daily state as wet or dry by previously calculated vconditional probabilities depending on the state of previous day. In this study, a synthetic generation model for obtaining the daily precipitation series is presented by classifying the precipitation amount in wet days into multi-states. To apply multi-state Markov chain model, the daily precipitation amounts for wet day are rearranged by grouping into thirty states with intervals for each state. Conditional probabilities as transition probability matrix are estimated from the computational scheme for stepping from the precipitation on one day to that on the following day. Statistical comparisons were made between the historical and synthesized chracteristics of daily precipitation series. From the results, it is shown that the proposed method is available to generate and simulate the daily precipitation series with fair accuracy and conserve the general statistical properties of historical precipitation series.

• Investigative Magnetic Resonance Imaging
• /
• v.4 no.1
• /
• pp.7-13
• /
• 2000

Purpose : To assess the distortion of MRI with the Leksell stereotactic radiosurgery system in variable pulse sequence and imaging plane through phantom study, to find most adequate imaging plane and pulse sequence for stereotactic radiosurgery system. Materials and methods : We made the phantoms for MRI and get images in variable conditions and analyzed the image distortion using image analysis program, and statistically using paired student t-test. Results : The transeverse plane images had acceptable error ranges bless than 1.5mm) in all pulse sequence in both the analysis of fiducial marker in stereotactic G-frame and the phantom study. The coronal plane images had unacceptable large errors (more than 1.7mm) in the analysis of fiducial marker in the stereotactic G-frame, but had corrected small errors (less than 1.5mm) in the phantom study. Conclusion : We find from the phantom study that the present MR machines are adequate for stereotactic surgery system in frequently used pulse sequences, and imaging planes.

• Journal of Genetic Medicine
• /
• v.6 no.2
• /
• pp.146-154
• /
• 2009

Multiplex ligation dependent probe amplification (MLPA) is a PCR-based method to detect gene dosage. Since its introduction, MLPA has been used to test a large number of genes for major deletions or duplications. Genetic testing, as a diagnostic tool for genetic disease, has been used primarily to identify point mutations, including base substitutions and small insertions/deletions, using PCR and sequence analysis. However, it is difficult to identify large deletions or duplications using routine PCR- gel based assays, especially in heterozygotes. The MLPA is a more feasible method for identification of gene dosage than another routine PCR-based methods, and better able to detect deleterious deletions or duplications. In addition to detection of gene dosage, MLPA can be applied to identify methylation patterns of target genes, aneuploidy during prenatal diagnoses, and large deletions or duplications that may be associated with various cancers. The MLPA method offers numerous advantages, as it requires only a small amount of template DNA, is applicable to a wide variety of applications, and is high-throughput. On the other hand, this method suffers from disadvantages including the possibility of false positive results affected by template DNA quality, difficulties identifying SNPs located in probe sequences, and analytical complications in quantitative aspects.

• Pediatric Infection and Vaccine
• /
• v.10 no.2
• /
• pp.178-185
• /
• 2003

Purpose : Group A streptococci(GAS) was isolated from the patients with acute pharyngitis. Epidemiological studies using T typing and emm genotyping was performed for GAS and compared with the results of the carriers. Methods : Throat cultures were taken from 246 children(123 boys, 123 girls) from November, 2001 to May, 2002 who visited a pediatrician's office located in Jinju, Gyeongnam province. T types were identified with slide agglutination and emm genotypes were identified with DNA sequencing after amplification of emm genes. Results : One hundred thirty(52.8%) out of 246 children yielded beta-hemolytic streptococci, of which 96.1% were group A. Children from 4 to 7 years old comprised 70.4% of the GAS positive group. T12 were the most common(35.2%) and T non-typeable strains were the next(30.4%). emm12 was most frequent(28.5%), and emm75(18.7%), emm22(13.0%), emm2(12.2%), and emm8(8.1%) were relatively common. Conclusion : Since GAS is so highly prevalent in acute pharyngitis, indeed being half of the population, good clinical practice dictates the systematic employment of throat culture for acute pharyngitis before prescribing antibiotics in a pediatric setting. The distribution of the T antigens and emm genes showed similar pattern between the acute pharyngitis and the carriers.