• Journal of Information Processing Systems
• /
• v.15 no.5
• /
• pp.1156-1170
• /
• 2019

In recent years, the problem of data drifted of the smart grid due to manual operation has been widely studied by researchers in the related domain areas. It has become an important research topic to effectively and reliably find the reasonable data needed in the Supervisory Control and Data Acquisition (SCADA) system has become an important research topic. This paper analyzes the data composition of the smart grid, and explains the power model in two smart grid applications, followed by an analysis on the application of each parameter in density-based spatial clustering of applications with noise (DBSCAN) algorithm. Then a comparison is carried out for the processing effects of the boxplot method, probability weight analysis method and DBSCAN clustering algorithm on the big data driven power grid. According to the comparison results, the performance of the DBSCAN algorithm outperforming other methods in processing effect. The experimental verification shows that the DBSCAN clustering algorithm can effectively screen the power grid data, thereby significantly improving the accuracy and reliability of the calculation result of the main grid's theoretical line loss.

• Journal of Korean Society of Coastal and Ocean Engineers
• /
• v.26 no.4
• /
• pp.207-216
• /
• 2014

Total organic carbon (TOC) is an important indicator used as an direct biological index in the research field of the marine carbon cycle. It is possible to produce the sufficient TOC estimation data by using the Chemical Oxygen Demand(COD) data because the available TOC data is relatively poor than the COD data. The outlier detection and treatment (removal) should be carried out reasonably and objectively because the equation for a COD-TOC conversion is directly affected the TOC estimation. In this study, it aims to suggest the optimal regression model using the available salinity, COD, and TOC data observed in the Korean coastal zone. The optimal regression model is selected by the comparison and analysis on the changes of data numbers before and after removal, variation coefficients and root mean square (RMS) error of the diverse detection methods of the outlier and influential observations. According to research result, it is shown that a diagnostic case combining SIQR (Semi - Inter-Quartile Range) boxplot and Cook's distance method is most suitable for the outlier detection. The optimal regression function is estimated as the TOC(mg/L) = $0.44{\cdot}COD(mg/L)+1.53$, then determination coefficient is showed a value of 0.47 and RMS error is 0.85 mg/L. The RMS error and the variation coefficients of the leverage values are greatly reduced to the 31% and 80% of the value before the outlier removal condition. The method suggested in this study can provide more appropriate regression curve because the excessive impacts of the outlier frequently included in the COD and TOC monitoring data is removed.

• Journal of Information Technology Services
• /
• v.15 no.1
• /
• pp.167-178
• /
• 2016

Spectrum sensing in cognitive radio (CR) has a great role in order to utilize idle spectrum opportunistically, since it is responsible for making available dynamic spectrum access efficiently. In this research area, collaboration among multiple cognitive radio users has been proposed for the betterment of detection reliability. Even though cooperation among them improves the spectrum sensing performance, some falsely reporting malicious users may degrade the performance rigorously. In this article, we have studied the detection and nullifying the harmful effects of such malicious users by applying some well known outlier detection methods based on Grubb's test, Boxplot method and Dixon's test in cooperative spectrum sensing. Initially, the performance of each technique is compared and found that Boxplot method outperforms both Grubb's and Dixon's test for the case where multiple malicious users are present. Secondly, a new algorithm based on reputation and weight is developed to identify malicious users and cancel out their negative impact in final decision making. Simulation results demonstrate that the proposed scheme effectively identifies the malicious users and suppress their harmful effects at the fusion center to decide whether the spectrum is idle.

• KSCE Journal of Civil and Environmental Engineering Research
• /
• v.34 no.5
• /
• pp.1455-1461
• /
• 2014

In this study, using flux data measured in Deokgokje reservoir watershed near Deokyu mountain in May, June, and July 2011, statistical analysis was conducted for outlier detection and replacement for vertical wind speed in the measurement of evapotranspiration based on eddy covariance method. To statistically analyze the outliers of vertical wind speed, the outlier detection method based on interquartile range (IQR) in boxplot was employed and the detected outliers were deleted or replaced with mean. The comparison was conducted for the measured evapotranspiration before and after the outlier replacement. The study results showed that there is a difference between evapotranspiration before outlier replacement and evapotranspiration after outlier replacement, especially during the rainy day. Therefore, based on the study results, the outliers should be deleted or replaced in the measurement of evapotranspiration.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.29 no.2
• /
• pp.113-121
• /
• 2011

In this paper, we proposed the method automatically to match areal feature based on similarity using spatial information. For this, we extracted candidate matching pairs intersected between two different spatial datasets, and then measured a shape similarity, which is calculated by an weight sum method of each matching criterion automatically derived from CRITIC method. In this time, matching pairs were selected when similarity is more than a threshold determined by outliers detection of adjusted boxplot from training data. After applying this method to two distinct spatial datasets: a digital topographic map and street-name address base map, we conformed that buildings were matched, that shape is similar and a large area is overlaid in visual evaluation, and F-Measure is highly 0.932 in statistical evaluation.

• Journal of Positioning, Navigation, and Timing
• /
• v.9 no.4
• /
• pp.397-403
• /
• 2020

In order to synchronize a remote system time to the reference time like Coordinated Universal Time (UTC), it is required to compare the time difference between the two clocks. The time comparison data may have some outliers and the time synchronization performance can be significantly degraded if the outliers are not removed. Therefore, it is required to employ an effective outlier detection algorithm for keeping high accurate system time. In this paper, an outlier detection method is presented for the time difference data of GNSS time transfer receivers. The time difference data between the system time and the GNSS usually have slopes because the remote system clock is under free running until synchronized to the reference clock time. For investigating the outlier detection performance of the proposed algorithm, simulations are performed by using the time difference data of a GNSS time transfer receiver corrected to a free running Cesium clock with intentionally inserted outliers. From the simulation, it is investigated that the proposed algorithm can effectively detect the inserted outliers while conventional methods such as modified Z-score and adjusted boxplot cannot. Furthermore, it is also observed that the synchronization performance can be degraded to more than 15% with 20 outliers compared to that of original data without outliers.

• Structural Engineering and Mechanics
• /
• v.87 no.3
• /
• pp.221-229
• /
• 2023

Urbanization and industrialization have significantly increased the amount of solid waste produced in recent decades, posing considerable disposal problems and environmental burdens. The practice of waste utilization in concrete has gained popularity among construction practitioners and researchers for the efficient use of resources and the transition to the circular economy in construction. This study employed Lytag aggregate, an environmentally friendly pulverized fuel ash-based lightweight aggregate, as a substitute for natural coarse aggregate. At the same time, fly ash, an industrial by-product, was used as a partial substitute for cement. Concrete mix M20 was experimented with using fly ash and Lytag lightweight aggregate. The percentages of fly ash that make up the replacements were 5%, 10%, 15%, 20%, and 25%. The Compressive Strength (CS), Split Tensile Strength (STS), and deflection were discovered at these percentages after 56 days of testing. The concrete cube, cylinder, and beam specimens were examined in the explorations, as mentioned earlier. The results indicate that a 10% substitution of cement with fly ash and a replacement of coarse aggregate with Lytag lightweight aggregate produced concrete that performed well in terms of mechanical properties and deflection. The cementitious composites have varying characteristics as the environment changes. Therefore, understanding their mechanical properties are crucial for safety reasons. CS, STS, and deflection are the essential property of concrete. Machine learning (ML) approaches have been necessary to predict the CS of concrete. The Artificial Fish Swarm Optimization (AFSO), Particle Swarm Optimization (PSO), and Harmony Search (HS) algorithms were investigated for the prediction of outcomes. This work deftly explains the tremendous AFSO technique, which achieves the precise ideal values of the weights in the model to crown the mathematical modeling technique. This has been proved by the minimum, maximum, and sample median, and the first and third quartiles were used as the basis for a boxplot through the standardized method of showing the dataset. It graphically displays the quantitative value distribution of a field. The correlation matrix and confidence interval were represented graphically using the corrupt method.

• International journal of advanced smart convergence
• /
• v.11 no.1
• /
• pp.11-18
• /
• 2022

Securities and investment services have and use large data. Investors started to invest through their own analysis methods. There are 22 major securities and investment companies in Korea and only 6 companies support open API. Python is effective for requesting and receiving, analyzing text data from open API. Daishin Securities Co. is the only open API that officially supports Python, and eBest Investment & Securities Co. unofficially supports Python. There are two important differences between CYBOS plus of Daishin Securities Co. and xingAPI of eBest Investment & Securities Co. First, we must log in to CYBOS plus to access the server of Daishin Securities Co. And the python program does not require a logon. However, to receive data using xingAPI, users log on in an individual Python program. Second, CYBOS plus receives data in a Request/Reply method, and zingAPI receives data through events. It can be thought that these points will show a difference in response time. Response time is important to users who use open APIs. Data were measured from August 5, 2021, to February 3, 2022. For each measurement, 15 repeated measurements were taken to obtain 420 measurements. To increase the accuracy of the study, both APIs were measured alternately under same conditions. A paired t-test was performed to test the hypothesis that the null hypothesis is there was no difference in means. The p-value is 0.2961, we do not reject null hypothesis. Therefore, we can see that there is no significant difference between means. From the boxplot, we can see that the distribution of the response time of eBest is more spread out than that of Cybos, and the position of the center is slightly lower. CYBOS plus has no restrictions on Python programming, but xingAPI has some limits because it indirectly supports Python programming. For example, there is a limit to receiving more than one current price.