• Title/Summary/Keyword: Sparsity Test

Search Result 16, Processing Time 0.023 seconds

Development of Optimal Power Flow for the Ancillary Service of Reactive Power Generation under Restructuring Environment (전력산업 구조개편 환경 하에서 무효전력 보조서비스 운용을 위한 최적조류계산법 개발)

  • Lee, Seung-Ryeol;Lee, Byeong-Jun;Song, Tae-Yong;Jeong, Min-Hwa;Mun, Yeong-Hwan
    • The Transactions of the Korean Institute of Electrical Engineers A
    • /
    • v.51 no.1
    • /
    • pp.37-44
    • /
    • 2002
  • This Paper suggests reasonable pricing mathod fur Reactive Power in Optimal Power Flow for the system analysis. Under restructuring, not only real power pricing but also reactive power pricing is important for the system analysis and operation. If people just focus on real power pricing, the Generators may no generate reactive power voluntarily, because the Generators may not recover the cost of the reactive power generation. So making a reasonable reactive power pricing is becoming more important than any other time. In this paper, the authors set a Proper Power factor and price the portion of the reactive power that exceeds the power factor using Interior Point Method. By applying this method, the System operator can use this strategy for the analysis of reactive power generation pricing and the Generator can get the motivation to generate reactive power. The author develops fully optimized fast Primal Dual Interior Point Method with sparsity technique and applies this method to Reliability Test System (RTS24) and KEPCO 674 bus system (684 buses. 1279 lines). It shows adaptability and usefulness.

Recommender Systems using Structural Hole and Collaborative Filtering (구조적 공백과 협업필터링을 이용한 추천시스템)

  • Kim, Mingun;Kim, Kyoung-Jae
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.4
    • /
    • pp.107-120
    • /
    • 2014
  • This study proposes a novel recommender system using the structural hole analysis to reflect qualitative and emotional information in recommendation process. Although collaborative filtering (CF) is known as the most popular recommendation algorithm, it has some limitations including scalability and sparsity problems. The scalability problem arises when the volume of users and items become quite large. It means that CF cannot scale up due to large computation time for finding neighbors from the user-item matrix as the number of users and items increases in real-world e-commerce sites. Sparsity is a common problem of most recommender systems due to the fact that users generally evaluate only a small portion of the whole items. In addition, the cold-start problem is the special case of the sparsity problem when users or items newly added to the system with no ratings at all. When the user's preference evaluation data is sparse, two users or items are unlikely to have common ratings, and finally, CF will predict ratings using a very limited number of similar users. Moreover, it may produces biased recommendations because similarity weights may be estimated using only a small portion of rating data. In this study, we suggest a novel limitation of the conventional CF. The limitation is that CF does not consider qualitative and emotional information about users in the recommendation process because it only utilizes user's preference scores of the user-item matrix. To address this novel limitation, this study proposes cluster-indexing CF model with the structural hole analysis for recommendations. In general, the structural hole means a location which connects two separate actors without any redundant connections in the network. The actor who occupies the structural hole can easily access to non-redundant, various and fresh information. Therefore, the actor who occupies the structural hole may be a important person in the focal network and he or she may be the representative person in the focal subgroup in the network. Thus, his or her characteristics may represent the general characteristics of the users in the focal subgroup. In this sense, we can distinguish friends and strangers of the focal user utilizing the structural hole analysis. This study uses the structural hole analysis to select structural holes in subgroups as an initial seeds for a cluster analysis. First, we gather data about users' preference ratings for items and their social network information. For gathering research data, we develop a data collection system. Then, we perform structural hole analysis and find structural holes of social network. Next, we use these structural holes as cluster centroids for the clustering algorithm. Finally, this study makes recommendations using CF within user's cluster, and compare the recommendation performances of comparative models. For implementing experiments of the proposed model, we composite the experimental results from two experiments. The first experiment is the structural hole analysis. For the first one, this study employs a software package for the analysis of social network data - UCINET version 6. The second one is for performing modified clustering, and CF using the result of the cluster analysis. We develop an experimental system using VBA (Visual Basic for Application) of Microsoft Excel 2007 for the second one. This study designs to analyzing clustering based on a novel similarity measure - Pearson correlation between user preference rating vectors for the modified clustering experiment. In addition, this study uses 'all-but-one' approach for the CF experiment. In order to validate the effectiveness of our proposed model, we apply three comparative types of CF models to the same dataset. The experimental results show that the proposed model outperforms the other comparative models. In especial, the proposed model significantly performs better than two comparative modes with the cluster analysis from the statistical significance test. However, the difference between the proposed model and the naive model does not have statistical significance.

Applications of Graph Theory for the Pipe Network Analysis (상수관망해석을 위한 도학의 적용)

  • Park, Jae-Hong;Han, Geon-Yeon
    • Journal of Korea Water Resources Association
    • /
    • v.31 no.4
    • /
    • pp.439-448
    • /
    • 1998
  • There are many methods to calculate steady-state flowrate in a large water distribution system. Linear method which analyzes continuity equations and energy equations simultaneously is most widely used. Though it is theoretically simple, when it is applied to a practical water distribution system, it produces a very sparse coefficient matrix and most of its diagonal elements are to be zero. This sparsity characteristic of coefficient matrix makes it difficult to analyze pipe flow using the linear method. In this study, a graph theory is introduced to water distribution system analysis in order to prevent from producing ill-conditioned coefficient matrix and the technique is developed to produce positive-definite matrix. To test applicability of developed method, this method is applied to 22 pipes and 142 pipes system located nearby Taegu city. The results obtained from these applications show that the method can calculate flowrate effectively without failure in converage. Thus it is expected that the method can analyze steady state flowrate and pressure in pipe network systems efficiently. Keywords : pipe flow analysis, graph theory, linear method.

  • PDF

Preconditioned Jacobian-free Newton-Krylov fully implicit high order WENO schemes and flux limiter methods for two-phase flow models

  • Zhou, Xiafeng;Zhong, Changming;Li, Zhongchun;Li, Fu
    • Nuclear Engineering and Technology
    • /
    • v.54 no.1
    • /
    • pp.49-60
    • /
    • 2022
  • Motivated by the high-resolution properties of high-order Weighted Essentially Non-Oscillatory (WENO) and flux limiter (FL) for steep-gradient problems and the robust convergence of Jacobian-free Newton-Krylov (JFNK) methods for nonlinear systems, the preconditioned JFNK fully implicit high-order WENO and FL schemes are proposed to solve the transient two-phase two-fluid models. Specially, the second-order fully-implicit BDF2 is used for the temporal operator and then the third-order WENO schemes and various flux limiters can be adopted to discrete the spatial operator. For the sake of the generalization of the finite-difference-based preconditioning acceleration methods and the excellent convergence to solve the complicated and various operational conditions, the random vector instead of the initial condition is skillfully chosen as the solving variables to obtain better sparsity pattern or more positions of non-zero elements in this paper. Finally, the WENO_JFNK and FL_JFNK codes are developed and then the two-phase steep-gradient problem, phase appearance/disappearance problem, U-tube problem and linear advection problem are tested to analyze the convergence, computational cost and efficiency in detailed. Numerical results show that WENO_JFNK and FL_JFNK can significantly reduce numerical diffusion and obtain better solutions than traditional methods. WENO_JFNK gives more stable and accurate solutions than FL_JFNK for the test problems and the proposed finite-difference-based preconditioning acceleration methods based on the random vector can significantly improve the convergence speed and efficiency.

Resolving the 'Gray sheep' Problem Using Social Network Analysis (SNA) in Collaborative Filtering (CF) Recommender Systems (소셜 네트워크 분석 기법을 활용한 협업필터링의 특이취향 사용자(Gray Sheep) 문제 해결)

  • Kim, Minsung;Im, Il
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.2
    • /
    • pp.137-148
    • /
    • 2014
  • Recommender system has become one of the most important technologies in e-commerce in these days. The ultimate reason to shop online, for many consumers, is to reduce the efforts for information search and purchase. Recommender system is a key technology to serve these needs. Many of the past studies about recommender systems have been devoted to developing and improving recommendation algorithms and collaborative filtering (CF) is known to be the most successful one. Despite its success, however, CF has several shortcomings such as cold-start, sparsity, gray sheep problems. In order to be able to generate recommendations, ordinary CF algorithms require evaluations or preference information directly from users. For new users who do not have any evaluations or preference information, therefore, CF cannot come up with recommendations (Cold-star problem). As the numbers of products and customers increase, the scale of the data increases exponentially and most of the data cells are empty. This sparse dataset makes computation for recommendation extremely hard (Sparsity problem). Since CF is based on the assumption that there are groups of users sharing common preferences or tastes, CF becomes inaccurate if there are many users with rare and unique tastes (Gray sheep problem). This study proposes a new algorithm that utilizes Social Network Analysis (SNA) techniques to resolve the gray sheep problem. We utilize 'degree centrality' in SNA to identify users with unique preferences (gray sheep). Degree centrality in SNA refers to the number of direct links to and from a node. In a network of users who are connected through common preferences or tastes, those with unique tastes have fewer links to other users (nodes) and they are isolated from other users. Therefore, gray sheep can be identified by calculating degree centrality of each node. We divide the dataset into two, gray sheep and others, based on the degree centrality of the users. Then, different similarity measures and recommendation methods are applied to these two datasets. More detail algorithm is as follows: Step 1: Convert the initial data which is a two-mode network (user to item) into an one-mode network (user to user). Step 2: Calculate degree centrality of each node and separate those nodes having degree centrality values lower than the pre-set threshold. The threshold value is determined by simulations such that the accuracy of CF for the remaining dataset is maximized. Step 3: Ordinary CF algorithm is applied to the remaining dataset. Step 4: Since the separated dataset consist of users with unique tastes, an ordinary CF algorithm cannot generate recommendations for them. A 'popular item' method is used to generate recommendations for these users. The F measures of the two datasets are weighted by the numbers of nodes and summed to be used as the final performance metric. In order to test performance improvement by this new algorithm, an empirical study was conducted using a publically available dataset - the MovieLens data by GroupLens research team. We used 100,000 evaluations by 943 users on 1,682 movies. The proposed algorithm was compared with an ordinary CF algorithm utilizing 'Best-N-neighbors' and 'Cosine' similarity method. The empirical results show that F measure was improved about 11% on average when the proposed algorithm was used

    . Past studies to improve CF performance typically used additional information other than users' evaluations such as demographic data. Some studies applied SNA techniques as a new similarity metric. This study is novel in that it used SNA to separate dataset. This study shows that performance of CF can be improved, without any additional information, when SNA techniques are used as proposed. This study has several theoretical and practical implications. This study empirically shows that the characteristics of dataset can affect the performance of CF recommender systems. This helps researchers understand factors affecting performance of CF. This study also opens a door for future studies in the area of applying SNA to CF to analyze characteristics of dataset. In practice, this study provides guidelines to improve performance of CF recommender systems with a simple modification.

  • Stock Price Prediction by Utilizing Category Neutral Terms: Text Mining Approach (카테고리 중립 단어 활용을 통한 주가 예측 방안: 텍스트 마이닝 활용)

    • Lee, Minsik;Lee, Hong Joo
      • Journal of Intelligence and Information Systems
      • /
      • v.23 no.2
      • /
      • pp.123-138
      • /
      • 2017
    • Since the stock market is driven by the expectation of traders, studies have been conducted to predict stock price movements through analysis of various sources of text data. In order to predict stock price movements, research has been conducted not only on the relationship between text data and fluctuations in stock prices, but also on the trading stocks based on news articles and social media responses. Studies that predict the movements of stock prices have also applied classification algorithms with constructing term-document matrix in the same way as other text mining approaches. Because the document contains a lot of words, it is better to select words that contribute more for building a term-document matrix. Based on the frequency of words, words that show too little frequency or importance are removed. It also selects words according to their contribution by measuring the degree to which a word contributes to correctly classifying a document. The basic idea of constructing a term-document matrix was to collect all the documents to be analyzed and to select and use the words that have an influence on the classification. In this study, we analyze the documents for each individual item and select the words that are irrelevant for all categories as neutral words. We extract the words around the selected neutral word and use it to generate the term-document matrix. The neutral word itself starts with the idea that the stock movement is less related to the existence of the neutral words, and that the surrounding words of the neutral word are more likely to affect the stock price movements. And apply it to the algorithm that classifies the stock price fluctuations with the generated term-document matrix. In this study, we firstly removed stop words and selected neutral words for each stock. And we used a method to exclude words that are included in news articles for other stocks among the selected words. Through the online news portal, we collected four months of news articles on the top 10 market cap stocks. We split the news articles into 3 month news data as training data and apply the remaining one month news articles to the model to predict the stock price movements of the next day. We used SVM, Boosting and Random Forest for building models and predicting the movements of stock prices. The stock market opened for four months (2016/02/01 ~ 2016/05/31) for a total of 80 days, using the initial 60 days as a training set and the remaining 20 days as a test set. The proposed word - based algorithm in this study showed better classification performance than the word selection method based on sparsity. This study predicted stock price volatility by collecting and analyzing news articles of the top 10 stocks in market cap. We used the term - document matrix based classification model to estimate the stock price fluctuations and compared the performance of the existing sparse - based word extraction method and the suggested method of removing words from the term - document matrix. The suggested method differs from the word extraction method in that it uses not only the news articles for the corresponding stock but also other news items to determine the words to extract. In other words, it removed not only the words that appeared in all the increase and decrease but also the words that appeared common in the news for other stocks. When the prediction accuracy was compared, the suggested method showed higher accuracy. The limitation of this study is that the stock price prediction was set up to classify the rise and fall, and the experiment was conducted only for the top ten stocks. The 10 stocks used in the experiment do not represent the entire stock market. In addition, it is difficult to show the investment performance because stock price fluctuation and profit rate may be different. Therefore, it is necessary to study the research using more stocks and the yield prediction through trading simulation.


    (34141) Korea Institute of Science and Technology Information, 245, Daehak-ro, Yuseong-gu, Daejeon
    Copyright (C) KISTI. All Rights Reserved.