• Journal of Intelligence and Information Systems
• /
• v.23 no.2
• /
• pp.1-17
• /
• 2017

The deep learning framework is software designed to help develop deep learning models. Some of its important functions include "automatic differentiation" and "utilization of GPU". The list of popular deep learning framework includes Caffe (BVLC) and Theano (University of Montreal). And recently, Microsoft's deep learning framework, Microsoft Cognitive Toolkit, was released as open-source license, following Google's Tensorflow a year earlier. The early deep learning frameworks have been developed mainly for research at universities. Beginning with the inception of Tensorflow, however, it seems that companies such as Microsoft and Facebook have started to join the competition of framework development. Given the trend, Google and other companies are expected to continue investing in the deep learning framework to bring forward the initiative in the artificial intelligence business. From this point of view, we think it is a good time to compare some of deep learning frameworks. So we compare three deep learning frameworks which can be used as a Python library. Those are Google's Tensorflow, Microsoft's CNTK, and Theano which is sort of a predecessor of the preceding two. The most common and important function of deep learning frameworks is the ability to perform automatic differentiation. Basically all the mathematical expressions of deep learning models can be represented as computational graphs, which consist of nodes and edges. Partial derivatives on each edge of a computational graph can then be obtained. With the partial derivatives, we can let software compute differentiation of any node with respect to any variable by utilizing chain rule of Calculus. First of all, the convenience of coding is in the order of CNTK, Tensorflow, and Theano. The criterion is simply based on the lengths of the codes and the learning curve and the ease of coding are not the main concern. According to the criteria, Theano was the most difficult to implement with, and CNTK and Tensorflow were somewhat easier. With Tensorflow, we need to define weight variables and biases explicitly. The reason that CNTK and Tensorflow are easier to implement with is that those frameworks provide us with more abstraction than Theano. We, however, need to mention that low-level coding is not always bad. It gives us flexibility of coding. With the low-level coding such as in Theano, we can implement and test any new deep learning models or any new search methods that we can think of. The assessment of the execution speed of each framework is that there is not meaningful difference. According to the experiment, execution speeds of Theano and Tensorflow are very similar, although the experiment was limited to a CNN model. In the case of CNTK, the experimental environment was not maintained as the same. The code written in CNTK has to be run in PC environment without GPU where codes execute as much as 50 times slower than with GPU. But we concluded that the difference of execution speed was within the range of variation caused by the different hardware setup. In this study, we compared three types of deep learning framework: Theano, Tensorflow, and CNTK. According to Wikipedia, there are 12 available deep learning frameworks. And 15 different attributes differentiate each framework. Some of the important attributes would include interface language (Python, C ++, Java, etc.) and the availability of libraries on various deep learning models such as CNN, RNN, DBN, and etc. And if a user implements a large scale deep learning model, it will also be important to support multiple GPU or multiple servers. Also, if you are learning the deep learning model, it would also be important if there are enough examples and references.

• Journal of Intelligence and Information Systems
• /
• v.19 no.1
• /
• pp.111-123
• /
• 2013

Semantic similarity/relatedness measure between two concepts plays an important role in research on system integration and database integration. Moreover, current research on keyword recommendation or tag clustering strongly depends on this kind of semantic measure. For this reason, many researchers in various fields including computer science and computational linguistics have tried to improve methods to calculating semantic similarity/relatedness measure. This study of similarity between concepts is meant to discover how a computational process can model the action of a human to determine the relationship between two concepts. Most research on calculating semantic similarity usually uses ready-made reference knowledge such as semantic network and dictionary to measure concept similarity. The topological method is used to calculated relatedness or similarity between concepts based on various forms of a semantic network including a hierarchical taxonomy. This approach assumes that the semantic network reflects the human knowledge well. The nodes in a network represent concepts, and way to measure the conceptual similarity between two nodes are also regarded as ways to determine the conceptual similarity of two words(i.e,. two nodes in a network). Topological method can be categorized as node-based or edge-based, which are also called the information content approach and the conceptual distance approach, respectively. The node-based approach is used to calculate similarity between concepts based on how much information the two concepts share in terms of a semantic network or taxonomy while edge-based approach estimates the distance between the nodes that correspond to the concepts being compared. Both of two approaches have assumed that the semantic network is static. That means topological approach has not considered the change of semantic relation between concepts in semantic network. However, as information communication technologies make advantage in sharing knowledge among people, semantic relation between concepts in semantic network may change. To explain the change in semantic relation, we adopt the cognitive semantics. The basic assumption of cognitive semantics is that humans judge the semantic relation based on their cognition and understanding of concepts. This cognition and understanding is called 'World Knowledge.' World knowledge can be categorized as personal knowledge and cultural knowledge. Personal knowledge means the knowledge from personal experience. Everyone can have different Personal Knowledge of same concept. Cultural Knowledge is the knowledge shared by people who are living in the same culture or using the same language. People in the same culture have common understanding of specific concepts. Cultural knowledge can be the starting point of discussion about the change of semantic relation. If the culture shared by people changes for some reasons, the human's cultural knowledge may also change. Today's society and culture are changing at a past face, and the change of cultural knowledge is not negligible issues in the research on semantic relationship between concepts. In this paper, we propose the future directions of research on semantic similarity. In other words, we discuss that how the research on semantic similarity can reflect the change of semantic relation caused by the change of cultural knowledge. We suggest three direction of future research on semantic similarity. First, the research should include the versioning and update methodology for semantic network. Second, semantic network which is dynamically generated can be used for the calculation of semantic similarity between concepts. If the researcher can develop the methodology to extract the semantic network from given knowledge base in real time, this approach can solve many problems related to the change of semantic relation. Third, the statistical approach based on corpus analysis can be an alternative for the method using semantic network. We believe that these proposed research direction can be the milestone of the research on semantic relation.

• The Korean Journal of Physiology and Pharmacology
• /
• v.15 no.6
• /
• pp.371-382
• /
• 2011

We performed experiments using Aplysia neurons to identify the mechanism underlying the changes in the firing patterns in response to temperature changes. When the temperature was gradually increased from $11^{\circ}C$ to $31^{\circ}C$ the firing patterns changed sequentially from the silent state to beating, doublets, beating-chaos, bursting-chaos, square-wave bursting, and bursting-oscillation patterns. When the temperature was decreased over the same temperature range, these sequential changes in the firing patterns reappeared in reverse order. To simulate this entire range of spiking patterns we modified nonlinear differential equations that Chay and Lee made using temperature-dependent scaling factors. To refine the equations, we also analyzed the spike pattern changes in the presence of potassium channel blockers. Based on the solutions of these equations and potassium channel blocker experiments, we found that, as temperature increases, the maximum value of the potassium channel relaxation time constant, ${\tau}_n(t)$ increases, but the maximum value of the probabilities of openings for activation of the potassium channels, n(t) decreases. Accordingly, the voltage-dependent potassium current is likely to play a leading role in the temperature-dependent changes in the firing patterns in Aplysia neurons.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.9 no.12
• /
• pp.4835-4855
• /
• 2015

TV white space (TVWS) is showing promise to become the first widespread practical application of cognitive technology. In fact, regulators worldwide are beginning to allow access to the TV band for secondary users, on the provision that they access the geolocation database. Device-to-device (D2D) can improve the spectrum efficiency, but large-scale D2D communications that underlie TVWS may generate undesirable interference to TV receivers and cause severe mutual interference. In this paper, we use an established geolocation database to investigate the power allocation problem, in order to maximize the total sum throughput of D2D links in TVWS while guaranteeing the quality-of-service (QoS) requirement for both D2D links and TV receivers. Firstly, we formulate an optimization problem based on the system model, which is nonconvex and intractable. Secondly, we use an effective approach to convert the original problem into a series of convex problems and we solve these problems using interior point methods that have polynomial computational complexity. Additionally, we propose an iterative algorithm based on the barrier method to locate the optimal solution. Simulation results show that the proposed algorithm has strong performance with high approximation accuracy for both small and large dimensional problems, and it is superior to both the active set algorithm and genetic algorithm.