• Title/Summary/Keyword: Scala

Search Result 27, Processing Time 0.027 seconds

Performance Comparison of Python and Scala APIs in Spark Distributed Cluster Computing System (Spark 기반에서 Python과 Scala API의 성능 비교 분석)

  • Ji, Keung-yeup;Kwon, Youngmi
    • Journal of Korea Multimedia Society
    • /
    • v.23 no.2
    • /
    • pp.241-246
    • /
    • 2020
  • Hadoop is a framework to process large data sets in a distributed way across clusters of nodes. It has been a popular platform to process big data, but in recent years, other platforms became competitive ones depending on the characteristics of the application. Spark is one of distributed platforms to enable real-time data processing and improve overall processing performance over Hadoop by introducing in-memory processing instead of disk I/O. Whereas Hadoop is designed to work on Java and data analysis is processed using Java API, Spark provides a variety of APIs with Scala, Python, Java and R. In this paper, the goal is to find out whether the APIs of different programming languages af ect the performances in Spark. We chose two popular APIs: Python and Scala. Python is easy to learn and is used in AI domain in a wide range. Scala is a programming language with advantages of parallelism. Our experiment shows much faster processing with Scala API than Python API. For the performance issues on AI-based analysis, further study is needed.

Feasibility Study Of Functional Programming In Scala Language By Implementing An Interpreter

  • Sugwoo, Byun
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.2
    • /
    • pp.111-119
    • /
    • 2023
  • In this paper, we investigate the feasibility of functional programming in the Scala language. The main issue is to what extent Scala is able to handle major properties of functional programming such as lambda expression, high-order functions, generic types, algebraic data types, and monads. For this purpose, we implement an interpreter of an imperative language. In this implementation, the same functional programming techniques are applied to both Haskell and Scala languages, and then these two versions of implementations are compared and analyzed. The abstract syntax tree of an imperative language is expressed as algebraic data types with generics and enum classes in Scala, and the state transition of imperative languages is implemented by using state monad. Extension and given, new features of Scala, are used as well.

Development of a Kernel Thread Web Accelerator (SCALA-AX) (커널 쓰레드 웹가속기(SCALA-AX) 개발)

  • Park, Jong-Gyu;Min, Byung-Jo;Lim, Han-Na;Park, Jang-Hoon;Chang, Whi;Kim, Hag-Bae
    • The KIPS Transactions:PartA
    • /
    • v.9A no.3
    • /
    • pp.327-332
    • /
    • 2002
  • Conventional proxy web cache, which is generally used to caching server, is a content-copy based system. This method focuses on speeding up the phase delivery not improving the webserver performance. However, if immense clients attempt to connect the webserver simultaneously, the proxy web cache cannot achieve the desired result. In this paper, we propose the web accelerator called the SCALA-AX, whitch improves web server performance by accelerating the delivery contents. The SCALA-AX is built in the Linux-based kernel as a kernel modulo and works in combination with the conventional webserver program. The SCALA-AX speeds up the processing rate of the webserver, because it processes the requests using the kernel thread. The SCALA-AX also applies the well-developed cache algorithm to the processing, and thus it obtains the advantage of the caching server without installing additional hardware. A banchmarking test demonstrates that the SCALA-AX improves webserver performance by up to 500% for content delivery.

Development of a Web Accelerator in the Kernel

  • Park, Jong-Gyu;Lim, Han-Na;Kim, Hag-Bae
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2001.10a
    • /
    • pp.70.3-70
    • /
    • 2001
  • In this paper, we suggest a kernel level multi thread web accelerator (called the SCALA-AX), which significantly improves the performance of the web soerver. In comparison with a conventional proxy web cache that is generally called a caching server and a simple content-copy based system, the primary functions and goals of SCALA-AX are designed to maximize the content services of a front end web server with high performance. Specifically, the SCALA-AX runs on the kernel level of a web sorrel, based on the newest caching techniques. Moreover, the SCALA-AX supports the http 1.1 protocol and allows the dynamic pages as well as static pages to be processed.

  • PDF

Comparison of Scala and R for Machine Learning in Spark (스파크에서 스칼라와 R을 이용한 머신러닝의 비교)

  • Woo-Seok Ryu
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.18 no.1
    • /
    • pp.85-90
    • /
    • 2023
  • Data analysis methodology in the healthcare field is shifting from traditional statistics-oriented research methods to predictive research using machine learning. In this study, we survey various machine learning tools, and compare several programming models, which utilize R and Spark, for applying R, a statistical tool widely used in the health care field, to machine learning. In addition, we compare the performance of linear regression model using scala, which is the basic languages of Spark and R. As a result of the experiment, the learning execution time when using SparkR increased by 10 to 20% compared to Scala. Considering the presented performance degradation, SparkR's distributed processing was confirmed as useful in R as the traditional statistical analysis tool that could be used as it is.

A LALR(1) Parse-Table Construction Combinator (LALR(1) 파스테이블 생성 콤비네이터)

  • Choi, Min-Yong;Doh, Kyung-Goo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.10a
    • /
    • pp.21-23
    • /
    • 2015
  • Scala의 파서 콤비네이터를 사용하면 프로그래밍 언어의 파서를 비교적 손쉽게 구현할 수 있다. 그런데 Scala의 파서 콤비네이터는 몇 가지 문제점이 있다. 첫째, 파싱을 할 때마다 파서를 생성해야 하는 온라인 방식이라 처리속도가 늦을 수밖에 없다. 둘째, 하향식 파서라 좌 재귀 문법을 처리하지 못하는 문제가 있다. 따라서 좌 재귀가 없도록 문법을 작성해야한다. 본 논문에서는 위의 문제들을 해결하기 위해 파서 콤비네이터의 장점인 직관적인 문법정의와 LALR(1)의 문법 표현력을 결합한 LALR(1) 파스테이블 생성 콤비네이터를 정의한다.

Evaluation of the Cochlear Electrode Behavior in the 3D Human Cochlea Model by FEM Analysis (3차원 달팽이관 모델에서의 인공와우 전극의 거동에 대한 유한요소해석)

  • 임윤섭;박세익;김용협;오승하;김성준
    • Journal of Biomedical Engineering Research
    • /
    • v.25 no.3
    • /
    • pp.207-215
    • /
    • 2004
  • A finite element analysis is used in this study to model 6 kinds of electrode by a genetic program in order to evaluate the mechanical effect on the 3D human cochlea model and the behavior of electrode. Human cochlea is modeled by the spiral-approximation method and the shape of scala tympani is extracted from the mid section of the human cochlea. Contact pressure at the tip and the insertion force are found to be highest when the wires stack horizontally. Axial rotation of electrode is minimal comparing with the stimulating current spread. The results indicate that the electrode stiffness is important to minimize the trauma.

Control the length of beam trajectory with a quadruple triplet for heavy ion accelerator

  • Wei, Shaoqing;Zhang, Zhan;Lee, Sangjin;Kim, Do Gyun;Kim, Jang Youl
    • Progress in Superconductivity and Cryogenics
    • /
    • v.18 no.4
    • /
    • pp.40-43
    • /
    • 2016
  • Beam trajectory is needed to be controlled in heavy ion accelerator system. Quadruple magnets are widely used in heavy ion accelerator for focusing the transporting particles. A quadruple triplet system which consists of three consecutive quadrupoles, Q1, Q2 and Q3, is used to control beam trajectory at a focused position. Q1 and Q3 have symmetry with respect to Q2. The beam trajectory in magnet system is affected by higher order fields existed in real fields. For quadrupoles, the representation simulation of beam trajectory was carried out to study the beam trajectory and to estimate an effect of higher order field in triplet system. SCALA program was used to simulate the beam trajectory in $Opera^{TM}$. SCALA can analyze a large number of beam trajectories at the same time by adjusting the size of finite element of the emitter. With $Opera^{TM}$ and $Matlab^{TM}$ programs, the position of focused beam spot in quadruple triplet system can be increased or decreased using evolution strategy (ES) method, therefore the length of triplet system can be controlled. Finally, the quadruple triplet system with the appropriate length and expected beam spot range was suggested in this paper.