• Title/Summary/Keyword: Big data Era

Search Result 361, Processing Time 0.024 seconds

Graph Database Solution for Higher Order Spatial Statistics in the Era of Big Data

  • Sabiu, Cristiano G.;Kim, Juhan
    • The Bulletin of The Korean Astronomical Society
    • /
    • v.44 no.1
    • /
    • pp.79.1-79.1
    • /
    • 2019
  • We present an algorithm for the fast computation of the general N-point spatial correlation functions of any discrete point set embedded within an Euclidean space of ${\mathbb{R}}n$. Utilizing the concepts of kd-trees and graph databases, we describe how to count all possible N-tuples in binned configurations within a given length scale, e.g. all pairs of points or all triplets of points with side lengths < rmax. Through benchmarking we show the computational advantage of our new graph-based algorithm over more traditional methods. We show that all 3-point configurations up to and beyond the Baryon Acoustic Oscillation scale (~200 Mpc in physical units) can be performed on current Sloan Digital Sky Survey (SDSS) data in reasonable time. Finally we present the first measurements of the 4-point correlation function of ~0.5 million SDSS galaxies over the redshift range 0.43< z <0.7. We present the publicly available code GRAMSCI (GRAph Made Statistics for Cosmological Information; bitbucket.org/csabiu/gramsci), under a GNU General Public License.

  • PDF

Fused inverse regression with multi-dimensional responses

  • Cho, Youyoung;Han, Hyoseon;Yoo, Jae Keun
    • Communications for Statistical Applications and Methods
    • /
    • v.28 no.3
    • /
    • pp.267-279
    • /
    • 2021
  • A regression with multi-dimensional responses is quite common nowadays in the so-called big data era. In such regression, to relieve the curse of dimension due to high-dimension of responses, the dimension reduction of predictors is essential in analysis. Sufficient dimension reduction provides effective tools for the reduction, but there are few sufficient dimension reduction methodologies for multivariate regression. To fill this gap, we newly propose two fused slice-based inverse regression methods. The proposed approaches are robust to the numbers of clusters or slices and improve the estimation results over existing methods by fusing many kernel matrices. Numerical studies are presented and are compared with existing methods. Real data analysis confirms practical usefulness of the proposed methods.

Enhancing E-commerce Security: A Comprehensive Approach to Real-Time Fraud Detection

  • Sara Alqethami;Badriah Almutanni;Walla Aleidarousr
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.4
    • /
    • pp.1-10
    • /
    • 2024
  • In the era of big data, the growth of e-commerce transactions brings forth both opportunities and risks, including the threat of data theft and fraud. To address these challenges, an automated real-time fraud detection system leveraging machine learning was developed. Four algorithms (Decision Tree, Naïve Bayes, XGBoost, and Neural Network) underwent comparison using a dataset from a clothing website that encompassed both legitimate and fraudulent transactions. The dataset exhibited an imbalance, with 9.3% representing fraud and 90.07% legitimate transactions. Performance evaluation metrics, including Recall, Precision, F1 Score, and AUC ROC, were employed to assess the effectiveness of each algorithm. XGBoost emerged as the top-performing model, achieving an impressive accuracy score of 95.85%. The proposed system proves to be a robust defense mechanism against fraudulent activities in e-commerce, thereby enhancing security and instilling trust in online transactions.

A Study on the Transforming Characteristics of the Entrance of Privately-built Apartment housing in Daegu (대구광역시 민영공동주택 출입구의 변화특성에 관한 연구)

  • Seo Hee-Sook;Lee Sang-Hong
    • Journal of the Korean housing association
    • /
    • v.17 no.1
    • /
    • pp.59-66
    • /
    • 2006
  • The purpose of this study was analyzing the Transforming Characteristics of the Entrance of Privately-built Apartment housing in Daegu from the 1970s to the 2000s. First of all, this study made an investigation into the general situation of Privately-built Apartment housing and the general idea of the Entrance. Through a field study, This study was analyzing change of an era of entrance of Apartment housing till quite recently existing in Daegu. The results are as the body III follows. Results of the research through change of an era is as follows. The change tendency is not to be big before 2003, but It has under gone change after 2003. Studies show that It is the care for the old and the weak, to take interest in natural lighting, to be going to change from passing the time of space to staying space and more space, to show interest in the preservation of public peace and design. So, The Entrance of Apartment housing has a bias towards more elegant. This research look forward to having an intention of furnishing preliminary data for a residence environment of high quality.

Analysis on the Media Content Research Trends in Media Convergence Era Based on Intellectual Information Technology (지능정보기술 기반 미디어 컨버전스 시대의 콘텐츠 연구경향 분석)

  • Jeon, Gyongran;Kim, Young-Chul
    • Journal of Korea Game Society
    • /
    • v.20 no.2
    • /
    • pp.113-122
    • /
    • 2020
  • This study is the research tendency(2016~2019) on the content and the intelligent information technology. After the IIT emerged as a social topic, related research increased, and interest in VR and AR was the highest. In games, more research has been done on VR and AR. In the case of big data technology, it was a tendency to pay attention to the study of movie contents. Many studies have attempted a technological approach to IIT. With regard to artificial intelligence technology, there were differences by technology and content area, mainly viewed from a legal and institutional perspective.

The Aims of Education in the Era of AI (21세기 인공지능시대에서의 교육의 목적)

  • Ree, Sangwook;Koh, Youngmee
    • Journal for History of Mathematics
    • /
    • v.30 no.6
    • /
    • pp.341-351
    • /
    • 2017
  • In the 21st century, the era of artificial intelligence, it is demanded to make a change of the paradigm of education by the recent impact of the 4th industrial revolution. The education up to now has emphasized knowledge, meanwhile the human resources for the future are required to be armed with four C's: critical thinking, creativity, communication and collaboration capability, rather than being equipped with just knowledge. That is because the future society demands such abilities, especially the creativity of each individual. At this point, we are asked to consider the aim of education and teaching methods. In school education, students are to be respected and considered able to develop their potentials by themselves. They shouldn't be estimated by tests in the process of learning as they are now. We reconsider the aim of education here by taking a look at Whitehead's opinions and the present educational situations.

Development of Artificial Intelligence-based Legal Counseling Chatbot System

  • Park, Koo-Rack
    • Journal of the Korea Society of Computer and Information
    • /
    • v.26 no.3
    • /
    • pp.29-34
    • /
    • 2021
  • With the advent of the 4th industrial revolution era, IT technology is creating new services that have not existed by converging with various existing industries and fields. In particular, in the field of artificial intelligence, chatbots and the latest technologies have developed dramatically with the development of natural language processing technology, and various business processes are processed through chatbots. This study is a study on a system that provides a close answer to the question the user wants to find by creating a structural form for legal inquiries through Slot Filling-based chatbot technology, and inputting a predetermined type of question. Using the proposal system, it is possible to construct question-and-answer data in a more structured form of legal information, which is unstructured data in text form. In addition, by managing the accumulated Q&A data through a big data storage system such as Apache Hive and recycling the data for learning, the reliability of the response can be expected to continuously improve.

Exploring the Job Competencies of Data Scientists Using Online Job Posting (온라인 채용정보를 이용한 데이터 과학자 요구 역량 탐색)

  • Jin, Xiangdan;Baek, Seung Ik
    • The Journal of Society for e-Business Studies
    • /
    • v.27 no.2
    • /
    • pp.1-20
    • /
    • 2022
  • As the global business environment is rapidly changing due to the 4th industrial revolution, new jobs that did not exist before are emerging. Among them, the job that companies are most interested in is 'Data Scientist'. As information and communication technologies take up most of our lives, data on not only online activities but also offline activities are stored in computers every hour to generate big data. Companies put a lot of effort into discovering new opportunities from such big data. The new job that emerged along with the efforts of these companies is data scientist. The demand for data scientist, a promising job that leads the big data era, is constantly increasing, but its supply is not still enough. Although data analysis technologies and tools that anyone can easily use are introduced, companies still have great difficulty in finding proper experts. One of the main reasons that makes the data scientist's shortage problem serious is the lack of understanding of the data scientist's job. Therefore, in this study, we explore the job competencies of a data scientist by qualitatively analyzing the actual job posting information of the company. This study finds that data scientists need not only the technical and system skills required of software engineers and system analysts in the past, but also business-related and interpersonal skills required of business consultants and project managers. The results of this study are expected to provide basic guidelines to people who are interested in the data scientist profession and to companies that want to hire data scientists.

Intelligent Sensor Technology Trend for Smart IT Convergence Platform (스마트 IT 융합 플랫폼을 위한 지능형 센서 기술 동향)

  • Kim, H.J.;Jin, H.B.;Youm, W.S.;Kim, Y.G.;Park, K.H.
    • Electronics and Telecommunications Trends
    • /
    • v.34 no.5
    • /
    • pp.14-25
    • /
    • 2019
  • As the Internet of Things, artificial intelligence and big data have received a lot of attention as key growth engines in the era of the fourth industrial revolution, data acquisition and utilization in mobile, automotive, robotics, manufacturing, agriculture, health care and national defense are becoming more important. Due to numerous data-based industrial changes, demand for sensor technologies is exploding, especially for intelligent sensor technologies that combine control, judgement, storage and communication functions with the sensors's own functions. Intelligent sensor technology can be defined as a convergence component technology that combines intelligent sensor units, intelligent algorithms, modules with signal processing circuits, and integrated plaform technologies. Intelligent sensor technology, which can be applied to variety of smart IT convergence services such as smart devices, smart homes, smart cars, smart factory, smart cities, and others, is evolving towards intelligent and convergence technologies that produce new high-value information through recognition, reasoning, and judgement based on artificial intelligence. As a result, development of intelligent sensor units is accelerating with strategies for miniaturization, low-power consumption and convergence, new form factor such as flexible and stretchable form, and integration of high-resolution sensor arrays. In the future, these intelligent sensor technologies will lead explosive sensor industries in the era of data-based artificial intelligence and will greatly contribute to enhancing nation's competitiveness in the global sensor market. In this report, we analyze and summarize the recent trends in intelligent sensor technologies, especially those for four core technologies.

Design of Trajectory Data Indexing and Query Processing for Real-Time LBS in MapReduce Environments (MapReduce 환경에서의 실시간 LBS를 위한 이동궤적 데이터 색인 및 검색 시스템 설계)

  • Chung, Jaehwa
    • Journal of Digital Contents Society
    • /
    • v.14 no.3
    • /
    • pp.313-321
    • /
    • 2013
  • In recent, proliferation of mobile smart devices have led to big-data era, the importance of location-based services is increasing due to the exponential growth of trajectory related data. In order to process trajectory data, parallel processing platforms such as cloud computing and MapReduce are necessary. Currently, the researches based on MapReduce are on progress, but due to the MapReduce's properties in using batch processing and simple key-value structure, applying MapReduce framework for real time LBS is difficult. Therefore, in this research we propose a suitable system design on efficient indexing and search techniques for real time service based on detailed analysis on the properties of MapReduce.