Application Research on the Evaluation Index System of Visual Anthropology : based on Delphi method and analytic hierarchy process

Abstract

This paper introduces the development and current prosperity of the branch discipline of visual anthropology and describes the status and current situation of modern visual anthropology and analyzes the rationality that the achievements of visual anthropology is viewed as texts and enjoy the same achievements, making it more basis and scientific in the selection of visual anthropology in subsequent research. This article comprehensively uses Delphi method and analytic hierarchy process to construct an evaluation system of visual anthropology, as well as put forward the basic principles of screening films using this evaluation system in subsequent studies and a simple outlook for future expansion.

I. INTRODUCTION

As we all know, social science research has traditionally taken the form of still and moving images. Such as papers, monographs, research reports, etc. But in recent years, with the development of science and technology, more and more new media come into people's vision. Such as database results, academic film and television works. In the field of anthropology, a typical example is the work of visual anthropology: anthropological documentaries/ethnographic films [1]. Therefore, it is urgent to make the evaluation system of visual anthropology work scientific and standardized. How to establish the evaluation system of academic achievements should be discussed.

The dynamic pictures and sounds of film and television are the only means of human expression that can be called "accurate" [2]. Therefore, visual anthropology has a unique methodological perspective and academic characteristics.

Due to the characteristics of visual anthropology works and the current situation of the society, it is urgent to establish an evaluation system. On the one hand, it takes as much time and effort to produce high-quality film and television works as text works, which has more social influence. On the other hand, from the perspective of the influence of the works, academic works and papers of anthropology are often only read by those in the same discipline and rarely have social effects. And an excellent work of visual anthropology, its influence and dissemination not only limited in the profession, but also throughout all walks of life, and even can form the power to change the social concept. Therefore, from the perspective of the workload required for the production of works and the impact after the production of works, the achievements of visual anthropology deserve the same attention as the achievements of texts. Therefore, it is urgent to establish an assessment system that can be recognized by peers, win the trust of the academic community, and at the same time have relatively clear quantitative standards and rules.

 

II. ESTABLISHING EVALUATION SYSTEM BY DELPHI METHOD

 

Table 1. The specific evaluation indicators of Zhang-Aining.

 

In the process of constructing the evaluation system, the most recognized and convincing method is to apply the Delphi expert method. Delphi Expert Method (Delphi). It is a consulting and decision-making technology that can be applied to almost any field summarized and proposed by the American RAND Corporation in 1964. The core is to conduct several rounds of letters and consultations to seek expert opinions through anonymous methods. The forecasting and evaluation leadership group summarized and summarized the opinions of each round, and then sent them to each expert as reference materials for the experts to analyze and judge, and put forward new arguments. After so many iterations, the opinions gradually converge, and a more consistent and reliable conclusion or plan is obtained [3].

 

2.1. Initial construction of evaluation index system

This study analyzes a large number of related texts, film evaluation index systems. Zhang-Aining, the most influential film therapist in China, according to the table, believes that the principle of film selection for film therapy should follow the "principle of values", “the principle of empathy ” , “ the principle of timeliness ” , “ the principle of realm ” , “ the principle of systemic desensitization”[4].

China's first film and television anthropologist Wu-Qiao pointed out that the ethnographic or anthropological film and television works should be evaluated in accordance with the "Artistic Principles" "Cycle of Film Production" "Award Finale Principles" [5]. The specific indicators are shown in the following table:

 

Table 2. The specific evaluation indicators of Wu-Qiao

 

After analysis, screening, and refinement, a film evaluation index system consisting of 8 first-level indicators and 21 second-level indicators was initially constructed.

 

2.2. Identify Review Panel

This research has passed the China Academy of Art (the highest institution of Chinese cultural research); Beijing Film Academy (the highest institution where Chinese film theorists and practitioners gather);China Shaman Cultural Association (China's most influential association of folk culture practitioners) ;The official website of the China Folklore Research Association (the institution where China ’s most influential anthropological scholars are located), the number of experts to be selected is 22, and 13 people have received responses after communicating via email and telephone app.

In order to ensure the typical of the membership of the expert group, there are four main types of experts invited: scholars of anthropology, scholars of film studies, practitioners of folk culture, and filmmakers. The membership of the expert group is as follows:

 

Table 3. Membership of the Expert Group.

 

2.3. The First Round Delphi Questionnaire

The purpose of the first round of Delphi questionnaires was to seek experts' opinions on the revision of the preliminary evaluation index system for films and their judgments on their importance. In the first round of the Delphi method, 13 questionnaires were distributed by mail, and 13 were recovered with a 100% recovery rate: Pi=100%. This shows that the selected experts are concerned about the research and have a high degree of participation.

The first part of the questionnaire is the index importance scoring table: experts are asked to rate the importance of the indicators. The scoring rules are based on the Like-rt scale. Ask the experts for changes to the preliminary indicators and whether new indicators need to be added.

The second part of the questionnaire is a questionnaire for experts, as follows:

 

Table 4. Experts Authoritative Case.

 

According to the reliability of expert opinions, and the general evaluation index selection criteria:

▪ The index of expert authority Cr ≥0.7, \(\left(\mathrm{Cr}=\frac{C a+C s}{2}\right)\)

▪ \(\bar{X}\) (the indicator average) < 4 points,

▪ CV (the coefficient of variation) > 0.3

 

The indicators that meet these conditions are initially listed as "deleted" indicators, and then the indicators can be modified to form the second round of expert consultation based on expert opinions. The second round of expert consultation was formed after the indicators were modified. Based on this condition, the details of the recovered expert data are as the following Table 5.

Through the analysis of the recovered data, it was f ound that the "Realm principle", "the principle of time liness", "the principle of systemic desensitization" and "the principle of friendliness” and “Recurrence principl e" are eligible for deletion. Therefore, it is necessary to discuss these four indicators.

 

2.4. The Second Round Delphi questionnaire

A total of 13 Delphi questionnaires were sent by mail in the second round, and 13 were eventually recovered, with 13 valid and 100% recovery rates. In the recovered data, the "timeliness" principle changes as shown in the table below.

 

Table 6. The Change of "Timeliness" Principle.

 

Analysis of the recovered data shows that the “the principle of timeliness” in the work specification is only 69.3% and that the coefficient of variation is greater than 0.25. It shows that except for the consistency of this index item, the consistency of the other index items has been reached.

Judging the expert consistency test of this study, the re are 16 secondary indicators in the study, of which 15 are consistent. The consensus rate of secondary ind icators = 15/16 = 93. 75%> 80%. Therefore, the auth or It is considered that the film evaluation index syste m developed in this study has reached a consensus aft er two rounds of revisions, so the Delphi survey can be ended. Based on this condition, the details of the r ecovered expert data are as in Table 5.

 

III. USE AHP TO ASSIGN WEIGHTS

3.1. Establish Hierarchical Structure Model

Based on the movie evaluation index system that has be en constructed, a hierarchical model of the movie evaluati on index system can be established. The entire hierarchical model can be divided into three layers; target layer A, cri terion layer B, and indicator layer C. The target layer is th e film evaluation index system. The criterion layer include s the five dimensions of the system construction, and the i ndex layer includes the 16 indicators of the system constru ction.

The architecture diagram is as follows:

Fig. 1. Architecture Diagram.

Table 5. Recovered Expert Data.

 

3.2. Construct Pairwise Comparison Judgment Matrix

 

Experts make a pairwise comparison of all the indicators in the hierarchy, judge their relative importance according to the rules, and quantify the judgment results to form a judgment matrix. The rule for quantifying the judgment result is the 1-9 scale method proposed by Professor A.L. Saaty, an American operations researcher [6]. The judgment matrix obtained is shown in the table.

 

Table 8. Matrix Normalization.

 

3.3. Hierarchical Single Ordering and Consistency Check

The decision matrix corresponds to the vector of the maximum eigenvalue. After normalization, it is the ranking weight of the relative importance of the corresponding factor at the same level to a certain factor of the previous level. This process is called hierarchical single ordering.

After the matrix is normalized by the data as shown in Table 9. In order to avoid decision errors caused by violating the decision matrix, consistency check of the decision matrix is required. A. L Saaty's average random consistency index was used to check the consistency of the judgment matrix (see Table for average consistency indexes). According to each average consistency index, the consistency index of the judgment matrix CI == λmax -n / n- 1. Random consistency ratio CR = CI / RI. If CR <0.1, the matrix is considered to have satisfactory consistency.

The results of various consistency tests are shown in Table 10 (given the value given by the consistency check, RI = 1.1185). It can be seen from the above table that the random consistency ratio CR of all levels of random consistency is less than 0.1, and the matrix constructed in this paper can be considered to have satisfactory consistency.

 

3.4. Hierarchical Total Ranking and Consistency Check

The ranking of the relative importance of all factors at the same level to the high-level (total goal) is called the total ranking of the hierarchy. This process is carried out layer by layer from the highest layer to the lowest layer. After the total ranking of the layers is completed, consistency is also required test [6]. When layer A, total sorting random consistency ratio:

 

\(\mathrm{CR}=\sum_{i=1}^{m} \frac{\text {CIYiY} \omega i}{\left[\sum_{i=1}^{m} \text {Rli\omegai}\right]}\)       (1)

 

If CR <0.1, it is considered that the hierarchical total ranking results have a satisfactory consistency and accept the analysis results.

 

Table 9. Weights of the Indexes of the Coefficients.

Table 7. Primary Index Judgment Matrix

The following is a total ranking consistency check of the movie's evaluation index system:

 

\(\mathrm{CR}=\sum_{i=1}^{5} \frac{\text { Clili } \omega i}{\sum_{i=1}^{5} R l i \omega i}=0.055<0.10\)        (2)

 

It shows that each decision matrix meets the consistency requirements. Finally, this article constructs a complete movie evaluation index system with weight distribution as shown in Table 10 (see Appendix also).

 

IV. ANALYSIS OF RESULTS OF INDICATOR SYSTEM

Through analysis, we can see that the consideration of anthropological film and television works should be divided into five dimensions, and the proportion from high to low should be: artistic sense principle, values principle, timeliness principle, empathy principle, and award qualified principle.

▪ In the Artistic Principles: attention should be paid to the rendering of details first, followed by the immersion of the scene, and the use of audiovisual language and the presentation of the state of movement should not be neglected.

▪ In the principles of values: we should first emphasize the subjectivity, then, we should stay cautious about the output of values while respecting them.

▪ In the principle of timeliness: we should not ignore the principle of timeliness in the basis of laying emphasis on effectiveness and unity.

▪ In the principle of empathy: we must first focus on telling stories with shots, and secondly the attraction of the story itself. At the same time, the psychological and degrees of education of the subjects cannot be ignored.

▪ In the principle of award finale: being short-listed for international awards can luster the work, then the national film festival or film festival. Meanwhile, the supporting situation of texts and film and television works must not be overlooked

 

V. CONCLUSION

Based on the analysis of the relevant literature and the existing evaluation index system, this article sorts out the preliminary evaluation indicators of Shaman cultural content film evaluation, comprehensively uses Delphi and AHP, and combines qualitative and quantitative methods to give full play to each method. Specialty, to avoid the shortcomings of using a method alone, and finally constructed a scientific and reasonable Shaman cultural content movie evaluation index system consisting of 5 firstlevel indicators and 16 second-level indicators.