An analysis of user behaviors on the search engine results pages based on the demographic characteristics

Abstract

The purpose of this survey-based study is to make an analysis of search engine users' behaviors on the Search Engine Results Pages (SERPs) based on the three demographic characteristics gender, age, and program studying. In this study, a questionnaire was designed with 12 closed-ended questions. Remaining questions other than the demographic characteristic related ones were about "tab", "advertisement", "spelling suggestion", "related query suggestion", "instant search suggestion", "video result", "image result", "pagination" and the amount of clicking results. The questionnaire was used and the data collected were analyzed with the descriptive statistics as well as the inferential statistics. 84.2% of the study population was reached. Some of the major results are as follows: Most of each demographic characteristic category (i.e. female, male, under-20, 20-24, above-24, English computer engineering, Turkish computer engineering, software engineering) have rarely or more click for tab, spelling suggestion, related query suggestion, instant search suggestion, video result, image result, and pagination. More than 50.0% of female category click advertisement rarely; however, for the others, 50.0% or more never click advertisement. For every demographic characteristic category, between 78.0% and 85.4% click 10 or fewer results. This study would be the first attempt with its complete content and design. Search engine providers and researchers would gain knowledge to user behaviors about the usage of the SERPs based on the demographic characteristics.

1. Introduction

Search engines like Google (with a huge system at the background) creates miracles by understanding what the user seeks with one or a few keywords and by serving the information to the user. In the beginning of 2012, it was found out that use of search engine was one of the most popular online activities in the United States for more than a decade, furthermore, it was found out that 91% of online American adults used search engines in order to find information on the Web [1]. Each of the search engines (like Google) has their own simple views. Besides, they have common components. One of them is Search Engine Results Pages (SERPs). “SERPs” is a set of Web page(s) that contains result(s) for a single query run. The users examine the SERPs to reach the information they need. Each user has individual behaviors on the SERPs. The studies regarding to user behaviors on the SERPs are valuable since they could have contribution on search engine improvement. This survey-based study is aimed to make an analysis of search engine users’ behaviors on the SERPs based on gender, age, and program studying.

The objectives are as follows: (1) To examine user behaviors on the usage of the SERPs based on gender; (2) to examine user behaviors on the usage of the SERPs based on age; and (3) to examine user behaviors on the usage of the SERPs based on program studying.

In this study, behavior represents a possible action of person on the SERPs such as “rarely advertisement click” and “6-10 results click”. The questionnaire was designed with twelve closed-ended questions in order to investigate the objectives and reveal answers. In the questionnaire, 3 questions were for the Demographic Characteristics (DCs); 8 questions were designed as five-level Likert-type items by considering the behavior templates “Never … Click”, “Rarely … Click”, “Occasionally … Click”, “Frequently … Click”, and “Very Frequently … Click” and they were about “tab”, “advertisement”, “spelling suggestion”, “related query suggestion”, “instant search suggestion”, “video result”, “image result”, and “pagination”; and one question was designed for the amount of clicking results by considering the behavior templates “Less-than-6 … Click”, “6-to-10 … Click”, “11-to-15 … Click”, “16-to-20 … Click”, and “More-than-20 … Click”. The questionnaire was used and the data collected were analyzed.

In the literature, there are studies related to this study. However, to best knowledge of the authors, this study would be the first attempt to have a survey-based study which makes an analysis of search engine users’ behaviors on the usage of the SERPs based on gender, age, and program studying with all the question topics and designs mentioned above. In order to have a support for this, some of the related studies are stated briefly in the following paragraphs. (Each one of the paragraphs consists of corresponding study oriented as well as independent explanations; but, the last corresponding paragraph consists of corresponding studies oriented as well as independent explanations with additional explanations about the relation to the authors’ study.)

Liu et al. [2] ran a three-session field-lab-combined study for examining the effects of user education from two separate sources on Web search behavior in two different search task scenarios. They also investigated if the behavioral effects continue firmly when the explicit search tips are removed. In this study, 185 task session data obtained from 31 undergraduate student participants in two fields and one lab sessions were used.

Sachse [3] investigated which snippet length on a SERP should be used in mobile Web search. Total 31 participants took part in this study. An eye-tracking experiment was conducted to show them search interfaces with snippets of one-line, three-lines, or five-lines on the mobile device in order to analyze their behaviors with 17 dependent variables.

Höchstötter and Lewandowski [4] made an investigation on the composition of SERPs. They run 500 popular and 500 rare type queries on the search engines Google, Yahoo, Live.com (this is Bing now), and Ask. They counted how often the defined results pages’ elements (e.g. organic results, advertisements) were used by each search engine and analyzed the data collected

Kumar [5], in his survey-based study, investigated the influence of user DCs (i.e. age, gender, academic majors, and academic attainment category) on the effect of Web searching on OPAC (Online Public Access Catalogue) usage. The objectives of this study were to reveal the relationships between user DCs and: OPAC usage pattern; awareness about information processing of OPAC and Web search engines; viewpoints after unsuccessful searches; or actions after unsuccessful searches. Thus, Kumar collected the data from questionnaire respondents and analyzed them statistically.

Niu and Kelly [6], addressed when the users use query suggestions, how it differs (according to search experience, temporal search segment, and topic difficulty), and its effect on the number of relevant documents issues with 23 student participants. They used a TREC collection as a corpus and four search topics selected manually from the collection. Participants integrated query suggestions into their searching rather quickly and participants used more query suggestions on topics that are more difficult.

Tomasi [7] investigated behavior of users on different SERPs in order to determine the most useful scent types to reach relevant information to complete tasks on the Web based on information foraging theory. Tomasi developed three interface prototypes: The first one was for providing Web results in an ordered list like SERPs of Google; the second one was for displaying categories by using a network tree; and the last one was for displaying categories as circles by using a map display through visualization. 15 participants were asked to complete seven tasks and their voices and actions were recorded. Keywords of categories, titles of categories or Web sites, orientation of results, and animation were found as strong scents.

Four objectives were taken into consideration by Kumar and Kumar [8]. These objectives considered the Indian academics, who were the faculty members and students of selected medical, engineering, and management colleges in India. These objectives were as follows: To know the place and purpose of Internet use; to examine the use of various search engines and meta search engines; to know if they use search strategy of various search engines; and to know the method of learning search strategies of various search engines. The data collection was based on questionnaires and follow-up interviews. 300 duly filled out questionnaires were collected (66.66% response rate). 91.93% respondents most frequently used Google and 65.26% respondents used the search strategy for information retrieval. Engineering respondents learnt the search strategies of search engines by discussing friends/colleagues (43.33%) followed by using the help message in the search tool (33.33%). It is the same for medical respondents (with 33.33% and 27.77%, respectively) and for management respondents (with 38.88% and 31.94%, respectively).

Alshare et al. [9] aimed to reveal perceptions of the students regarding their use of search engines. They delivered 600 questionnaires and 330 of them were returned. 93.6% respondents used search engines for class-related activities. Females were more likely than males to use the engines for retrieving travel information. Business students were more likely to use advanced search techniques compared to non-business students. Non-business students were more likely to use help features than business students.

In the study of Fu and Karan [10], in order to investigate Chinese college students’ search behavior on the search engines Baidu and Google, 257 Chinese college students studying in China and abroad filled out an online questionnaire at the beginning of 2014. The questionnaire was comprised of 27 questions and fill out process was completed approximately in two months. The first factor to Baidu’s popularity was “familiarity with the interface” and the last was “less censorship”, while the first factor to Google’s popularity was “best results in English” and the last was “multi-functions at one site”. Fu and Karan stated that “The longer they studied abroad and the more educated they were, the more likely students would prefer Google to Baidu”.

Maghferat and Stock [11] determined 5 research questions, which was about the selection of online services (i.e. Web search engines and professional information services in the deep Web), operator use, search term modifications, targeted search, and satisfaction with search results, and worked to analyze information search behavior based on gender DC. The data were collected with the questionnaires delivered to 26 males and 19 female information science students. Women were generally more satisfied with the search results.

In Zhou’s study [12], 107 undergraduate students filled out the questionnaire regarding the perception about the Web usage for learning purposes. Afterwards, they searched online with search engine in order to answer two specific questions. While they were searching, their search activities (including the type of activities during search, the frequency of each activity, and the time spent on each activity) were logged. Zhou compared participants by gender in terms of their Web search efficacy, Web search anxiety, frequency counts of different Web search activities (e.g. number of searches, number of Web pages viewed), time spent on each search activity, and search task performance.

Ménard and Khashman [13] stated three stages of a research project that aimed to develop a bilingual (English and French) interface for digital image retrieval. In the first stage, they explored best practices for image retrieval. In the second stage, they investigated the roles and usefulness of the search characteristics and functionalities from the user’s point of view for the image search in a bilingual context. For the final stage, they claimed that they will develop the search interface model based on the previous two stages. In the second stage, a survey (consisted of questionnaire and semi-structured interview) was done. 23 closed-ended questions about the characteristics and functionalities identified in the first stage were designed with Likert scale responses. These questions were in six categories as follows: “Search habits” (2 questions); “query types” (5 questions”); “search functionalities” (6 questions”); “colour functionalities” (4 questions”); “image searching and languages” (3 questions”); and “image searcher satisfaction” (3 questions”). 8 open-ended questions were also included in the questionnaire in order to get verbal descriptions of the users own image search process. 20 English speaking and 20 French speaking respondents filled out the questionnaire. Furthermore, semi-structured interviews were conducted. The details of the interview and the results of the interview analysis were presented in another paper by Ménard et al. [14]. Note that only the study of Ménard and Khashman [13] and the study of Ménard et al. [14] were presented in regarding the research project; besides, [13] was fully accessed while [14] was partially accessed. When considering [13] and [14] based on the content as well as the design, only spelling-suggestion-related question from the 23 questions is related with the authors’ study’s spelling-suggestion-related question. This content-as-well-as-design-based relation mentioned above is that this question and its answer options are similar with the authors’ study’s spelling-suggestion-related question and its answer options. However, unlike the main results of the question in the authors’ study, this question’s results are not based on DCs. Furthermore, this question is about image retrieval behavior. A result of this question is that 47.5% of the respondents strongly agreed or agreed 2844 Bitirim et al.: An analysis of user behaviors on the that it is helpful when the system checks spelling/entry and asks “Do you mean …?” Note that they stated that participants had Google Images in their minds while answering the survey questions. (Only this study was encountered in the literature that has a content-as-well-as-design-based relation with the authors’ study.)

With this study, search engine providers and researchers would gain knowledge about user behaviors on the usage of the SERPs based on the DCs. This could be useful for search engine improvement. The purpose of the next section is to describe the methodology of this study in details. Then, the “results and discussion” section will follow. The last section will present the conclusion part.

2. Methodology

2.1 The list related to usage of the SERPs

There are various items on the SERPs that are available to be used by the user. The items were grouped for each of the popular international search engines Google, Bing, and Yahoo. (It is worth to share that, during the report stage of this study, it was encountered that the total worldwide usage trend for all these search engines was given as more than 96% for each platform (i.e. desktop, mobile, tablet, and console) for each year from 2013 to 2015 by StatCounter (http://gs.statcounter.com - accessed on 25 January 2016)). Then, common groups were determined by considering purposes (instead of item lists). Afterwards, a part of the common groups was selected to be used in this study. The selected groups, named as “tab”, “advertisements”, “spelling suggestions”, “related query suggestions”, “instant search suggestions”, “video results”, “image results”, and “pagination”, constituted the list related to usage of the SERPs.

2.2 Study population

Based on the list of registered computer engineering students taken from the computer center at Eastern Mediterranean University as of the date March 14, 2014, there were 385 active undergraduate students from various countries. “Active undergraduate student” refers to the undergraduate student who registered for one of the Eastern Mediterranean University’s Computer Engineering (CMPE) department programs consisting of English CMPE, Turkish CMPE, and software engineering (SE) programs and taken at least one actual course from the department. (“Actual course” term is used to represent any more-than-zero-credited undergraduate course excluding graduation project courses.) The reason for at-least-one-actual-course-taken issue was that students would be needed to be available in the department. Note that the authors work in this department.

It could be thought that these 385 active undergraduate students were familiar with the search engines since they were CMPE department students. Furthermore, they were directly and easily reachable for the authors. Because of these, the mentioned 385 students constituted the study population. 173, 146, and 66 students of the study population were from English CMPE, Turkish CMPE, and SE programs, respectively. (In the department, there were a few double major program students who were firstly registered to English CMPE or SE and, then, admitted for doing double major in the other program (English CMPE/SE). These students were considered for the program they registered first.) 47 of them were female and 338 of them were male. The minimum age was 15 (one student). The maximum age was 36 (one student), 31 came next (one student), and the rest was 30 or below. Instead of using sampling method, it was tried to reach to whole study population.

2.3 Demographic Characteristics (DCs)

The three DCs gender, age, and program studying were used in this study. Categories were determined for each DC. The DC categories female and male were used for gender; the DC categories under-20, 20-24, and above-24 were used for age; and the DC categories English CMPE, Turkish CMPE, and SE were used for program studying.

2.4 Questionnaire

As it is demonstrated on “https://doi.org/10.6084/m9.figshare.12377156.v1”, the questionnaire consisted of twelve closed-ended questions which were determined to investigate the objectives and reveal answers.

Questions 1, 2, and 3 were for the DC gender with female and male categories, the DC age with under-20, 20-24, and above-24 categories, and the DC program studying with English CMPE, Turkish CMPE, and SE categories, respectively. (Note that, in question 3, the fourth answer option was used for those respondents who were in double major program. If a respondent marked this option, the respondent was asked to mention her/his program registered first and this program, which is equal to one of the first three answer options, was considered as answer of question 3.)

The rest of the questions (from 4 to 12) were determined to explore the usage of the SERPs. In the questionnaire, it was asked from the respondents to answer these questions by considering their own most frequently using search engines’ results pages come after query run, in order to obtain actual behavior of each respondent. The questions from 4 to 11 were based on the list related to usage of the SERPs. Each of these questions was designed as five-level Likert-type item by using the following answer option format: (1) never; (2) rarely; (3) occasionally; (4) frequently; and (5) very frequently. However, the sixth answer option was also added to each of these questions for those respondents who did not see the feature asked in her/his search engine, use a search engine which does not has the feature asked, or do not know if her/his search engine has the feature asked. Furthermore, these questions were supported with sample screenshots from the popular international search engines Google, Bing, and Yahoo in order to make sure that the respondents understand the meaning of the questions. Question 12 was for the amount of clicking results on the SERPs and designed with five answer options representing the corresponding amounts.

2.5 The process for performing filling out of the questionnaires

Due to these students were attending laboratory sessions more attentive, it was decided to use laboratory sessions for trying to reach all the study population. There were 37 groups of actual courses offered in the department at that semester and all had laboratory sessions. Instead of attending to all 37 laboratory sessions, it was tried to decrease the number of actual course and group pairs’ laboratory sessions for the time saving while preserving the coverage of whole the study population. Simple software was developed to be used in this task.

All class rosters of 37 groups of actual courses taken from the Microsoft-Windows-operating-system-based-registration-software of the university on March 14, 2014 were transferred to the developed software; however, those students who were not in the list of 385 students -referring to not active undergraduate students- were removed from the class rosters before the transfer. (These were some graduate students of the CMPE department and some undergraduate students from another department.) Only one anomaly was encountered here. One student from a class roster who should have been an active undergraduate student was not in the list of 385 students. This student’s information was checked via student information search page on the online portal of the university and it was seen that this student status was “left”. Therefore, it was decided to remove this student as well. After 974 records were transferred to the developed software, 22 groups of actual courses were manually selected with the help of the developed software by considering the courses, groups, and professors. By selecting about 60% of all groups’ of actual courses, time saving has been provided in this study.

A time table was prepared for the laboratory sessions for selected 22 groups of actual courses, after the necessary permissions were taken from the department chair’s office as well as the professors. The time table was checked and an appropriate laboratory session was chosen for the pilot study. Then, the pilot study was conducted with 18 respondents on April 2, 2014. Since no problem was encountered in the pilot study, the filled out questionnaires were included in the study and, then, the other visits scheduled on the time table were started. All visits were finished on May 22, 2014. It was taken more time than expected due to several reasons (e.g. laboratory sessions were not done during the mid-term exams period).

Two active undergraduate students were accepted to volunteer in helping some of the visits. Some visits were done with at least one of the volunteer students; in some visits, help was provided by laboratory assistant(s); and in some, the visits were achieved as one person. However, in every visit, the same procedure was applied. This procedure was as follows:

— Step 1: Present pre-questionnaire speech to everyone in order to prevent graduate students (if any) and non-CMPE-department-students (if any) filling out the questionnaire and, additionally, the students who were already filled out the questionnaire before (if any).

— Step 2: For Turkish CMPE students, deliver the questionnaire in Turkish language. For English CMPE and SE students, deliver the questionnaire either in Turkish language or in English language based on student’s preference. Note that: education language is in English for both English CMPE and SE programs; however, the questionnaires in Turkish language are already prepared for Turkish CMPE students, due to this, the questionnaires in Turkish language can also be used for English CMPE and SE students according to their preference.

— Step 3: Help students during the filling out if it is requested.

— Step 4: Collect the questionnaires.

2.6 Hypotheses

Additionally, nine null hypotheses (H0) were determined for each objective. The nine hypotheses for the first objective, which is related to gender DC, are as follows:

— There is no relationship between gender and tab click behavior.

— There is no relationship between gender and advertisement click behavior.

— There is no relationship between gender and spelling suggestion click behavior.

— There is no relationship between gender and related query suggestion click behavior.

— There is no relationship between gender and instant search suggestion click behavior.

— There is no relationship between gender and video result click behavior.

— There is no relationship between gender and image result click behavior.

— There is no relationship between gender and pagination click behavior.

— There is no relationship between gender and amount of results click behavior.

Also, the aforementioned nine hypotheses were used for the second objective related to age DC by replacing the word “gender” with the word “age” as well as for the third objective related to program studying DC by replacing the word “gender” with the words “program studying”.

2.7 Data analysis

In this study, the following behavior templates were considered: “Never … Click”, “Rarely … Click”, “Occasionally … Click”, “Frequently … Click”, and “Very Frequently … Click” for questions 4-11; “Less-than-6 … Click”, “6-to-10 … Click”, “11-to-15 … Click”, “16-to-20 … Click”, and “More-than-20 … Click” for question 12.

The collected questionnaires were examined one by one and a collective data table was prepared accordingly. Afterwards, the data table was used in order to figure out the frequency for each DC category and behavior pair for each question (4 to 12). Note that the followings are for each of the aforementioned questions. Each frequency presents the number of respondents in the corresponding DC category that chose the corresponding behavior. Every frequency was converted into percentage with the consideration of all frequencies in the same DC category. The percentages are called behavior values (BVs).

The followings were also done for each of the aforementioned questions: Total-frequency was calculated for each DC and behavior pair. Furthermore, every total-frequency was converted into total-frequency-percentage with the consideration of all total-frequencies in the same DC; in each DC, BV difference was calculated between every DC category pair for each behavior and, then, average BV difference was calculated for every DC category pair.

Remarks about frequencies, total-frequencies, BVs, BV differences, average BV differences, total-frequency-percentages, total-frequency-percentage-sums, and behavior-value-sums used in the study are as follows:

— The data table was prepared in Microsoft Excel 2010 and frequencies were figured out by using this data table. Furthermore, total-frequencies were calculated from frequencies. All were carried out in Microsoft Excel 2010.

— BVs were calculated from frequencies, then, by using unformatted (non-rounded) versions of these BVs, BV differences were calculated, and afterwards, by using unformatted (non-rounded) versions of these BV differences, average BV differences were calculated. All the calculations were carried out in Microsoft Excel 2010. Then, each of BVs and average BV differences in Microsoft Excel 2010 were rounded to one decimal place in Microsoft Excel 2010 and considered in this format.

— In Microsoft Excel 2010, total-frequency-percentages were calculated from total-frequencies. Afterwards, they were rounded to one decimal place in Microsoft Excel 2010 and considered in this format. Be aware that in some cases, sum of some total-frequency-percentages was needed. While calculating the sum, unformatted (non-rounded) versions of total-frequency-percentages were used instead of the formatted (rounded to one decimal place) versions. After the sum was calculated in Microsoft Excel 2010, it was rounded to one decimal place in Microsoft Excel 2010 and considered in this format.

— In some cases, sum of some BVs was needed. (E.g. the sum was used and the result “90.5% click advertisement rarely or none” was obtained.) While calculating the sum, unformatted (non-rounded) versions of BVs were used instead of the formatted (rounded to one decimal place) versions. After the sum was calculated in Microsoft Excel 2010, it was rounded to one decimal place in Microsoft Excel 2010 and considered in this format.

— For each result value which unformatted (non-rounded) versions of the corresponding inputs were used, if formatted (rounded to one decimal place) versions of the corresponding inputs are considered, slight difference can be seen.

Furthermore, for response rate and percentages about number of respondents, formatted (rounded to one decimal place) versions were considered.

In addition to the descriptive statistics, the Chi-square test of independence was applied to test the aforementioned hypotheses of no association. The value of p < 0.05 was considered as statistically significant. In the study of Sirkin [15], “The Chi-square test for contingency flowchart” is given. The method below is modified version of it and was followed by the authors.

A Chi-square test was considered as valid when no more than 20% of the expected counts were less than 5 and all individual expected counts were 1 or greater. In the case where this requirement was violated, grouping DC categories and/or grouping adjacent behaviors were applied. During grouping, combinations were evaluated one by one until this shortcoming was overcome as well as a significant relationship was obtained. Nevertheless, when not achieved, 2-by-2 contingency tables which have at least one expected count less than 5 (25% or more of expected counts less than 5) were evaluated one by one again with the Fisher’s Exact Test in order to find out a significant relationship.

SPSS (version 18.0.0) was used for the inferential statistics. Every corresponding result value was considered in the format it was presented by SPSS. Note that the same type result values had the same format, e.g. p values were rounded to three decimal places.

3. Results and Discussion

The number of respondents was 326 in total. However, one of the respondents filled out only one of the DCs in 12 questions of the questionnaire and another respondent filled out only two of the DCs in 12 questions of the questionnaire. These two respondents’ questionnaires were excluded from the study due to the excessive missing data. Thus, the total number of respondents has been accepted as 324, which means 84.2% of the study population was reached.

Table 1 presents the number of respondents for the DCs gender, age, and program studying separately in detail. Four respondents (1.2%) were given no answer for the DC age and another respondent (0.3%) had an incorrect marking for the DC program studying. These respondents were also considered in the percentage calculations of the DC categories as it is shown in Table 1. 87.0% of the 324 respondents were male. In terms of age, more than half of the respondents (58.3%) were between 20 and 24. Most of the respondents (45.7%) were from English CMPE program. In four questionnaires, no answer was given for age. These questionnaires were excluded when age was considered. In another questionnaire, an incorrect marking was done while answering program studying. This questionnaire was excluded when program studying was considered. Sixth answer option was marked for questions 4, 5, 7, 8, and 11 by 4, 1, 4, 2, and 2 respondents, respectively. The sixth-answer-option-marked answer(s) were excluded when the corresponding question was considered. No answer was given for questions from 6 to 12 by 3, 3, 7, 4, 4, 9, and 8 respondents, respectively. The empty answers were excluded when the corresponding question was considered.

Table 1. Number of respondents for each DC in detail.

In Table 2, frequency and BV of every DC category and behavior pair for the answers of question 4 are shown. Female category and male category have their own highest BVs at occasionally tab click. The average BV difference of female category and male category is 5.3%. 20-24 and above-24 categories have their own highest BVs at occasionally tab click and under-20 category has its own at rarely tab click. The highest average BV difference belongs to under-20 and 20-24 pair with 8.2%. Under-20 and above-24 pair is the next with 5.6% and then, 20-24 and above-24 pair comes with 5.4%. SE category has its own highest BV at frequently tab click and the others have their own at occasionally tab click. The highest average BV difference is between Turkish CMPE and SE categories with 10.3%. English CMPE and SE pair comes next with 10.2% and then, English CMPE and Turkish CMPE pair follows with 5.1%. According to the Chi-square test results: There is a significant relationship between program studying and tab click behaviour (χ² ( 8 , N = 319 ) = 22.571, p = 0.004, with 6.7% of the expected counts < 5 and the minimum expected count = 4.08); there is no significant relationship between gender and tab click behaviour (χ² ( 4 , N = 320 ) = 2.965, p = 0.564, with 10.0% of the expected counts < 5 and the minimum expected count = 3.41) and between age and tab click behaviour (χ² ( 8 , N = 316 ) = 12.595, p = 0.127, with 6.7% of the expected counts < 5 and the minimum expected count = 4.11).

Table 2. Frequencies and BVs of the answers to question 4 (Do you click tab (representing collection name) when appears?)

Note: “TC” stands for “Tab Click”.

Table 3 presents frequency and BV of every DC category and behavior pair for the answers of question 5. Female category has its own highest BV at rarely advertisement click and male category has its own at never advertisement click. The average BV difference of female category and male category is 10.1%. Each age category has its own highest BV at never advertisement click. The highest average BV difference is between under-20 and above-24 pair with 4.3%. 20-24 and above-24 pair comes next with 2.6% and then, under-20 and 20-24 pair follows with 2.4%. English CMPE, Turkish CMPE, and SE categories have their own highest BVs at never advertisement click. The greatest average BV difference is between English CMPE and SE categories with 7.9%. Turkish CMPE and SE pair comes next with 5.6% and afterwards, English CMPE and Turkish CMPE pair follows this with 3.9%. The Chi-square test could not be directly applied for gender since the requirement was violated (30.0% of the expected counts < 5 and the minimum expected count = 0.13). When grouping the behaviours as “Never advertisement click”, “Rarely advertisement click”, and “Occasionally or more advertisement click”, there is a significant relationship between gender and advertisement click behaviour (χ² ( 2 , N = 323 ) = 11.264, p = 0.004, with 0.0% of the expected counts < 5 and the minimum expected count = 7.15). The Chi-square test could not be directly applied for program studying since the requirement was violated (40.0% of the expected counts < 5 and the minimum expected count = 0.16). When grouping the behaviours as “Never advertisement click”, “Rarely advertisement click”, “Occasionally advertisement click”, and “Frequently or more advertisement click” and grouping the DC categories as “CMPE (English and Turkish)” and “SE”, there is a significant relationship between program studying and advertisement click behaviour (χ² ( 3 , N = 322 ) = 12.562, p = 0.006, with 12.5% of the expected counts < 5 and the minimum expected count = 1.11). The Chi-square test could not be directly applied for age since the requirement was violated (40.0% of the expected counts < 5 and the minimum expected count = 0.16). Grouping DC categories and/or grouping adjacent behaviours were applied; however, although all combinations appropriate to the test were considered, no significant relationship was encountered between age and advertisement click behaviour.

Table 3. Frequencies and BVs of the answers to question 5 (Do you click advertisement when appears?)

Note: “AC” stands for “Advertisement Click”.

Table 4 demonstrates frequency and BV of every DC category and behavior pair for the answers of question 6. Female category and male category have their own highest BVs at frequently spelling suggestion click. The average BV difference of female category and male category is 4.5%. Each age category has its own highest BV at frequently spelling suggestion click. The biggest average BV difference is between under-20 and 20-24 pair with 8.4%. Under-20 and above-24 pair is the next with 7.3% and then, 20-24 and above-24 pair follows with 4.4%. English CMPE and Turkish CMPE categories have their own highest BVs at frequently spelling suggestion click and SE group has its own at very frequently spelling suggestion click. The highest average BV difference is between Turkish CMPE and SE pair with 10.2%. English CMPE and SE pair comes next with 6.6% and then, English CMPE and Turkish pair comes with 5.1%. According to the Chi-square test results: There is a significant relationship between age and spelling suggestion click behaviour (χ² ( 8 , N = 317 ) = 17.588, p = 0.025, with 13.3% of the expected counts < 5 and the minimum expected count = 2.74); there is no significant relationship between gender and spelling suggestion click behaviour (χ² ( 4 , N = 321 ) = 3.556, p = 0.469, with 20.0% of the expected counts < 5 and the minimum expected count = 2.17) and between program studying and spelling suggestion click behaviour (χ² ( 8 , N = 320 ) = 14.733, p = 0.065, with 6.7% of the expected counts < 5 and the minimum expected count = 2.66).

Table 4. Frequencies and BVs of the answers to question 6 (Do you click spelling suggestion when appears?)

Note: “SSC” stands for “Spelling Suggestion Click”.

In Table 5, frequency and BV of every DC category and behavior pair for the answers of question 7 are presented. Female category and male category have their own highest BVs at occasionally related query suggestion click. The average BV difference of them is 6.6%. Each age category has its own highest BV at occasionally related query suggestion click. The highest average BV difference is between 20-24 and above-24 age categories with 2.5%. Under-20 and above-24 pair comes next with 2.2% and afterwards, under-20 and 20-24 pair comes with 1.9%. Each program studying category has its own highest BV at occasionally related query suggestion click. The highest average BV difference is between English CMPE and SE categories with 8.2%. Then, Turkish CMPE and SE pair follows with 7.9% and after that, English CMPE and Turkish CMPE pair comes with 1.4%. According to the Chi-square test results: There is a significant relationship between program studying and related query suggestion click behaviour (χ² ( 8 , N = 316 ) = 21.560, p = 0.006, with 20.0% of the expected counts < 5 and the minimum expected count = 1.24); there is no significant relationship between gender and related query suggestion click behaviour (χ² ( 4 , N = 317 ) = 5.323, p = 0.256, with 20.0% of the expected counts < 5 and the minimum expected count = 1.01) and between age and related query suggestion click behaviour (χ² ( 8 , N = 314 ) = 2.591, p = 0.957, with 20.0% of the expected counts < 5 and the minimum expected count = 1.09).

Table 5. Frequencies and BVs of the answers to question 7 (Do you click related query suggestion when appears?)

Note: “RQSC” stands for “Related Query Suggestion Click”.

Table 6 presents presents frequency and BV of every DC category and behaviour pair for the answers of question 8. Each gender category has its own highest BV at frequently instant search suggestion click. The average BV difference between female category and male category is 7.3%. Each age category has its own highest BV at frequently instant search suggestion click. The greatest average BV difference is between 20-24 and above-24 pair with 4.7%. Under-20 and above-24 pair comes next with 4.1% and then, under-20 and 20-24 pair follows with 2.2%. English CMPE and Turkish CMPE categories have their own highest BVs at frequently instant search suggestion click and SE category has its own at both frequently instant search suggestion click and very frequently instant search suggestion click. The greatest average BV difference is between Turkish CMPE and SE pair with 6.7%. English CMPE and SE pair comes next with 6.1% and then, English CMPE and Turkish CMPE pair follows with 5.3%. The Chi-square test could not be directly applied for program studying since the requirement was violated (26.7% of the expected counts < 5 and the minimum expected count = 1.53). When grouping the behaviours as “Occasionally or less instant search suggestion click”, “Frequently instant search suggestion click”, and “Very frequently instant search suggestion click”, there is a significant relationship between program studying and instant search suggestion click behaviour (χ² ( 4 , N = 314 ) = 10.077, p = 0.039, with 0.0% of the expected counts < 5 and the minimum expected count = 13.30). According to the Chi-square test results, there is no significant relationship between gender and instant search suggestion click behaviour (χ² ( 4 , N = 315 ) = 5.635, p = 0.228, with 20.0% of the expected counts < 5 and the minimum expected count = 1.21) and between age and instant search suggestion click behaviour (χ² ( 8 , N = 311 ) = 5.979, p = 0.650, with 20.0% of the expected counts < 5 and the minimum expected count = 1.42).

Table 6. Frequencies and BVs of the answers to question 8 (Do you click instant search suggestion when appears?)

Note: “ISSC” stands for “Instant Search Suggestion Click”.

Table 7 shows frequency and BV of every DC category and behaviour pair for the answers of question 9. Female category and male category have their own highest BVs at occasionally video result click. The average BV difference of them is 6.9%. Each age category has its own highest BV at occasionally video result click. The highest average BV difference belongs to under-20 and 20-24 pair with 1.6%. 20-24 and above-24 pair comes next with 1.5% and afterwards, under-20 and above-24 pair follows with 1.4%. English CMPE and Turkish CMPE categories have their own highest BVs at occasionally video result click and SE category has its own at frequently video result click. The highest average BV difference is between English CMPE and SE pair with 6.1%. Turkish CMPE and SE pair comes after with 4.0% and then, English CMPE and Turkish CMPE pair comes with 2.5%. According to the Chi-square test results, there is no significant relationship between gender and video result click behaviour (χ² ( 4 , N = 320 ) = 4.689, p = 0.321, with 20.0% of the expected counts < 5 and the minimum expected count = 1.71), between age and video result click behaviour (χ² ( 8 , N = 316 ) = 1.164, p = 0.997, with 13.3% of the expected counts < 5 and the minimum expected count = 1.78), and between program studying and video result click behaviour (χ² ( 8 , N = 319 ) = 5.366, p = 0.718, with 6.7% of the expected counts < 5 and the minimum expected count = 2.19).

Table 7. Frequencies and BVs of the answers to question 9 (Do you click video result when appears?)

Note: “VRC” stands for “Video Result Click”.

Table 8 demonstrates frequency and BV of every DC category and behaviour pair for the answers of question 10. Each gender category has its own highest BV at occasionally image result click. The average BV difference between female category and male category is 7.5%. Each age category has its own highest BV at occasionally image result click. The maximum average BV difference is between under-20 and above-24 pair with 2.9%. 20-24 and above-24 pair comes next with 2.1% and then, under-20 and 20-24 pair follows with 1.4%. English CMPE and Turkish CMPE categories have their own highest BVs at occasionally image result click and SE category has its own at both occasionally image result click and frequently image result click. The greatest average BV difference is between English CMPE and SE categories with 5.5%. Turkish CMPE and SE pair comes next with 5.0% and afterwards, English CMPE and Turkish CMPE pair follows this with 2.7%. The Chi-square test could not be directly applied for gender since the requirement was violated (30.0% of the expected counts < 5 and the minimum expected count = 0.98). When grouping the behaviours as “Rarely or less image result click”, “Occasionally image result click”, “Frequently image result click”, and “Very frequently image result click”, there is a significant relationship between gender and image result click behaviour (χ² ( 3 , N = 320 ) = 7.987, p = 0.046, with 12.5% of the expected counts < 5 and the minimum expected count = 4.51). According to the Chi-square test results, there is no significant relationship between age and image result click behaviour (χ² ( 8 , N = 316 ) = 2.011, p = 0.981, with 20.0% of the expected counts < 5 and the minimum expected count = 1.13) and between program studying and image result click behaviour (χ² ( 8 , N = 319 ) = 7.915, p = 0.442, with 20.0% of the expected counts < 5 and the minimum expected count = 1.25).

Table 8. Frequencies and BVs of the answers to question 10 (Do you click image result when appears?)

Note: “IRC” stands for “Image Result Click”.

In Table 9, frequency and BV of every DC category and behaviour pair for the answers of question 11 are presented. Female category has its own highest BV at rarely pagination click and male category has its own at frequently pagination click. The average BV difference of female category and male category is 5.3%. 20-24 and above-24 age categories have their own highest BVs at frequently pagination click and under-20 age category has its own at rarely pagination click. The highest average BV difference belongs to under-20 and above-24 pair with 15.5%. Under-20 and 20-24 pair is the next with 11.3% and then, 20-24 and above-24 pair comes with 7.2%. English CMPE category has its own highest BV at rarely pagination click. The others have their own at frequently pagination click. The highest average BV difference is between English CMPE and Turkish CMPE pair with 7.8%. English CMPE and SE pair comes next with 6.0% and then, Turkish CMPE and SE pair follows with 4.7%. According to the Chi-square test results: There is a significant relationship between age and pagination click behaviour (χ² ( 8 , N = 309 ) = 26.839, p = 0.001, with 13.3% of the expected counts < 5 and the minimum expected count = 1.64); there is no significant relationship between gender and pagination click behaviour (χ² ( 4 , N = 313 ) = 3.221, p = 0.522, with 10.0% of the expected counts < 5 and the minimum expected count = 1.53) and between program studying and pagination click behaviour (χ² ( 8 , N = 312 ) = 14.125, p = 0.079, with 13.3% of the expected counts < 5 and the minimum expected count = 1.81).

Table 9. Frequencies and BVs of the answers to question 11 (Do you click pagination when appears?)

Note: “PC” stands for “Pagination Click”.

Table 10 presents frequency and BV of every DC category and behaviour pair for the answers of question 12. Each gender category has its own highest BV at 6-to-10 results click. The average BV difference between female category and male category is 3.9%. 20-24 and above-24 age categories have their own highest BVs at 6-to-10 results click and under-20 age category has its own at less-than-6 results click. The highest average BV difference belongs to under-20 and above-24 pair with 4.1%. Under-20 and 20-24 pair is the next with 3.4% and then, 20-24 and above-24 pair comes with 1.2%. Each program studying category has its own highest BV at 6-to-10 results click. The greatest average BV difference is between Turkish CMPE and SE pair with 5.2%. English CMPE and SE pair comes next with 4.6% and then, English CMPE and Turkish CMPE pair follows with 2.2%. According to the Chi-square test results, there is no significant relationship between program studying and amount of results click behaviour (χ² ( 8 , N = 315 ) = 8.210, p = 0.413, with 20.0% of the expected counts < 5 and the minimum expected count = 1.83). The Chi-square test could not be directly applied for gender as well as for age since the requirement was violated at each (for gender, 30.0% of the expected counts < 5 and the minimum expected count = 1.56 and for age, 26.7% of the expected counts < 5 and the minimum expected count = 1.85). Grouping DC categories and/or grouping adjacent behaviours were applied both for gender and for age; however, although all combinations (appropriate to the test) for each were considered, a significant relationship was encountered neither between gender and amount of results click behaviour nor between age and amount of results click behaviour. Note that, while a respondent answered this question, s/he considered the items that were the results from her/his own perspective.

Table 10. Frequencies and BVs of the answers to question 12 (How many results do you click?)

Note: “RC” stands for “Results Click”.

For questions 4 and 6-10, the highest BV of each DC category is at occasionally, frequently, very frequently, occasionally & frequently, or frequently & very frequently click except under-20 age category in question 4 which is at rarely click; for question 5, every DC category’s highest BV is at never click except female category which is at rarely; for question 11, every DC category’s highest BV is at rarely or frequently click; for question 12, the highest BV of each DC category is at less-than-6 or 6-to-10 results click.

When all questions are considered, BV range is 0.0% (frequently advertisement click - female category, very frequently advertisement click - female category, and very frequently related query suggestion click - female category) & 54.8% (never advertisement click - male category) for gender, 0.0% (very frequently advertisement click - under-20 category, frequently advertisement click - above-24 category, and very frequently advertisement click - above-24 category) & 53,8% (never advertisement click - under-20 category) for age, and 0.0% (very frequently advertisement click - English CMPE category, frequently advertisement click - Turkish CMPE category, very frequently advertisement click - Turkish CMPE category, 16-to-20 results click - SE category, and never instant search suggestion click - SE category) & 57.8% (never advertisement click - English CMPE category) for program studying.

The majority of each DC category have rarely or more click for tab, spelling suggestion, related query suggestion, instant search suggestion, video result, image result, and pagination and have 6 or more results click. For advertisement, most of female category, most of SE category, and half of Turkish CMPE category have rarely or more click.

50.0% or above BV at single behavior is shown for some DC category and question pairs: For male category, all age categories, and English CMPE category, more than 50.0% do not click (never click) advertisement; for Turkish CMPE category, 50.0% do not click (never click) advertisement; more than 50.0% of female category click advertisement rarely; 50.0% of female category click instant search suggestion frequently; and half of SE category click 6-to-10 results.

For all DC categories, the common characteristics that each consists of majority of respondents (for each category) as well as single behavior are as follows:

— For related query suggestion, the highest BV, which is between 35.0% and 44.9%, is at occasionally click.

— For instant search suggestion, the highest BV, which is between 35.2% and 50.0%, is at frequently click. Note that SE has its own highest BV at very frequently as well.

— For image result, the highest BV, which is between 38.0% and 46.2%, is at occasionally click. Note that SE has its own highest BV at frequently as well.

For all DC categories, the common characteristics that each consists of more than 50.0% respondents (for each category) as well as two behaviors are as follows:

— Between 79.7% and 90.5% click advertisement rarely or none.

— Between 52.5% and 68.8% click related query suggestion rarely or occasionally.

— Between 60.7% and 67.5% click related query suggestion occasionally or frequently.

— Between 54.2% and 68.4% click instant search suggestion occasionally or frequently.

— Between 62.3% and 79.2% click instant search suggestion frequently or very frequently.

— Between 64.1% and 74.0% click video result occasionally or frequently.

— Between 69.2% and 77.2% click image result occasionally or frequently.

— Between 50.0% and 62.5% click from 6 to 15 results.

— Between 78.0% and 85.4% click 10 or fewer results

When all questions are considered, the average BV difference range is 3.9% (question 12) & 10.1% (question 5) for gender, 1.2% (question 12 - between 20-24 and above-24 categories) & 15.5% (question 11 - between under-20 and above-24 categories) for age, and 1.4% (question 7 - between English CMPE and Turkish CMPE categories) & 10.3% (question 4 - between Turkish CMPE and SE categories) for program studying.

The average BV difference between 20-24 & above-24 categories is less than 5.0% for questions 5-10 and 12. Under-20 & 20-24 and under-20 & above-24 pairs have the same except question 6 and English CMPE & Turkish CMPE pair has the same except questions 6 and 8. The average BV difference between female & male categories is less than 5.0% for questions 6 and 12, between Turkish CMPE & SE categories it is the same for questions 9 and 12, and between English CMPE & SE categories it is the same only for question 12.

Gender was the only DC entered by all the respondents with correct marking. It would be good to consider total-frequency-percentages for gender for every question from 4 to 12 and discuss behaviors without considering DCs in order to present additional information for the reader. Some remarks are as follows:

— 72.5% click tab occasionally or less.

— 76.0% click related query suggestion occasionally or less.

— 56.9% click video result occasionally or less.

— 55.3% click image result occasionally or less.

— 57.2% click pagination occasionally or less.

— 52.6% never click advertisement. 83.0% click advertisement rarely or none.

— 58.6% click spelling suggestion frequently or more. (As the details given before, content-as-well-as-design-based relation was encountered between the authors’ study’s spelling-suggestion-related question and spelling-suggestion-related question in the study of Ménard and Khashman [13]. Although Ménard and Khashman’s question is on image retrieval behavior, it is not a far different topic; therefore, due to the relation and the non-demographic-characteristics-based results of the Ménard and Khashman’s question, it could be good to say the following result of their question here: 47.5% of the respondents strongly agreed or agreed that it is helpful when the system checks spelling/entry and asks “Do you mean …?” Note that they stated that participants had Google Images in their minds while answering the survey questions.)

— 69.8% click instant search suggestion frequently or more.

— 80.4% click 10 or fewer results and 96.2% click 20 or fewer results.

4. Conclusions

This paper reports user behaviors about the usage of the SERPs based on the DCs gender, age, and program studying. In this survey-based study, 84.2% of the study population was reached. 87.0% of the population were male, more than 58.0% of them had age between 20 and 24, and more than 45.0% of them were from English CMPE program.

Only for advertisement click, every DC category (except female) has its highest BV at never click. The majority of each DC category have rarely or more click for tab, spelling suggestion, related query suggestion, instant search suggestion, video result, image result, and pagination and have 6 or more results click. For advertisement, most of female category, most of SE category, and half of Turkish CMPE category have at least rarely click. 50.0% or above BV at single behavior is shown for some DC category and question pairs: For male category, all age categories, and English CMPE category, more than 50.0% do not click (never click) advertisement; for Turkish CMPE category, 50.0% do not click (never click) advertisement; more than 50.0% of female category click advertisement rarely; 50.0% of female category click instant search suggestion frequently; and half of SE category click 6-to-10 results. For all DC categories, there are common characteristics that each consists of more than 50.0% respondents (for each category) as well as two behaviors. Some of them are as follows: between 79.7% and 90.5% click advertisement rarely or none; between 52.5% and 68.8% click related query suggestion rarely or occasionally; between 62.3% and 79.2% click instant search suggestion frequently or very frequently; between 78.0% and 85.4% click 10 or fewer results. When all questions are considered, BVs are below 58.0% and average BV differences are below 16.0% (0.0% BVs were encountered. For all questions, average BV difference was calculated between every DC category pair in each DC; however, any DC category pair which has no difference in terms of average BV difference was not encountered).

In results and discussion section, it is suggested to additionally consider content-based relations with the related studies which have such relations. It is believed that sharing the following experiences could also be good: (*) in the questionnaire, the item “Comp. Eng. (Turkish)” under “Double Major” in question 3 was unnecessary. Although any inconvenience was not occurred because of this item, in order to prevent any possible inconvenience due to this item, the item should have been removed since Turkish CMPE students can not apply to double major for English CMPE or SE; (*) in fact, there was one more question (question 13) which was “how much time do you spend to examine?” in the questionnaire, in order to have idea about the time that the users spend for just examining the SERPs. Yet, due to the question was not clear enough, respondents could have been answered this question with two different points of view which are as follows: The one is the time spend for just examining the SERPs; the other one is the time spend for examining the SERPs as well as the page(s)/document(s) opened from the SERPs. Therefore, this research topic is not included in the paper.