Quality Driven Approach for Product Line Architecture Customization in Patient Navigation Program Software Product Line

Abstract

Patient Navigation Program (PNP) is considered as an important implementation of health care systems that can assist in patient's treatment. Due to the feasibility of PNP implementation, a systematic reuse is needed for a wide adoption of PNP computerized system. SPL is one of the promising systematic reuse approaches for creating a reusable architecture to enabled reuse in several similar applications of PNP systems which has its own variations with other applications. However, stakeholder decision making which result from the imprecise, uncertain, and subjective nature of architecture selection based on quality attributes (QA) further hinders the development of the product line architecture. Therefore, this study aims to propose a quality-driven approach using Multi-Criteria Decision Analysis (MCDA) techniques for Software Product Line Architecture (SPLA) to have an objective selection based on the QA of stakeholders in the domain of PNP. There are two steps proposed to this approach. First, a clear representation of quality is proposed by extending feature model (FM) with QA feature to determine the QA in the early phase of architecture selection. Second, MCDA techniques were applied for architecture selection based on objective preference for certain QA in the domain of PNP. The result of the proposed approach is the implementation of the PNP system with SPLA that had been selected using MCDA techniques. Evaluation for the approach is done by checking the approach's applicability in a case study and stakeholder validation. Evaluation on ease of use and usefulness of the approach with selected stakeholders have shown positive responses. The evaluation results proved that the proposed approach assisted in the implementation of PNP systems.

1. Introduction

In Malaysia, statistics show that the number of deaths caused by chronic diseases keeps increasing each year [1]. This is mainly because of the late presentation of treatment and the lack of extensive care for the patient. Similar to the case in other countries such as the United States, there had been reports on the insufficiencies of a fragmented healthcare system that failed to provide access to timely, equitable, and standard care to everyone [2]. A program created by Dr. Harold Freeman, Patient Navigation Program (PNP), is designed to promote access to timely diagnosis and treatment of cancer and other chronic diseases by eliminating barriers to care [3]. A feasibility study which has been done on the only computerized PNP implementation in Hospital Tengku Ampuan Rahimah (HTAR) futher shows the improvement the time taken for diagnostic and treatment [4]. The study further proposes a wider adaptation of PNP for monitoring the progress of the cancer patient’s treatment. PNP has been done manually in other hospitals in Malaysia which resulted to require extra energy and time to complete. Therefore, by implementing a computerized PNP system in hospitals all over the country, the treatment process of any patients can be monitored efficiently.

The differences in the software system’s implementation reduced the PNP implementation process’s effectiveness in other hospitals as it is known to have variation according to the technicalities, medicinal and technological advancement. There is also the case of different resources and facilities of each medical department that contribute to the difference of processes and practices. Thus, it is logical to have systematic reuse that can assist in the system’s mass development such as Software Product Line architecture (SPLA) approach to cater to this issue. SPLA produces a system with similar function products that utilize different variables and configurable options. There is also a problem of unclear description of quality requirements from stakeholders that further delay the decision-making process. Thus, by inserting the quality attribute (QA) with non-functional requirement (NFR) description in Feature Model (FM) helps software architects to determine the needed quality and feature in the early stages of system development in SPL. Besides that, there is a problem with the architecture selection because of human decision-making’s subjective nature. Therefore, the goal of this research is to enhance the decision-making process with the implementation of Multi Criteria Decision Analysis (MCDA) techniques for an objective decision for a suitable architecture pattern for developing SPLA in PNP domain. Validation of the approach is done using a survey-based method to gain stakeholder feedbacks and case study method to show the approach validity.

The rest of this paper is as follows: Section 2 provides the literature review of related works. Section 3 describes the proposed approach’s framework for EFM and SPLA selection with MCDA techniques. Section 4 contained the experiments with the proposed approach. Lastly, Section 5 provides conclusions and possible future directions.

2. Related Works

In this section, three essential aspects of this research are PNP, quality attribute representation, and MCDA techniques used in architecture selection. An in-depth investigation of these three aspects is performed to obtain the gaps and necessary information on past research in a similar domain.

2.1 Patient Navigation Program

The PNP model founded by Dr. Freeman, combined with free or low-cost breast cancer screening, has contributed to the detection of early-stage cancer, which eventually increased the 5-year survival rate. Thus, the initial PNP model has been expanded to include several elements including prevention, early detection, and post-treatment survivorship. This model is widely used in other chronic illness PNP to reduce mortality rates [5]. Thus, there is a need to have a wider adoption of the PNP computerized implementation, especially in the health care sector.

2.2 Quality Attribute Identification and Architecture Pattern Selection

There is a problem with decision-making in architecture selection, hindering the implementation and derivation of the products. To tackle for the root of the problem, a literature review has been done in domain architecture. Studies by Etxeberria and Sagardui [6], Yrjönen and Merilinna [7], Zhang et al. [8], and Sanchez et al. [9] explore QA representation in the early phase of software development. Two of the studies [6] [8] represented QA by extending their feature model (FM), while in [7], the QA represented by extending the nonfunctional requirement (NFR) framework with a concept of measurable NFR that enables them to verify the realization of defined NFR in a product empirically. Besides that, Sanchez et al. [9] quantitatively evaluate the trades-off between different QA to select the best candidate for an architecture pattern.

Other than that, Guana and Correal [10] proposed a model-driven strategy to prove that it can simplify and automate SPL’s definition process and improve the reusable assets’ selection process. The proposed conciliation model clarifies the relationships between the SPLA, components model, quality attribute requirements model, and sensitivity point model. Focusing on system architecture selection, Lytra et al. [11] discussed possible ways on how variability and architectural decisions interact, as well as their management and integration in a systematic manner. The technique used is by demonstrating the integration between the two types of decisions in a motivating case and leverage existing tools. Horcas et al. [12] adopted and followed the classical SPL engineering approach to specify the functional QA variability model as a Functional QA Domain Engineering Process.

Researchers developed architecture based on software pattern. An investigation has been done by reviewing the past studies the architecture pattern implementation for PNP system. Previous studies as shown in Table 1 depicted the highest architecture implementation is layered while Broker is the least chosen architecture pattern.

Table 1. Architecture Pattern Implemented for PNP System

There are several approaches extending FM with the QA representation. Nonetheless, a few improvements can be made to increase QA representation in FM. Factors such as whether the QA is in the latest ISO standard, following the domain quality, and the domain experts’ requirements, must be considered for the EFM. Further investigation on the improvement of the FM extension will be discussed in the next sub-topic. This ensures that the QA identified in the early phase of the system’s implementation is suitable and correct. Many researchers study both the requirement and architecture rather than studying the topic separately. Thus, by following the trend, this research will also focus on linking requirements to architecture. There are not many studies done on the representation of QA in the feature model.

2.3 Extended Feature Model

Several researchers are exploring the representation of QA in FM in the past as it is significant for architecture selection and in product line architecture. As investigated, there is a noticeable lack of QA representation in architecture selection. The researchers solved a few issues regarding the identification and selection of QA for some products. Zhang et al. [8] studied the QA assessment for the configured product they mentioned were neglected in most existing product configuration approaches. They also stated that there is a high-cost factor to fix any problem that may arise if the produced product cannot meet the customers’ requirements on quality attributes. Therefore, a specific product’s QA concern should be identified in the earliest stage of product development in SPL [8].

Fig. 1. Extension of feature model with QA Sub-Feature Tree [8].

To ease and accelerate the QA identification process, researchers represented QA in FM by extending their current FM with a sub-feature tree to represent QA, as shown in Fig. 1. Other than that, Etxeberria et al. [6] highlighted the need for eliciting and refining QA requirements because QA has a different and imprecise meaning depending on the domain. Researchers extended their FM with a quality feature, consisting of a mechanism from Architecture Trade-off Analysis Method (ATAM) evaluations. This evaluation represents quality attributes, their variability, and their influence on the quality of the functional, architectural, and implementation features. Fig. 2 shows the EFM of the study.

Fig. 2. Extension of quality feature in FM [6].

Both studies implemented an EFM to cater to the problem of QA representation for a product line. However, improvements can be made by considering whether the QA is in the latest ISO standard, following the domain quality, and match with the domain experts. An EFM specifically for PNP will be implemented with the enhancements. Table 2 summarizes the advantages and disadvantages of the proposed EFM of the two studies.

Table 2. Advantages and disadvantages of past extended feature models

2.4 MCDA Techniques in Architecture Selection

The selection of the MCDA technique to be implemented is determined by considering the most popular techniques used by researchers on similar topics. Firstly, looking at research was done by Zaki et al. [13], they concentrated on the hybridization of the Fuzzy and Analytical Hierarchy Process (AHP), where the fuzzy model handles the imprecise judgments made during architecture style selection. At the same time, the AHP will assist in the pair-wise comparison of the architecture styles. In line with that, research by Halim et al. [14] implemented FAHP. This MCDA technique is enhanced with a fuzzy technique to provide the linguistic scale to overcome the deterministic scale originally proposed in AHP. The linguistic scale known as Triangular Fuzzy Number (TFN) represents the fuzziness and uncertainty in human decision making. Besides that, Upadhyay et al. [15] implemented the TOPSIS technique in their proposed framework to reincorporate the complex domains composed of diverse internal and external factors in the software architecture evaluation process to obtain user’s requirements satisfaction. An in-depth interdisciplinary exploratory study conducted by Mjeda et al. [16] showed that several MCDA techniques could be used to improve SPLE approaches. Other than that, Bouayad et al. [17] used AHP, a multi-criteria decision analysis (MCDA) method that aids a decision-maker decide and choose the organization’s best suitable framework. Following the problem of this research, which mainly focuses on selecting architecture patterns with identified QA, elimination of Fuzzy AHP had to be done as the problem of fuzziness and uncertainty in decision making is not considered. Therefore, to assist in improving the decision-making of the PNP system’s architecture pattern, the MCDA techniques that will be investigated are AHP and TOPSIS.

3. The Proposed Approach

In this paper, the proposed approach consists of 2 phases as shown in Fig. 3. The first phase is the QA representation using EFM. We enhanced the EFM by adopting the method from past paper and mapping of QA from ISO standard and Health Information System Quality Model. The QA identified then are used for the next phase of MCDA techniques implementation. In this phase, we experiment with 2 of the MCDA techniques, AHP and TOPSIS, to analyse the differences in the results.

Fig. 3. The process of the proposed approach

3.1 Extending FM with QA

In this study, we extended the FM of PNP to assist in QA identification for the architecture selection. Using both methods from [6][8], we further enhanced the EFM. First, the FM was extended by a QA sub-tree that contain the clear description of the QA. Then, the functional feature of the QA is connected in the EFM. Lastly, the QA to extend on the feature model are mapped with the standard ISO which is ISO/IEC 25010 and the domain quality model which is HIS-DQM. This is to ensure that the QA is within the PNP domain.

We construct the EFM using Feature IDE tools in the Eclipse software system. Fig. 4 shows the FM of PNP without the extension of QA. We extended the FM by adding a QA sub-tree, which is shown in Fig. 5.

Fig. 4. Feature Model of PNP

Fig. 5. Sub-tree of quality attribute

We enhanced the EFM by extending the feature model with QA mapped with the standard ISO, ISO/IEC 25010, and the domain quality model, HIS-DQM. This is to ensure that the QA is within the PNP domain. We also combine both methods used in [6][8]. Fig. 6 shows the EFM of PNP with the QA feature tree.

Fig. 6. Feature model of PNP with QA extension

3.2 MCDA with AHP and TOPSIS

Then, by adopting the steps by Ulkhaq et al. [19], we implemented the combination method of both techniques. The steps for the combination of AHP and TOPSIS are such as below: (1) First, questionnaires have been distributed to the stakeholders for the pair-wise comparison data The AHP was then applied to calculate the weights or the importance degrees for each criterion.

(2) Then, from the result of the AHP, weights of the criteria for the TOPSIS process were determine. Fig. 4 shows the framework of the proposed method of AHP combined with TOPSIS.

Fig. 4. The framework of AHP combine with TOPSIS method.

4. Experiments with The Proposed Approach

The experiment for the proposed approach is conducted to observe and analyse the results. Based on guidelines by Runeson et al. [20], the case study experimentation has several steps, which consist of data collection, evidence collection, analysis of data collection, and discussion.

4.1 Data Collection

In the early phase of the research study, an interview and observation on PNP’s domain were done to obtain the stakeholders’ requirements. The requirements collected are the functional requirements and non-functional requirements. Regarding implementing the MCDA technique, the artifact needed is the QA identified from the non-functional requirement. Besides that, there are five architecture patterns chosen for the case study selected from the SPL literature review connected to the architecture selection and PNP system. Besides, a questionnaire has been distributed to obtain pair-wise comparison data for the MCDA technique.

4.2 Collecting Evidence

The selection of QA for the PNP domain is determined by mapping all the QA with the latest standard quality assurance, ISO/IEC 25010 Quality Model. Adopted from a study done by Losavio et al. [21], the QA will also be mapped to the health information system, HIS-DQM a quality model under PNP’s domain. Figure 5 shows the QA from the three criteria that are chosen for the PNP. The selected QA are efficiency, portability, compatibility, functional suitability, security, reliability, and maintainability which contain in all of the 3 criteria.

Fig. 5. The QA listed by the stakeholder, ISO/IEC 25010 and HIS-DQM

Calculation of the QA pattern score is based on the equation of discrete ordinal integer values of x ∈[ −2, 2] to fulfill the alternative QA in each architecture pattern. The rest of the equation that represents the value x is as shown below.

\(x=\left\{\begin{array}{c} -2 \text { if symbol }=-- \\ -1 \text { if symbol }=- \\ 0 \text { if symbol }=0 \\ 1 \text { if symbol }=+ \\ 2 \text { if symbol }=++ \end{array}\right\}\)

Based on [22], the impact of each QA according to the architecture pattern are transformed into symbols as shown in Table 3. The mapping shows in Table 3 describe the meanings of the symbols used are as follows: ‘+ +’ means that a style strongly supports a property, ‘+’ stands for some support, ‘o’ stands for neutral or no support, ‘–’ means that the style has a negative influence on achieving a property and ‘– –’ indicates a strong negative impact on the achievement of a property. In order to populate Table 3 with suitable values, the researcher has identified the quality attributes for each pattern using the available design pattern reference.

Table 3. Quality attributes fulfilment for an architecture pattern

4.3 Analysis and Discussion

The MCDA techniques being analysed are AHP, TOPSIS, and the combination of both AHP and TOPSIS. First for the AHP, by following the steps, a pair-wise comparison was made by distributing questionnaires to 10 domain experts. From there, a comparison matrix was built. Using both the weight gained from the calculation using AHP and the fulfilment or score for each architecture pattern’s quality attributes, identification of architecture pattern that has the highest results of the best-matched architecture is done by the weight scoring method. Table 4 shows the calculation adopted from [22].

Table 4. Numerical decision matrix

Next, following the TOPSIS technique’s steps, the first steps determine the weight for the criteria [23]. The weight of the criteria can be defined in different ways [24]. Therefore, depending on the type of criteria, the weight may differ depending on the information and research that need to be done. In this study, we used the average weight for the calculation of TOPSIS. Table 5 below shows the last step, which calculates the ideal best and ideal worst values and the architecture pattern’s performance score.

Table 5. TOPSIS Performance Score Result

Lastly, to improve the ranking of both techniques, the experiments on the combination of both AHP and TOPSIS were conducted. The first step is to build the normalized decision matrix consists of an alternative value and criteria weights which are shown in Table 6. The weights of the criteria for the TOPSIS process are determined from the AHP. Therefore, duplicate from the result from the data analysis of AHP, the QA weights are already determined.

Table 6. Normalized decision matrix for AHP and TOPSIS

Overall results for each MCDA based on experiments using AHP, TOPSIS and a combination of AHP and TOPSIS are shown in Table 7.

Table 7. Comparison of results

The overall results showed that layered is the best architecture pattern, followed by Blackboard using TOPSIS and AHP combined with TOPSIS techniques, although this pattern obtained third place using AHP. Client-Server architecture pattern is placed third for AHP combine with TOPSIS and last for both AHP and TOPSIS. Next, the Hierarchical Layer architecture pattern is second for AHP, third for TOPSIS, and fourth for AHP combine with TOPSIS. Lastly, the Broker architecture pattern is placed fourth for both AHP and TOPSIS and fifth for AHP combine with TOPSIS. The combination of AHP and TOPSIS that had been proposed is validated by the sum of a collection of architecture patterns used for the PNP system all over the world in past studies journals and papers. Table 8 shows the list of architecture patterns used for the PNP system.

Table 8. Collection of architecture patterns used for PNP system

Most of the studies shown in Table 8 used Layered architecture patterns for their PNP system. The value is determined by obtaining differences between each technique and the total of the architecture pattern. Below is the formula of the MCDA techniques results and the sum of past studies used. Table 9 shows the evaluation of the respective techniques.

\(\frac {Similar \ Ranking \ with \ Previous \ Studies} {Total \ of \ rank \ Place} \times 100\)

Table 9. Comparison and percentage of similarities between different architecture

In conclusion, based on the experiment done, AHP has the lowest percentage of similarities with previous studies which is 40%. TOPSIS technique has 60% similarities with previous studies, and AHP combine with TOPSIS records the highest percentage of 80% similarities with previous studies. Therefore, the best architecture pattern selection is with the combination of AHP and TOPSIS, which will produce a more accurate result.

5.4 Validation and Discussion of The Approach

Validation of the proposed approach is conducted by implementing Goal Question Metric Model (GQM) promoted by Victor Basili [35]. GQM defines a three-level measurement model, which are conceptual level (goal), operational level (question), and quantitative level (metric). The interpretation of the metrics after the measurement is rather effortless because GQM creates a direct link between the measured data and the goals of measuring before data collection. Fig. 7 shows the GQM model for research validation. For Research Question 1 (RQ1), the validation is on how the approach is implemented in a case study, which is the SPLA of PNP. Meanwhile, Research Question 2 (RQ2) is on the users’ perception to accept and use the approach in their practice. The implementation of the approach in a case study will be the derivation of the SPLA of PNP for RQ1. Besides that, five domain experts from the IT department of Hospital Sultanah Aminah are being interviewed using a semi-structured interview script for RQ2. The semi-structured interview script adopted from [36] consists of 6 questions to obtain the responses for understandability, learnability, user satisfaction, efficiency, flexibility, and correctness of the approach.

Fig. 7. GQM model for approach validation

In PNP system architecture, the architecture pattern will be derived from the result using a combination of AHP and TOPSIS approach. As the selected SPLA for the PNP system is Layered architecture, each layer of the architecture’s components will be determined. There are several main components for the PNP system, such as the Treatment, Search Patient, Appointment, and Referral Management. The interdependencies between QA with functional features have already been determined in Section 4.1, for example, the QA feature of security interdependencies with functional feature Diagnosis. Therefore, the related components will be the Treatment component, which contains the functional feature and the QA for the Layered architecture pattern of PNP. An example of the derived architecture is given in Fig. 8.

Fig. 8. PNP system security components conforming to Layered architecture.

PNP system architecture in Fig. 8 uses a Layered architecture pattern, which consists of three layers that are Presentation Layer, Business Layer, and Database Layer. An example of a component for the Presentation Layer is WebForms used for the interface of the system and user access control to the system. Meanwhile, Business Layer consists of components such as Diagnosis, Security (which is the QA inter-dependencies with its functional features), and Patient Treatments. Lastly, there is a Database Server of the PNP system in the Database Layer. The derived SPLA of PNP shows that the selected architecture pattern can be implemented in the PNP system case study.

Aside from that, as mentioned before, a semi-structured interview is also being conducted to validate the approach. The semi-structured interview script contained three parts: background of the interviewee, question about the proposed approach, and general question. Answers from the proposed approach will represent the quality validation for the understandability, learnability, user satisfaction, efficiency, flexibility, and correctness criteria. Table 10 shows the questions asked during the interview and the corresponding QA it represented.

Table 10. Interview Question Validation.

Summary of the domain experts (DE) details are listed in Table 11. The DE are presented with the implementation of the approach in PNP use case SPL before the interview to validate the approach.

Table 11. Results of the interview for the proposed approach validation.

The responses are summarized in Table 12. The ID represents the five domain experts that had been interviewed for the validation. The symbol “+” represents positive evaluation; “–” represents negative evaluation opinions. Correspondingly, the “++” shows a very favourable view, and “– –” shows a very negative opinion. As interviewees often mentioned both positive and negative perceptions, “+/–” represents positive answers with potential negative concerns and vice versa.

Table 12. Results of the interview for the proposed approach validation.

The results show positive perceptions and feedback regarding understandability, learnability, users’ satisfaction, and efficiency based on the overall answers. Meanwhile, the feedback on flexibility is also interrelated with the support tool, but some interviewees mentioned using the approach with minor adaption flexibly. This is the same with the correctness of the approach, as there should be more detailed or specific testing made to determine the correct implementation of the system. Overall, the stakeholders’ responses confirmed that the proposed approach assists in SPL architecture selection in the PNP domain’s system development. In this case, the implementation of our approach assists in selecting SPLA for the PNP system. This is because the EFM gives a clear picture of the QA needed for the SPLA selection. The EFM also helps in the derivation of the PNP system architecture as it provides the interdependencies between QA with functional features.

6. Conclusion

In this paper, we proposed a quality-driven approach that consists of extending the PNP feature model and MCDA techniques application to SPLA selection. Clear QA representation from the EFM assists in determining the QA to be used for the SPLA selection. The selection of SPLA was then conducted by applying the combination of MCDA’s two techniques: AHP and TOPSIS. The result of the approach using the combination of both AHP and TOPSIS shows that the layered architecture patterns is the most suitable for PNP system implementation. The PNP system’s architecture can then be derived using the selected architecture pattern to validate the approach’s applicability. Moreover, positive responses from stakeholders verified the ease of use and usefulness of the approach. Nevertheless, this research can be further improved by enhancing the step of obtaining TOPSIS initial weight for a more efficient selection process. SPLA selection using only a few architecture patterns such as layered, Client-Server, Blackboard, Hierarchical Layer and Broker, may have limitations. Therefore, other architecture patterns, such as pipe-filter, peer-to-peer, model-view-controller, and interpreter should be explored in future studies.