PROMETHEE is a multi-criteria decision making methodology which has been widely used to solve multi-criteria decision problems ranging from environmental to computer science, stock market, medicine etc. Behzadian et al. (2010) presented in detail various PROMETHEE applications in his review paper. Basically, PROMETHEE helps an outranking multi-criteria decision (Brans, 1985; Brans, 1986). The PROMETHEE methodology has six steps, as follows:

• ∙ First step: the first step involves selection of alternative determination. Here, we need to determine some of the alternatives that already exist and later we choose one as the solutions. For verification and validation purposes, we take top four websites appearing from Google result as alternatives. In our study, we analyze 30 keywords with four websites in each keyword: overall we evaluate 120 websites. Note that the keywords used in this study are chosen randomly for statistical purpose and the search engine used is Google.kr.
• ∙ Second step: we determine some criteria that are used in decision-making processes. In order to predict the search engine result, in this paper we use Google search - we determine the search criteria by Google which consist of five factors: keywords, link, content, design and address (Solehati et al., 2011; Esuli and Sebastiani, 2007); for each factor, we give the value scale ranging from 1 to 5 depending on the analysis of the website, and then analyze five factors for all 120 sites one by one; all of the factors are listed in <Table 1>. Note that there are more than 200 factors influencing the ranking; for the sake of brevity in order to proof our proposed concept, we select five most significant factors as in earlier work (Solehati et al., 2011; Eusli and Sebastiani. 2007; Zhou et al., 2017). Further, in factor number 3 and 4, i.e., content and design, it comprises of five and four subfactors, respectively. Each sub-factor is analyzed separately and the total scale is calculated by summing all the subfactors.
  
  Table 1. 
  Factors and subfactors used for PROMETHEE website popularity ranking.
  

  Factor	How the value is determined	Scale
  Keyword (f1) max	Meta tag keyword in HTML code	1-5
  Link (f2) max	Toolbar check pagerank	1-5
  Content (f3) min	∙How many times of duplication in the sites page ∙How relevant the keyword with the content ∙How many page that is attempted for phishing, Trojan, virus and other badware ∙How many pages that has affiliation program or non-original software ∙How many pages that is used as a cheating page	1-5 1-5 1-5 1-5 1-5
  Design (f4) min	∙How many link scheme it has ∙How many hidden text or hidden link in the page ∙How many wrong/dead HTML code ∙How many link in the page	1-5 1-5 1-5 1-5
  Address (f5) max	∙How same website address chosen with the keyword	1-5

  
  
• ∙ Third step: we assign the weight value (degrees-of-importance) for each criterion. This step is important since all of the assigned weight need to be judged objectively. Thus, a clear and consistent standard need to be established in order to correctly assign the degree-of-importance weight of each factor and subfactors to avoid misjudgment. In Google, however, the weight value for the search algorithm becomes their top secret against competitors (Google information for webmaster, 2021). Thus, in this study, we determine the weight value by parameter adaption using Taguchi method to fit the data from Google search results.
• ∙ Fourth step: we calculate and format the matrix according to whether it is minimization function or maximization function in <Table 2>. If the factor has maximization function, higher scale value is considered better, while for the factor with minimization function, the site performance is higher with smaller scale value. The function ‘f_n(a_n)’ return the scale value after analyzing the website according to each factor in <Table 1>.
  
  Table 2. 
  Initial matrix formation
  

  alternative	f1(max)	f2(max)	f3(min)	f4(min)	f5(max)
  a1	f1(a1)	f2(a1)	f3(a1)	f4(a1)	f5(a1)
  a2	f1(a2)	f2(a2)	f3(a2)	f4(a2)	f5(a2)
  a3	f1(a3)	f2(a3)	f3(a3)	f4(a3)	f5(a3)
  a4	f1(a4)	f2(a4)	f3(a4)	f4(a4)	f5(a4)

  
  
• ∙ Fifth step: we search basic matrix and matrix transpose. For the maximum factor condition, the matrix is composed by dividing row over column; whereas for the minimum factor condition, it is in an opposite way, column divided by row. The basic matrix is defined as
  

  basic matrix=initial matrix/∑n=14fnan

  (1)

• ∙ Sixth step: all the websites are ranked based on the net flow, Φ⁺-Φ^-; that is to calculate positive and negative preference flows for each alternative. Leaving flow, Φ⁺, the positive flow expresses how much an alternative is dominating (power) the other ones. While Entering flow, Φ^-, the negative flow relates to how much it is dominated (weakness) by the other ones, after which, we calculate the net flow.

We implement Taguchi statistical method to search for best combination of weight values. This is expected to improve the accuracy level of PROMETHEE prediction as compared to experimental data from search engine results. It is a powerful engineering tool for experimental optimization and one of the most well-known robust design methods. Generally, it is used to find the sensitivity of each parameter and determine the optimum combination of the design factors (Fowlkes and Creveling, 1995; Barker, 2005). It has been proven to be successfully implemented in several engineering area, such as fuel cell (Solehati et al., 2012), supply chain (Yang et al, 2011), marketing (Hong, 2012), machining (Zhang et al., 2013), acoustics (Garg et al., 2013), bioheat transfer (Jamil and Ng, 2013), to name but a few. Lin et al. (2015) integrated Taguchi method with neural network training and genetic algorithm to aid in searching for optimum product design and development. Therefore, we introduce this method for the first time in this area due to its robustness yet easy implementation. The weight values for each factor in <Table 1>, namely keyword (f₁), link (f₂), content (f₃), design (f₄) and address (f₅), are evaluated using Taguchi method utilizing five level designs, means that the range of weight value used in this study is from 1 to 5. <Table 3> shows the combination of each weight parameter and their levels.

We evaluate the objective function of the best combination of weight parameters based on the accuracy of the prediction, calculated by

Where Err is the prediction error as compared to Google result, 4! is the possible combination of ranking from four websites and 30 is the total number of keywords analysed; while, the signal-to-noise ratio (S/N) evaluated is based on Larger-the-better which means that the higher the accuracy, the closer the weight values:

Once the optimum combination of each parameter has been determined, we verify the predicted results from Taguchi method with Google search results. The confidence interval (CI) of the estimated value is calculated by:

where F_α,v1,v2 is the F-ratio required, v₁ is the number of degree of freedom of the mean, v₂ is the number of degree freedom of the error, V_ep is the error of variance, r is the sample size in the confirmation test, and n_eff is the effective sample size, defined as

where N is total number of trials and DOF_opt is the total degree of freedom that are associated with items used to estimate optimum n_opt.

Once the PROMETHEE prediction is validated against search engine data, in this particular case of Google, we aim to implement the prediction method of improving the popularity of the website to rise to a higher ranking in search engine. Here, we introduce a feedback loop from the PROMETHEE analysis to the web developer in order to improve the popularity of the website and, further, raise the rank of the website. That is to say, increasing the rank of existing website or develop new website according to PROMETHEE result guidelines so that it will appear in the first rank in the search engine. <Figure 2> shows the concept of PROMETHEE feedback loop concept. First, the website popularity rank is analyzed using PROMETHEE and the results are verified and validated against Google search. If the rank is at high rank, there is nothing to do with the website; however, if the website rank is low, PROMETHEE will provide the feedback to the web developer to modify the website accordingly. To improve the website, each of the criteria, factors and subfactors together with their degrees-of-importance during PROMETHEE assessment need to be considered and implemented to the website.

Figure 2. 
Feedback Loop Concept to Raise Popularity of Website Rank using PROMETHEE

For example, if a company wants to improve their website search engine ranking, firstly, the PROMETHEE analysis needs to be carried out together with their competitor’s website and validate the search engine ranking, as presented in previous Section. Thus, assessment for each criterion in <Table 1> need to be conducted to see which factor is lacking according to Google ranking method in the PROMETHEE matrix. Thus, the lacking factor needs to be improved and simulated in PROMETHEE software to see the net-flow and sensitivity analysis of each factor. The proses is repeated until it surpasses the ranking of competitor’s website. This can be done manually or automatically by creating a looping function in the software.

Besides verification of the software results with analytical solution derived from the first principal concept, validation of the proposed method with experimental evidence from Google search engine results is of paramount importance for the judgment of successful method. Here, the weight value of each criterion is not known a priori; Further, the search method is also a top secret in Google against competitor. Thus, we need to find best combination of weight value for each criterion. In this study, we implement Taguchi statistical method to search for best combination of weight value.

Here, we have five weight parameters, e.g. keyword (f₁), link (f₂), content (f₃), design (f₄), address (f₅), with five level values, ranging from 1 to 5, for each weight parameter (see <Table 3>). An L25 orthogonal array (OA) was employed in the experiment matrix, as shown in <Table 4> (except for last column), to search for best combination. Thus, we systematically calculate each combination of weight function in the orthogonal array (25 combinations) and apply to 30 analyzed keywords (120 websites). Note that if we had chosen to carry out all possible combination of parameters using full fractional approach, the number of experiments would have been more than several hundred which is impractical.

<Table 4> in the last column shows the accuracy of PROMETHEE prediction for combination of each weight value as compared to Google results. It is obvious that changing the weight values affects the accuracy of PROMETHEE prediction. However, we note that the accuracy changes only about 4% as it varies from 93 to 96% for all 25 combinations, which means that the PROMETHEE prediction already has reasonably good accuracy. A closer inspection reveals that the main error on the PROMETHEE prediction is due to same value of net flow which is mirrored by the same ranking result. Thus, it can be deduced that PROMETHEE method has reasonable accuracy for prediction quality.

In order to get the most accurate result, we need to determine the best combination of each weight values. In Taguchi method, we analyze the sensitivity of each factor by looking at the signal-to-noise ratio graph. <Figure 3> shows the sensitivity of each weight parameter with regard to the accuracy of prediction. It is seen that content (f₃) yields the most sensitive weight criteria with the S/N ratio of 0.24, thus, according to S/N analysis, the weight value for f₃ is 5. The second most sensitive factor according to Taguchi analysis is address (f₅) with the sensitivity of S/N ratio of 0.09, the weight value for f₅ is therefore determined to be 3. Now, the third most sensitive weight factor with the sensitivity of S/N ration of 0.07, interestingly, is link (f₂), and design (f₄); the best weight values for f₂ and f₄ is 4. While the least sensitive S/N ratio of weight factor of 0.04 is keyword (f₁) and the values for weight is 3.

Figure 3. 
S/N response graph for weight value for each criterion

Thus far, we have obtained the best combination of weight value for each factor. Now, we implement the obtained weight value obtained from Taguchi method to the PROMETHEE analysis. The results of leaving flow, entering flow and net flow together with the rank results from both Visual PROMETHEE software as well as Google results are summarized in <Table 5>. Here several features are apparent; foremost among them is a good agreement between PROMETHEE results and Google results with accuracy of 98% as compared to the previous results without adapting weight value using Taguchi with the highest accuracy of 96%, see last column in <Table 4>. Clearly, applying Taguchi method improves the quality/accuracy of PROMETHEE prediction, albeit in this particular case is less significant. This may be due to the fact that PROMETHEE method is already a robust method for multi-criteria decision which has close similarity to one used in search engine website ranking.

With respect to ranking quality, we note that the error of ranking prediction is mainly due to similar result in net flow which is mirrored by same ranking result. On closer inspection, for the optimized result after applying new weight values obtained from Taguchi method, only two ranking out of total four combinations from thirty keywords (4! × 30) is deviate from Google results; especially for the case where the website has very close similarity. For example, in the websites number 41 and 42 with the keyword of “buy ginseng online”, both website, i.e., http://www.onlineginsengstore.com/ and http://www.buyginseng.org/ have very much similarity; thus, albeit the degrees-of-importance for scales of factors and subfactors are different, it, surprisingly, turns out to have similar net flow output which, in turn, leads to the same ranking. This deviation may be due to different ranking algorithm by Google; however, this is insignificant (error of 2%). Thus, statistically speaking, the confidence level of the PROMETHEE method for ranking quality is found to be ~ 98%, as can be inferred from <Table 5>. Clearly PROMETHEE method together with Taguchi method to search for combination of weight values is a reliable method and tool to predict website popularity rank, which shows potential to be used for practical applications.

Now, we move forward to propose feedback loop concept, as introduced in Section 3.3, to improve the ranking in search engine result. In this paper, for a limiting case, we demonstrate the feedback loop concept to the existing website in our analysis, for example, for the keyword “branded bag”, website no 109-112, for which the net flow analysis is shown in <Figure 4>. It is seen that the first, second and third rank website has a very close net flow, which means that small modification can lead to surpass the rank. <Table 6> shows the detail scale evaluations and degree-of-importance used for PROMETHEE assessment for each factor and subfactor. We note that the highest evaluation scales are observed for factor keyword (f₁), content (f₃), design (f₄) and address (f₅) at the first rank website; thus, the chance for website no 2 to raise to first rank is by improving their factors (f₁ and f₅) to the highest level (or minimum level for f₃); while for factor f₂, the website no 2 is already higher than website no 1, but it can be improved further as well.

Figure 4. 
PROMETHEE Partial Ranking (a) and complete ranking (b) for keyword “branded bag” before feedback loop is implemented.

It is seen that from the assessment values, there is still room for improvement if one would design/improve their existing website to surpass the first rank website. It is seen that for website no 2, at a constant design factor (f₄), and gives raise the content (f₃) and address factors (f₅) will lift-up the net flow. Closer inspection reveals that to bring site no 2 to surpass the first rank, site no 2 should improve its content, address and keyword simultaneously; whereas the link factor (f₂) is already higher than that of website no 1, but the scale is not at a maximum level yet, and, hence, it still can be improved further. Therefore, the new factor scales for website no 2 after feedback loop improvement are: f₁ = 5; f₂ = 5; f₃ = 5 (1×5 subfactors); f₄ = 4 (1×4 subfactors); and f₅ = 5. The results of which are presented in <Figure 5>. Clearly, after feedback loop concept is implemented, the website no 2 is able to surpass website no 1 and turns out to be jump to the first rank popularity at the search engine. On closer inspection, we note that the net flow for website no 2 increases significantly from 0.0185 before the feedback loop improvement to 0.1343 thereafter, which surpasses the net flow of website no 1 (Φ⁺-Φ^- = 0.0417).

Figure 5. 
PROMETHEE partial ranking (a) and complete ranking (b) for keyword “branded bag” after feedback loop improvement

On a final note, by carefully consider all the factors and subfactors analyzed in PROMETHEE method, one can easily improve the popularity of existing website by giving a feedback loop control to the website. It can also be deduced that one can design a website which can surpass the highest rank website by carefully evaluating and fine-tune the factor and subfactor obtained from PROMETHEE in the search engine as a feedback loop for the web developer, both for existing and newly developed website.