Determining criteria and weights for evaluating the socio-economic efficiency of scientific and technological topics/projects after acceptance

ource: 
The process of determining the evaluation weight for each criterion and evaluation content is according to the following figure: 
Fig. 2. Diagram of steps for determining the evaluation weights following AHP method 
(Source: Authors synthesize from research results) 
Expert’s opinions matrix 
W1 Consistency ratio matrix 
W2 Consistency ratio matrix 
Consistency ratio CR ≤ 0.1 
Weight is assigned to each evaluation criterion 
Wrong 
Right 
Expert group discussion 
 112
4. The research results and discussion 
The calculation of evaluation weight for each criterion or each evaluation content is determined by the authors based on the 
order of weight assignment that is presented in the research methodology. In particular, an illustrative topic / project in the 
Agricultural Science that has weights of the evaluation criteria as follows: 
Table 5 
Illustrating the weights of the evaluation criteria in the Agricultural field 
 Evaluation criteria Order assigned Weight Standardized weight (%)
1 The efficiency in terms of science 1 3 20,00
2 The efficiency in terms of technology 1 3 20,00
3 The efficiency in terms of economics 1 3 20,00
4 The efficiency in terms of environment 2 2 13,33
5 The efficiency in terms of culture, society 3 1 6,67
6 The efficiency in terms of information, management 3 1 6,67
7 The efficiency in terms of training 2 2 13,33
 Total 15 100,00
(Source: Authors synthesize from research results) 
From this calculation method, the results of calculating the evaluation weight of the topics / projects in scientific fields following 
as: 
Fisrtly, in the field of Agricultural Science 
Fig. 3. Weight of each criterion for evaluating socio-economic efficiency of Agricultural Science topics / projects 
(Source: Authors synthesize from research results) 
The above results show that the importance of criteria in terms of science, technology and economics efficiency is most 
concerned, followed by environmental and training effectiveness while the efficiency in terms of culture, society and 
information, management is considered the least important for the topics/projects in Agricultural field. 
Secondly, in the field of Natural Science 
S1: 19.05% 
S2: 4.76% 
S3: 9.52% 
S4: 9.52% 
S5: 14.29% 
S6: 19.05% 
T1: 25.00% 
T2: 8.33% 
T3: 25.00% 
T4: 16.67% 
T5: 16.67% 
T6: 8.33% 
E1: 18.18% 
E2: 9.09% 
E3: 13.64% 
E4: 4.55% 
E5: 9.09% 
E6: 13.64% 
E7: 9.09% 
E8: 13.64% 
En1: 14.29%
En2: 14.29%
En3: 7.14%
En4: 21.43%
En5: 21.43%
En6: 14.29%
En7: 7.14%
CS1: 15.79%
CS2: 10.53%
CS3: 10.53%
CS4: 10.53%
CS5: 21.05%
CS6: 15.79%
CS7: 5.26%
CS8: 10.53%
IM1: 27.27%
IM2: 9.09%
IM3: 27.27%
IMT4: 9.09%
IM5: 9.09%
IM6: 18.18%
Tr1: 25.00%
Tr2: 8.33%
Tr3: 16.67%
Tr4: 25.00%
Tr5: 16.67%
Tr6: 8.33%
The results of evaluating topics / projects 
S: 20.00%  T: 20.00%  E: 20.00%  En: 13.33% CS: 6.67% IM: 6.67% Tr: 13.33% 
S7: 14.29% 
S8: 4.76% 
S9: 4.76%  E9: 4.55% 
E10: 4.55% 
N. T. Nguyen et al. /Accounting 6 (2020) 113
Fig. 4. Weight of each criterion for evaluating socio-economic efficiency of natural science topics /projects 
(Source: Authors synthesize from research results) 
The results demonstrate that the importance of criteria with regard to science, information and management efficiency is most 
concerned, followed by the efficiency in terms of technology and environment. By contrast, the efficiency in terms of economics, 
culture, society and training is considered the least important in the topics/projects in the natural science field. 
Third, in the field of social sciences and humanities: 
Fig. 5. Weight of each criterion for evaluating socio-economic efficiency of social sciences and humanities topics /projects 
(Source: Authors synthesize from research results) 
The results illustrate that the importance of criteria in terms of economics, culture, society, technology and management 
efficiency is most concerned, followed by the efficiency in terms of science and training. By contrast, the efficiency in terms of 
technology and environment is the least important in the topics/projects in the social sciences and humanities field. 
En: 15.38% IM: 23.08% Tr: 7.69% 
S1: 19.05% 
S2: 14.29% 
S3: 9.52% 
S4: 9.52% 
S5: 14.29% 
S6: 14.29% 
T1: 16.67% 
T2: 25.00% 
T3: 25.00% 
T4: 16.67% 
T5: 8.33% 
T6: 8.33% 
E1: 8.33% 
E2: 4.17% 
E3: 4.17% 
E4: 8.33% 
E5: 12.50% 
E6: 16.67% 
E7: 12.50% 
E8: 16.67% 
En1: 15.00%
En2: 15.00%
En3: 20.00%
En4: 10.00%
En5: 5.00%
En6: 15.00%
En7: 20.00%
CS1: 6.67%
CS2: 20.00%
CS3: 20.00%
CS4: 13.33%
CS5: 13.33%
CS6: 13.33%
CS7: 6.67%
CS8: 6.67%
IM1: 15.38%
IM2: 15.38%
IM3: 15.38%
IM4: 23.08%
IM5: 23.08%
IM6: 7.69%
Tr1: 25.00%
Tr2: 16.67%
Tr3: 16.67%
Tr4: 16.67%
Tr5: 16.67%
Tr6: 8.33%
T: 6.67%  En: 6.67% IM: 20.00% Tr: 13.33% 
S1: 16.67% 
S2: 16.67% 
S3: 4.17% 
S4: 4.17% 
S5: 12.50% 
S6: 16.67% 
The results of evaluating topics / projects 
S: 23.08%  T: 15.38%  E: 7.69%  CS: 7.69%
S7: 9.52% 
S8: 4.76% 
S9: 4.76%  E9: 12.50% 
E10: 4.17% 
The results of evaluating topics / projects 
S: 13.33%  E: 20.00%  CS: 20.00%
S7: 8.33% 
T1: 16.67% 
S8: 8.33% 
S9: 12.50% 
T2: 8.33% 
T3: 8.33% 
T4: 8.33% 
E9: 4.00% 
T5: 33.33% 
E10: 8.00% 
T6: 25.00% 
E1: 8.00% 
E2: 12.00% 
E3: 8.00% 
E4: 8.00% 
E5: 12.00% 
E6: 8.00% 
E7: 16.00% 
E8: 16.00% 
En1: 5.56%
En2: 5.56%
En3: 22.22%
En4: 16.67%
En5: 16.67%
En6: 11.11%
En7: 22.22%
CS1: 6.25%
CS2: 18.75%
CS3: 12.50%
CS4: 18.75%
CS5: 6.25%
CS6: 18.75%
CS7: 12.50%
CS8: 6.25%
IM1: 16.67%
IM2: 25.00%
IM3: 16.67%
IM4: 8.33%
IM5: 8.33%
IM6: 25.00%
Tr1: 25.00%
Tr2: 8.33%
Tr3: 8.33%
Tr4: 16.67%
Tr5: 16.67%
Tr6: 25.00% 
 114
Fourthly, in the field of medical sciences: 
Fig. 6. Weight of each criterion for evaluating socio-economic efficiency of medical sciences topics /projects 
(Source: Authors synthesize from research results) 
The results show that the importance of criteria in terms of technology and science is most concerned, followed by the efficiency 
in terms of society, culture, infomation, management and training. By contrast, the efficiency in terms of economics and 
environment is the least important in the topics/projects in the medical sciences field. Finally, in the field of engineering and 
technological sciences, 
Fig. 6. Weight of each criterion for evaluating socio-economic efficiency of engineering and technological sciences topics 
/projects 
 (Source: Authors synthesize from research results) 
The results of evaluating topics / projects 
S: 21.43%  T: 21.43%  E: 7.14% 
S7: 8.70% 
S8: 4.35% 
S9: 4.35%  E9: 4.17% 
E10: 4.17% 
En: 7.14% CS: 14.29% IM: 14.29% Tr: 14.29% 
S1: 17.39% 
S2: 17.39% 
S3: 13.04% 
S4: 13.04% 
S5: 13.04% 
S6: 8.70% 
T1: 25.00% 
T2: 8.33% 
T3: 16.67% 
T4: 8.33% 
T5: 25.00% 
T6: 16.67% 
E1: 8.33% 
E2: 8.33% 
E3: 8.33% 
E4: 8.33% 
E5: 12.50% 
E6: 16.67% 
E7: 16.67% 
E8: 12.50% 
En1: 21.43%
En2: 7.14%
En3: 14.29%
En4: 14.29%
En5: 7.14%
En6: 21.43%
En7: 14.29%
CS1: 14.29%
CS2: 7.14%
CS3: 14.29%
CS4: 7.14%
CS5: 21.43%
CS6: 21.43%
CS7: 7.14%
CS8: 7.14%
IM1: 23.08%
IM2: 23.08%
IM3: 15.38%
IM4: 7.69%
IM5: 23.08%
IM6: 7.69%
Tr1: 25.00%
Tr2: 8.33%
Tr3: 25.00%
Tr4: 16.67%
Tr5: 8.33%
Tr6: 16.67% 
The results of evaluating topics / projects 
S: 18.75%  T: 18.75%  E: 12.50%  CS: 6.25%
S7: 8.70% 
S8: 4.35% 
S9: 4.35%  E9: 4.00% 
E10: 4.00% 
En: 12.50% IM: 12.50% Tr: 18.75% 
S1: 17.39% 
S2: 13.04% 
S3: 13.04% 
S4: 13.04% 
S5: 17.39% 
S6: 8.70% 
T1: 16.67% 
T2: 25.00% 
T3: 8.33% 
T4: 8.33% 
T5: 16.67% 
T6: 25.00% 
E1: 16.00% 
E2: 8.00% 
E3: 8.00% 
E4: 8.00% 
E5: 12.00% 
E6: 16.00% 
E7: 12.00% 
E8: 12.00% 
En1: 21.43%
En2: 21.43%
En3: 14.29%
En4: 7.14%
En5: 7.14%
En6: 14.29%
En7: 14.29%
CS1: 15.38%
CS2: 7.69%
CS3: 15.38%
CS4: 7.69%
CS5: 23.08%
CS6: 15.38%
CS7: 7.69%
CS8: 7.69%
IM1: 25.00%
IM2: 8.33%
IM3: 25.00%
IM4: 16.67%
IM5: 16.67%
IM6: 8.33%
Tr1: 30.00%
Tr2: 10.00%
Tr3: 10.00%
Tr4: 10.00%
Tr5: 20.00%
Tr6: 20.00% 
N. T. Nguyen et al. /Accounting 6 (2020) 115
The results show that the importance of criteria in terms of science, technology and training is most concerned, followed by the 
efficiency in terms of economics, environment, infomation and management while the efficiency in terms of culture and society 
is considered the least important in the topics/projects in engineering and technological sciences field. Based on the above 
weights, we can calculate the total score of the topics/projects by the following formulas: 
Q = ∑ ݆ܲ଻௝ ∑ ܶ݅ ∗ ܲ݅௡௜ (%) 
In which: Q: the total score of a topic / project 
 Pj: the weight of the criterion (j = 1,2, ..., 7) and is calculated as a percentage (%) 
 Ti: the score of the criterion got according to the questionnaire. 
 Pi: the weight of component criterion ith (i = 1,2 ... n), calculated in percentage (%) 
The score of a topic/ project can be ranked according to the following levels: 
Grade A (or Excellent) includes topics / projects with the score from over 80 to 100 points. Topics / projects of this level are 
assessed to have very significant socio-economic efficiency. 
Grade B (or very good) includes topics / projects with the score from over 60 to 80 points. Topics / projects of this level are 
assessed to have significant socio-economic efficiency. 
Grade C (or Good) includes topics/ projects with the score from over 40 to 60 points. Topics / projects of this level are assessed 
to have moderate socio-economic efficiency. 
Grade D (or Medium) includes topics/ projects with the score from over 20 to 40 points. Topics/projects of this level are assessed 
to have low socio-economic efficiency. 
Grade E (or inefficient) includes topics/projects with a score of 0 to 20 points. Topics/projects of this level are assessed to have 
no socio-economic efficiency. It is possible that the completed project may not be transferred to organizations / individuals for 
use even though the project is practical. 
5. Conclusion 
The combination of expert method and Analytic Hierarchy Process method in determining evaluation weights for evaluating 
socio-economic efficiency of topics/projects after acceptance according to scientific sub-disciplines in Binh Dinh province 
which includes agricultural science, natural science, social sciences and humanities, medical sciences, engineering and 
technological sciences is very important. It helps Binh Dinh government to assess the socio-economic efficiency of the topics/ 
projects after acceptance. Thereby, Binh Dinh province has suitable orientations and investments in research activities in 
different fields so that they can bring high efficiency. The results show that the importance of the evaluation criteria is different 
in each research field. In details, for the topics/ projects in Agricultural Science, the efficiency criteria of science, technology 
and economics are considered the most important criteria; For topics/projects in the field of Natural Science, the efficiency of 
science and management information is most appreciated; For topics/projects in the field of Social Sciences and Humanities, 
the efficiency criteria of economic, socio-cultural efficiency and management- information are considered the most important 
criteria; For topics/projects in the field of medical Science, the efficiency criteria of science and technology are the most 
important criteria; For topics/ projects in the field of engineering and technological sciences, the criteria of science, technology 
and training are considered the most important criteria. 
Acknowledgment 
The authors gratefully acknowledge financial support from Binh Dinh Province People’s Committee for research code: 12-05-
2018.	
References 
Airaghi, A., Busch, N. E., Georghiou, L., Kuhlmann, S., Ledoux, M. J., van Raan, A., & Viana Baptista, J. (1999). Options and 
limits for assessing the socio-economic impact of European RTD programmes. Report to the European Commission. 
Geuna, A., & Piolatto, M. (2016). Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least 
for a while). Research Policy, 45(1), 260-271. 
Antonella, P., Francesca, S., Shenja, V. D. G., & Marie, D. (2012). Socio-economic Impact Assessment of Research Projects, 
Project funded by the European Commission “Information Society and Media Directorate - General. Support Action. 
Griniece, E., Alasdair R., & Jelena A. (2015). Evaluating and Monitoring the SocioEconomic Impact of Investment in Research 
Infrastructures. www.technopolis-group.com. 1-18. 
 116
European Commission (2014). Horizon 2020 Interim Evaluation – Maximising the Impact of EU Research and Innovation. 
Hien, P. D. (2010). A comparative study of research capabilities of East Asian countries and implications for Vietnam. Higher 
Education, 60(6), 615-625. 
Kolsch, D., Saling, P., Kicherer, A., Grosse-Sommer, A., & Schmidt, I. (2008). How to measure social impacts? A socio-eco-
efficiency analysis by the SEEBALANCE® method. International Journal of Sustainable Development, 11(1), 1-23. 
Colledge, L. (2016). A fresh look at the socio-economic impact of research. https://www.elsevier.com/connect/a-fresh-look-at-
the-socio-economic-impact-of-research 
Nguyen Ba Tra, & Vo Ngoc Anh (2015). Evaluating socio-economic efficiency of science and technology tasks of Binh Dinh 
province in the period of 1995-2005. Provincial science and technology project. Binh Dinh. Vietnam. 
Nguyen Truong Xuan (2012). Establish a system of criteria for evaluating scientific and technological performance as well as 
scientific research of universities and colleges of the Ministry of Education and Training. Ministry-level scientific research 
projects. Ministry of Agriculture and Rural Development. Ha Noi. 
Nguyen Van Tuan (2016). Vietnam's scientific productivity in the 2001-2015 period. 
2001-2015.html 
Russell Group. (2010). The economic impact of research conducted in Russell Group universities. London: Russell Group. 
Saaty, T.L (1980). The Analytic Hierarchy Process. McGraw-Hill, New York. 
Saaty, T. L. (2008). Decision making with the analytic hierarchy process. International journal of services sciences, 1(1), 83-
98. 
Thai Ngoc Chien (2012). Evaluate the efficiency of provincial projects using the budget for science & technology in Khanh Hoa 
province in the 2006-2010 period. Provincial science and technology project. Khanh Hoa. Viet Nam. 
Vietnam Academy for ater resources (2014). Report on developing criteria for evaluating the efficiency of irrigation’s science 
and technology activities. Ha Noi. 
© 2019 by the authors; licensee Growing Science, Canada. This is an open access article distributed 
under the terms and conditions of the Creative Commons Attribution (CC-BY) license 
(