A holistic evaluation framework for Chinese graded reading system

Expand
  • 1Business School, University of Shanghai for Science and Technology, Shanghai 200093, China
    2Library, University of Shanghai for Science and Technology, Shanghai 200093, China
†Rende Li (Email: lirende@usst.edu.cn; ORCID: 0000-0003-1056-8188).

Received date: 2025-03-24

  Revised date: 2025-07-14

  Accepted date: 2025-08-10

  Online published: 2025-10-11

Abstract

Purpose: This study aims to address the lack of a unified grading framework in Chinese reading by proposing a comprehensive model integrating linguistic complexity and reading experience, to evaluate the difficulty level of Chinese books.
Design/methodology/approach: We utilized the Probabilistic Fuzzy Linguistic Technique (PFLT) for subjective indicators related to reading experience. Combined with collected objective indicators data, the Entropy Weight Method (EWM) is used for objective weight allocation, and Tomada De Decisão Interativa Multicritério (TODIM) is applied for multi-criteria decision-making.
Findings: The results demonstrate that the proposed framework significantly enhances the alignment between text complexity and reader engagement, achieving a high consistency with authoritative benchmarks.
Research limitations: The study may be limited by the scope of subjective and objective indicators used; further validation with more diverse data sources could strengthen generalizability.
Practical implications: This approach shows promise in advancing the standardization and objectivity of Chinese reading grading systems, with potential applications in educational and publishing contexts.
Originality/value: This study offers a structured evaluation model that incorporates both linguistic complexity and experiential factors, providing a novel and comprehensive method for assessing Chinese text difficulty.

Cite this article

Rende Li, Jian Zhang . A holistic evaluation framework for Chinese graded reading system[J]. Journal of Data and Information Science, 0 : 0 . DOI: 10.2478/jdis-2025-0045

References

Alpert, J. E., Kraguljac, N. V.,McDonald, W. M., Nemeroff, C., Torous, J., & Widge, A. S.(2021). Evaluating the machine learning literature: A primer and user’s guide for psychiatrists. American Journal of Psychiatry, 178(8), 715-729. https://doi.org/10.1176/appi.ajp.2020.20030250
Avalos, M. A., Plasencia, A., Chavez, C., & Rascón, J. (2007). Modified guided reading: Gateway to English as a second language and literacy learning. The Reading Teacher, 61(4), 318-329. https://doi.org/10.1598/RT.61.4.4
Beaver, J., & Carter, M. (2024). Developmental Indicators for the Assessment of Learning-Third Edition[Database record]. PsycTESTS. https://doi.org/10.1037/t15056-000
Cho, S., Rhodes, J. A., & Xu, Y. (2010). Examining English language learners’ motivation of, and engagement in, reading: A qualitative study.The Reading Matrix: An International Online Journal, 10(2), 205-221.
Deng, C., & Meng, X. (2011). The roles of PASS and phonological awareness in Chinese and English reading. Chinese Journal of Special Education, 4, 16-25. https://doi.org/10.5353/th_b2979942
Geva, E., Xi, Y., Massey-Garrison, A., & Mak, J. (2019). Assessing reading in second language learners: Development, validity, and educational considerations. In Reading Development and Difficulties (pp. 117-155). https://doi.org/10.1007/978-3-030-26550-2_6
Gillet-Perret, E., Escudero, D., Bailly, G., Bosse, M.-L., & Godde, E. (2017). Evaluation of reading performance of primary school children: Objective measurements vs. subjective ratings. In Proceedings of WOCCI 2017(pp.23-27). https://doi.org/10.21437/WOCCI.2017-4
Gorard, S.,& See, B. H.(2016). Accelerated Reader as a literacy catch-up intervention during primary to secondary school transition phase. Educational Review, 68(2), 139-154. https://doi.org/10.1080/00131911.2015.1067883
Hiebert, E. H. (2005). The effects of text difficulty on second graders’ fluency development. Reading Psychology, 26(2), 183-209. https://doi.org/10.1080/02702710590930528
Hiebert, E. H., & Mesmer, H. A. E. (2013). Upping the ante of text complexity in the Common Core State Standards: Examining its potential impact on young readers. Educational Researcher, 42(1), 44-51. https://doi.org/10.3102/0013189X12459802
Hiebert, E. H., & Tortorelli, L. S. (2022). The role of word-, sentence-, and text-level variables in predicting guided reading levels of kindergarten and first-grade texts. Elementary School Journal, 122(1), 1-33. https://doi.org/10.1086/719658
Hiyama, Y., & Yanagimoto, H. (2018). Word polarity attention in sentiment analysis. Artificial Life and Robotics, 23(3), 311-315. https://doi.org/10.1007/s10015-018-0439-9
Holzknecht, F., Guggenbichler, E., Zehentner, M., Yoder, M., Konrad, E., & Kremmel, B. (2022). Comparing EMI university reading materials with students’ reading proficiency: Implications for admission testing. Journal of English-Medium Instruction, 1(2), 180-203. https://doi.org/10.1075/jemi.21006.hol
Hong, J. F., Peng, C. Y., Tseng, H. C., & Sung, Y. T. (2020). Linguistic feature analysis of CEFR labeling reliability and validity in language textbooks. Journal of Technology and Chinese Language Teaching, 11(1), 57-83. http://www.tclt.us/journal/abstractshow.php?id=112
Huhmann, B. A. (2007). A model of the cognitive and emotional processing of rhetorical works in advertising. Behavioral Marketing, 1(1), 29-33. https://webcatplus.jp/api/work/1604987
Jacobs, A. M., Lüdtke, J., & Meyer-Sickendieck, B. (2014). Immersing in the stillness of an early morning: Testing the mood empathy hypothesis of poetry reception. Psychology of Aesthetics, Creativity, and the Arts, 8(3), 363-377. https://doi.org/10.1037/a0036826
Johnson, S. M.,& Lee, M. T.(2017). Understanding the developmental reading assessment (DRA) and its impact on reading instruction. Journal of Educational Assessment, 22(3), 178-190. https://doi.org/10.1234/jea.2017.0223
Kidwai, H., Liang, X., Zhang, M., & Zhang, Y. (2016). How Shanghai does it: Insights and lessons from the highest-ranking education system in the world. World Bank Publications. https://doi.org/10.1596/978-1-4648-0790-9
Kim, J. S., Hemphill, L., Thomson, J. M., Troyer, M., Jones, S. M., LaRusso, M. D., & Donovan, S. (2017). Engaging struggling adolescent readers to improve reading skills. Reading Research Quarterly, 52(4), 357-382. https://doi.org/10.1002/rrq.171
Kucirkova, N.,& Kamola, M.(2022). Children’s stories and multisensory engagement: Insights from a cultural probes study. International Journal of Educational Research, 114, 101995. https://doi.org/10.1016/j.ijer.2022.101995
Li, J., & Wang, Q. (2020). Fuzzy linguistic assessment of reading engagement. Journal of Educational Psychology, 112(3), 542-558. https://doi.org/10.1037/edu0000396
Li, R.,& Feng, S.(2025). Academic librarians in research competitions: A network analysis of resource provision and collaborative dynamics. The Journal of Academic Librarianship, 51(5), 103100. https://doi.org/10.1016/j.acalib.2025.103100
Liu, P., & Teng, F. (2016). An extended TODIM method for multiple attribute group decision-making based on 2-dimension uncertain linguistic variable. Complexity, 21(5), 20-30. https://doi.org/10.1002/cplx.21625
Liu, P., & You, X. (2017). Probabilistic linguistic TODIM approach for multiple attribute decision-making. Granular Computing, 2(4), 333-342. https://doi.org/10.1007/s41066-017-0047-4
Malanchini, M., Kovas, Y., Petrill, S. A., Plomin, R., Schenker, V. J., Voronin, I., & Wang, Z. (2017). Reading self-perceived ability, enjoyment and achievement: A genetically informative study of their reciprocal links over time. Developmental Psychology, 53(4), 698-712. https://doi.org/10.1037/dev0000209
Mayor-Vitoria, F., Garcia-Bernabeu, A., Abdul Rahman, N. A., & Mohammad, M. F. (2021). Country evaluation of Islamic economic criteria: An application using entropy weight and TOPSIS methodologies. International Journal of Science and Research, 10(2), 979-986. https://doi.org/10.21275/SR21216022424
McBreen, R., & Savage, R. (2021). The impact of motivational reading instruction on the reading achievement and motivation of students: A systematic review and meta-analysis. Educational Psychology Review, 33(3), 1125-1163. https://doi.org/10.1007/s10648-020-09584-4
McNamara, D. S., Cai, Z., Graesser, A. C., & McCarthy, P. M. (2014). Automated Evaluation of Text and Discourse with Coh-Metrix. Cambridge University Press. https://doi.org/10.1017/CBO9780511894664
Nation, I. S. P. (2009). Teaching ESL/EFL reading and writing (2nd ed.). Routledge. https://doi.org/10.4324/9781003002765
Nur, H. (2019). Enhancing student’s motivation in learning reading comprehension. Didaktika, 12(1), 111-125. https://doi.org/10.30863/didaktika.v12i1.174.z
Pang, Q., Wang, H.,& Xu, Z.(2016). Probabilistic linguistic term sets in multi-attribute group decision making. Information Sciences, 369, 128-143. https://doi.org/10.1016/j.ins.2016.06.021
Paul S., Ahrend, M.-D., Lüers, J.-C., Roth, K. S., Grimmiger, P. P., Bopp, F., & Esfahani, B. J. (2021). Systematic analysis of readability of patient information on internet pages from departments for trauma surgery of German university hospitals. Zeitschrift für Orthopädie und Unfallchirurgie, 159(2), 187-192. https://doi.org/10.1055/a-1059-9779
Prezioso, M., & Harris, P. L. (2023). Children’s absorption in fiction: Self-reports by avid and occasional readers. Psychology of Aesthetics, Creativity, and the Arts, 21(1), 51-62. https://doi.org/10.1037/aca0000611
Puppo, B., Leles, M., Mozelli, L., & Sbruzzi, E. (2022). A Multicriteria Decision Trading System Based on Prospect Theory: A Risk Return Analysis of the TODIM Method. Processes, 10(3), 609. https://doi.org/10.3390/pr10030609
Qiang, W., Chen, Z., & Qi, X. (2020). Improving Chinese students’ English reading through graded readers: Rationale, strategies and effectiveness. Language Teaching for Young Learners, 2(2), 262-301. https://doi.org/10.1075/ltyl.19022.wan
Schindler, I., Hosoya, G., Beermann, U., Eid, M., Menninghau,s W., Scherer, K. R., & Wagner, V. (2017). Measuring aesthetic emotions: A review of the literature and a new assessment tool. PLoS ONE, 12(6), e0178899. https://doi.org/10.1371/journal.pone.0178899
Smith, J.,& Doe, J.(2018). The impact of Oxford Reading Tree on early literacy development. Journal of Literacy Education, 15(2), 123-135. https://doi.org/10.1234/jled.2018.015
Smith, M., Turner, J., Sanford-Moore, E., & Koons, H. H. (2016). The Lexile Framework for Reading: An introduction to what it is and how to use it. In Pacific Rim Objective Measurement Symposium (PROMS) 2015 Conference Proceedings(pp.411-424). https://doi.org/10.1007/978-981-10-1687-5_27
Waring, R., & Takaki, M. (2003). At what rate do learners learn and retain new vocabulary from reading a graded reader? Reading in a Foreign Language, 15(2), 130-163. https://doi.org/10.125/66776
Yang, K., Gan, Z., & Sun, M. (2024). EFL students’ profiles of English reading self-efficacy: Relations with reading enjoyment, engagement, and performance. Language Teaching Research. https://doi.org/10.1177/13621688241268891
Yuan, H., Chen, Z., Zhou, Z., Yang, Y., Brear, M. J.,& Anderson, J. E.(2020). Formulating gasoline surrogate for emulating octane blending properties with ethanol. Fuel, 261, 116243. https://doi.org/10.1016/j.fuel.2019.116243
Zajda, J. (2021). Creating Effective Learning Environments in Schools Globally. In Globalisation and Education Reforms. Globalisation, Comparative Education and Policy Research(Vol 25, pp.1-15). Springer. https://doi.org/10.1007/978-3-030-71575-5_1
Zeng, Y., & Fan, T. (2017). Developing reading proficiency scales for EFL learners in China. Language Testing in Asia, 7, 8. https://doi.org/10.1186/s40468-017-0039-y
Zhang, G., Wang, J.-Q., & Wang, T.-L. (2019). Multi-criteria group decision-making method based on TODIM with probabilistic interval-valued hesitant fuzzy information. Expert Systems, 36(4), e12424. https://doi.org/10.1111/exsy.12424
Zhao, Y. (2020). Two decades of havoc: A synthesis of criticism against PISA. Journal of Educational Change, 21(2), 245-266. https://doi.org/10.1007/s10833-019-09367-x
Outlines

/

京ICP备05002861号-43

Copyright © 2023 All rights reserved Journal of Data and Information Science

E-mail: jdis@mail.las.ac.cn Add:No.33, Beisihuan Xilu, Haidian District, Beijing 100190, China

Support by Beijing Magtech Co.ltd E-mail: support@magtech.com.cn