- 无标题文档
查看论文信息

论文题名(中文):

 融合临床指南引用目的分析的论文重要性评价研究    

姓名:

 杨爽    

论文语种:

 chi    

学位:

 硕士    

学位类型:

 学术学位    

学校:

 北京协和医学院    

院系:

 北京协和医学院医学信息研究所    

专业:

 图书情报与档案管理-情报学    

指导教师姓名:

 安新颖    

校内导师组成员姓名(逗号分隔):

 安新颖 胥美美    

论文完成日期:

 2025-04-10    

论文题名(外文):

 A study on the evaluation of the importance of papers integrating the analysis of the citation purposes of clinical guidelines    

关键词(中文):

 国际临床指南 引用目的 大语言模型 引文重要性分析    

关键词(外文):

 International Clinical guidelines Purpose of reference Large language model Citation importance analysis    

论文文摘(中文):

在医学科技成果评价中,学术成果被临床指南引用是衡量论文临床重要性的重要指标。被引频次能够在一定程度上反映其被临床指南使用的程度,但是实际评价过程中将所有的引用视为同等重要,忽视了施引文献引用的目的,以及被引论文对施引论文的作用。这影响了评价过程中被引论文的重要性得分,进而影响科技人才、机构、项目评价的科学性。因此,亟需从引用目的角度细粒度判断引用特征,确立能够全面反映学术论文对临床指南贡献的量化指标。

为解决上述问题,本研究旨在建立融合引用强度、引用目的的引文重要性评价指数,从而更准确地衡量被引文献的价值,以进一步提升医学科技成果的质量。本研究主要研究内容包括:(1)借助文献调研法和专家咨询法确定临床指南引用目的分类体系,同时结合优序图法计算各引用目的重要性权重,基于引用强度及引用目的重要性权重计算引文重要性指数。(2)构建国际临床指南引用目的标注数据集,基于大语言模型构建临床指南引文引用目的分类模型CMCP-CPG。同时,对比不同深度学习模型、不同引用目的分类模型的优劣,评价临床指南引用目的分类模型的效果。(3)将构建的临床指南引用目的分类模型应用于肺癌领域临床指南,实现引用目的自动分类的同时,计算引文重要性指数。将基于引用目的的引文重要性排名结果与基于引用强度、引用位置指标下的引文重要性排名结果对比分析,探讨融合引用强度和引用目的引文重要性指标的作用和意义。

研究发现,最终确定的临床指南引用目的分类体系与初步构建的分类体系一致,包括“背景阐述”“方法借鉴”“局限性分析”“差异对比”“潜在方案”“证据支持-高等”“证据支持-中等”“证据支持-低等”“证据支持-极低等”“证据支持-无证据等级”共十类。其次,研究发现高等、中等级别的证据支持及提供潜在方案指导的引用目的权重最高。此外,经过术语增强、指令微调得到的临床指南引用目的分类模型正确率为0.82,宏平均精确率为0.80,微平均精确率为0.82。通过与其他模型对比可知,本研究提出的CMCP-CPG模型在临床指南引用目的分类任务中能更好地捕捉语句中的语义特征,提升了分类效果。通过实证分析得到如下结论:(1)引用目的加权后的指标可有效反映引文重要性。引用强度的指标排名变化幅度小,无法表征引用强度为1的引文重要性差异,因此应同时将引用目的纳入指标范畴。此外,引文所在期刊影响因子和被引频次,均具有较高影响力。可见,基于引用目的的引文重要性指数可有效识别临床指南中的高质量研究。(2)引用目的相较于引用位置评价结果更精确。引用位置难以区分同一位置下引文的重要性差异,且引用位置指标可能对某些非重要论文赋予高于其本身价值的权重,导致其得分较高,掩盖了引文的实际价值。因此,基于引用目的的引文指标识别效果更为精准,能够识别引用位置指标难以发现的具有潜在价值的引文。

论文文摘(外文):

In the evaluation of medical scientific and technological achievements, the academic achievements cited by clinical guidelines is an important index to measure the clinical importance of papers. Citation frequency can reflect the degree to which it is used by clinical guidelines to a certain extent, but in the actual evaluation process, all citations are regarded as equally important, ignoring the purpose of citing the cited literature and the role of the cited paper on the cited paper. This affects the importance score of the cited papers in the evaluation process, and then affects the scientific nature of the evaluation of scientific and technological talents, institutions and projects. Therefore, it is urgent to judge the citation characteristics from the perspective of citation semantics and establish quantitative indicators that can fully reflect the contribution of academic papers to clinical guidelines.

To solve the above problems, this study aims to establish a citation importance evaluation index integrating citation intensity and citation purpose, so as to more accurately measure the value of cited literature and further improve the quality of medical scientific and technological achievements. The main research contents of this study include: (1) The classification system of citation purposes of clinical guidelines was determined by the literature research method and the expert consultation method, and the importance weight of each citation purpose was calculated by combining the order diagram method, and the citation importance index was calculated based on the citation intensity and importance weight of citation purposes. (2) Construct the labeled dataset of the citation purposes of international clinical guidelines, and build the classification model CMCP-CPG of the citation purposes of clinical guidelines based on the large language model. Meanwhile, compare the advantages and disadvantages of different deep learning models and classification models for different citation purposes, and evaluate the effect of classification models for citation purposes in clinical guidelines. (3) The constructed classification model for the purpose of citation of clinical guidelines was applied to clinical guidelines in the field of lung cancer to realize automatic classification of the purpose of citation and calculate the citation importance index. The results of citation importance ranking based on citation purpose are compared with those based on citation intensity and citation location, and the function and significance of combining citation intensity and citation purpose are discussed.

The study found that the final classification system for the purpose of clinical guidelines citation was consistent with the preliminary classification system. Including "background elaboration", "method reference", "limitation analysis", "difference comparison", "potential scheme", "evidence support - high", "evidence support - medium", "evidence support - low", "evidence support - extremely low", "evidence support - no evidence level" a total of ten categories. Secondly, the research found that the citation purposes of high-level and medium-level evidence support and providing potential plan guidance had the highest weight. Furthermore, the accuracy rate of the classification model for the purpose of clinical guideline citation obtained through term enhancement and instruction fine-tuning was 0.82, the macro average accuracy rate was 0.80, and the micro average accuracy rate was 0.82. By comparing with other models, it can be known that the CMCP-CPG model proposed in this study can better capture the semantic features in sentences in the classification task for the purpose of clinical guideline citation, and improve the classification effect. The following conclusions are obtained through empirical analysis: (1) The index weighted by the purpose of citation can effectively reflect the importance of citation. The index ranking of citation intensity has a small change range and cannot represent the difference of citation importance when citation intensity is 1. Therefore, the purpose of citation should be included in the index category. In addition, the impact factor and citation frequency of the journal where the citation is located have a high influence. It can be seen that the citation importance index based on the purpose of citation can effectively identify high-quality studies in clinical guidelines. (2) The purpose of citation is more accurate than the evaluation result of the citation location. The citation position makes it difficult to distinguish the differences in the importance of citations at the same position. Moreover, the citation position indicator may assign weights higher than its own value to some non-important papers, resulting in higher scores and masking the actual value of the citations. Therefore, the recognition effect of citation index based on the purpose of citation is more accurate, and it can identify the citations with potential value that the citation location index is difficult to find.

开放日期:

 2025-06-12    

无标题文档

   京ICP备10218182号-8   京公网安备 11010502037788号