Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions
<p dir="ltr">The assessment and evaluation of the academic influence of the researcher is a challenging task. This task allows the scientific community to make valuable decisions, such as identifying leading experts in a specific field, nominating candidates for scientific awards, aw...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , , , |
| Published: |
2023
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513527428939776 |
|---|---|
| author | Bilal Ahmed (341166) |
| author2 | Li Wang (15202) Ghulam Mustafa (458105) Muhammad Tanvir Afzal (4162504) Adnan Akhunzada (3134064) |
| author2_role | author author author author |
| author_facet | Bilal Ahmed (341166) Li Wang (15202) Ghulam Mustafa (458105) Muhammad Tanvir Afzal (4162504) Adnan Akhunzada (3134064) |
| author_role | author |
| dc.creator.none.fl_str_mv | Bilal Ahmed (341166) Li Wang (15202) Ghulam Mustafa (458105) Muhammad Tanvir Afzal (4162504) Adnan Akhunzada (3134064) |
| dc.date.none.fl_str_mv | 2023-09-11T06:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1109/access.2023.3309416 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Evaluating_the_Effectiveness_of_Author-Count_Based_Metrics_in_Measuring_Scientific_Contributions/25239493 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Engineering Electrical engineering Electronics, sensors and digital hardware Materials engineering Measurement Indexes Mathematics Correlation Productivity Current measurement Benchmark testing Ranking (statistics) Publishing Authoring systems Author assessment parameters AMS H index IMU LMS NASL researcher ranking |
| dc.title.none.fl_str_mv | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">The assessment and evaluation of the academic influence of the researcher is a challenging task. This task allows the scientific community to make valuable decisions, such as identifying leading experts in a specific field, nominating candidates for scientific awards, awarding scholarships/grants, promoting researchers, and selecting tenure positions. Scientists have proposed various varied and multifaceted metrics to determine the most influential researchers. These metrics are the citation count, total number of publications, hybrid approaches, h-index, and variants of the h-index. Contemporary research in this domain shows that there is no universally accepted yardstick available for determining the finest parameter to recognize the most influential researcher within a particular domain. Moreover, to recognize the potential metric, some researchers have conducted evaluation surveys. In these evaluation surveys, the researchers utilized a limited number of indices on a small and imbalanced dataset as well as on fictional case scenarios, which makes it challenging to determine the significance and influence of each metric over the others. The present study computed fourteen distinct metrics based on the author-count. Our aim is to determine the potential metrics. For experimental purposes, we collected 1050 researchers’ data from the mathematics domain. For the benchmark dataset, we have collected the awardee’s data of the last three decades of four different societies of mathematics domain. To evaluate these metrics, we first computed the Spearman correlations among the obtained values of these metrics to assess their similarities and differences. The results showed a high degree of correlation among these metrics. However, some metrics represent weak correlations, signifying that their rankings are highly dissimilar to those of the other metrics. Furthermore, the position of award winners is checked in the top 10, 50, and 100 return records based on a ranked list of each metric. The potential value of each metric such as the hf metric, indicates that 60% of the awardees in the top 10% of the ranked list are associated with this, whereas the potential value of fractional g metric is linked with 49% of the awardees in the top 100% of the ranked list. In addition, it is further scrutinized that most of the award winners lie in a top position belonging to IMS, LMS, and AMS society return by fractional g-index, gf index, and gm index, which indicates that there is some relationship between these societies and metrics.</p><h2>Other Information</h2><p dir="ltr">Published in: IEEE Access<br>License: <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1109/access.2023.3309416" target="_blank">https://dx.doi.org/10.1109/access.2023.3309416</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_868e29403098a14fe6e4d72394c2e791 |
| identifier_str_mv | 10.1109/access.2023.3309416 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/25239493 |
| publishDate | 2023 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific ContributionsBilal Ahmed (341166)Li Wang (15202)Ghulam Mustafa (458105)Muhammad Tanvir Afzal (4162504)Adnan Akhunzada (3134064)EngineeringElectrical engineeringElectronics, sensors and digital hardwareMaterials engineeringMeasurementIndexesMathematicsCorrelationProductivityCurrent measurementBenchmark testingRanking (statistics)PublishingAuthoring systemsAuthor assessment parametersAMSH indexIMULMSNASLresearcher ranking<p dir="ltr">The assessment and evaluation of the academic influence of the researcher is a challenging task. This task allows the scientific community to make valuable decisions, such as identifying leading experts in a specific field, nominating candidates for scientific awards, awarding scholarships/grants, promoting researchers, and selecting tenure positions. Scientists have proposed various varied and multifaceted metrics to determine the most influential researchers. These metrics are the citation count, total number of publications, hybrid approaches, h-index, and variants of the h-index. Contemporary research in this domain shows that there is no universally accepted yardstick available for determining the finest parameter to recognize the most influential researcher within a particular domain. Moreover, to recognize the potential metric, some researchers have conducted evaluation surveys. In these evaluation surveys, the researchers utilized a limited number of indices on a small and imbalanced dataset as well as on fictional case scenarios, which makes it challenging to determine the significance and influence of each metric over the others. The present study computed fourteen distinct metrics based on the author-count. Our aim is to determine the potential metrics. For experimental purposes, we collected 1050 researchers’ data from the mathematics domain. For the benchmark dataset, we have collected the awardee’s data of the last three decades of four different societies of mathematics domain. To evaluate these metrics, we first computed the Spearman correlations among the obtained values of these metrics to assess their similarities and differences. The results showed a high degree of correlation among these metrics. However, some metrics represent weak correlations, signifying that their rankings are highly dissimilar to those of the other metrics. Furthermore, the position of award winners is checked in the top 10, 50, and 100 return records based on a ranked list of each metric. The potential value of each metric such as the hf metric, indicates that 60% of the awardees in the top 10% of the ranked list are associated with this, whereas the potential value of fractional g metric is linked with 49% of the awardees in the top 100% of the ranked list. In addition, it is further scrutinized that most of the award winners lie in a top position belonging to IMS, LMS, and AMS society return by fractional g-index, gf index, and gm index, which indicates that there is some relationship between these societies and metrics.</p><h2>Other Information</h2><p dir="ltr">Published in: IEEE Access<br>License: <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank">https://creativecommons.org/licenses/by/4.0/</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1109/access.2023.3309416" target="_blank">https://dx.doi.org/10.1109/access.2023.3309416</a></p>2023-09-11T06:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1109/access.2023.3309416https://figshare.com/articles/journal_contribution/Evaluating_the_Effectiveness_of_Author-Count_Based_Metrics_in_Measuring_Scientific_Contributions/25239493CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/252394932023-09-11T06:00:00Z |
| spellingShingle | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions Bilal Ahmed (341166) Engineering Electrical engineering Electronics, sensors and digital hardware Materials engineering Measurement Indexes Mathematics Correlation Productivity Current measurement Benchmark testing Ranking (statistics) Publishing Authoring systems Author assessment parameters AMS H index IMU LMS NASL researcher ranking |
| status_str | publishedVersion |
| title | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| title_full | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| title_fullStr | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| title_full_unstemmed | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| title_short | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| title_sort | Evaluating the Effectiveness of Author-Count Based Metrics in Measuring Scientific Contributions |
| topic | Engineering Electrical engineering Electronics, sensors and digital hardware Materials engineering Measurement Indexes Mathematics Correlation Productivity Current measurement Benchmark testing Ranking (statistics) Publishing Authoring systems Author assessment parameters AMS H index IMU LMS NASL researcher ranking |