Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels
<p dir="ltr">Although the effect of hyperparameters on algorithmic outputs is well known in machine learning, the effects of hyperparameters on information systems that produce user or customer segments are relatively unexplored. This research investigates the effect of varying the n...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | , |
| Published: |
2023
|
| Subjects: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1864513530922795008 |
|---|---|
| author | Bernard J. Jansen (7434779) |
| author2 | Soon-gyo Jung (7434773) Joni Salminen (7434770) |
| author2_role | author author |
| author_facet | Bernard J. Jansen (7434779) Soon-gyo Jung (7434773) Joni Salminen (7434770) |
| author_role | author |
| dc.creator.none.fl_str_mv | Bernard J. Jansen (7434779) Soon-gyo Jung (7434773) Joni Salminen (7434770) |
| dc.date.none.fl_str_mv | 2023-04-22T03:00:00Z |
| dc.identifier.none.fl_str_mv | 10.1007/s10796-023-10395-5 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/journal_contribution/Finetuning_Analytics_Information_Systems_for_a_Better_Understanding_of_Users_Evidence_of_Personification_Bias_on_Multiple_Digital_Channels/24981711 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Information and computing sciences Data management and data science Information systems Machine learning Personas Hyperparameters Analytic bias Machine learning |
| dc.title.none.fl_str_mv | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| dc.type.none.fl_str_mv | Text Journal contribution info:eu-repo/semantics/publishedVersion text contribution to journal |
| description | <p dir="ltr">Although the effect of hyperparameters on algorithmic outputs is well known in machine learning, the effects of hyperparameters on information systems that produce user or customer segments are relatively unexplored. This research investigates the effect of varying the number of user segments on the personification of user engagement data in a real analytics information system, employing the concept of persona. We increment the number of personas from 5 to 15 for a total of 330 personas and 33 persona generations. We then examine the effect of changing the hyperparameter on the gender, age, nationality, and combined gender-age-nationality representation of the user population. The results show that despite using the same data and algorithm, varying the number of personas strongly biases the information system’s personification of the user population. The hyperparameter selection for the 990 total personas results in an average deviation of 54.5% for gender, 42.9% for age, 28.9% for nationality, and 40.5% for gender-age-nationality. A repeated analysis of two other organizations shows similar results for all attributes. The deviation occurred for all organizations on all platforms for all attributes, as high as 90.9% in some cases. The results imply that decision makers using analytics information systems should be aware of the effect of hyperparameters on the set of user or customer segments they are exposed to. Organizations looking to effectively use persona analytics systems must be wary that altering the number of personas could substantially change the results, leading to drastically different interpretations about the actual user base.</p><p dir="ltr">Correction: Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels: <a href="https://dx.doi.org/10.1007/s10796-023-10403-8" target="_blank">https://dx.doi.org/10.1007/s10796-023-10403-8</a>, published online 19 May 2023.</p><h2>Other Information</h2><p dir="ltr">Published in: Information Systems Frontiers<br>License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1007/s10796-023-10395-5" target="_blank">https://dx.doi.org/10.1007/s10796-023-10395-5</a></p> |
| eu_rights_str_mv | openAccess |
| id | Manara2_5635c6b332fdacffb01e80b7e47236a6 |
| identifier_str_mv | 10.1007/s10796-023-10395-5 |
| network_acronym_str | Manara2 |
| network_name_str | Manara2 |
| oai_identifier_str | oai:figshare.com:article/24981711 |
| publishDate | 2023 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital ChannelsBernard J. Jansen (7434779)Soon-gyo Jung (7434773)Joni Salminen (7434770)Information and computing sciencesData management and data scienceInformation systemsMachine learningPersonasHyperparametersAnalytic biasMachine learning<p dir="ltr">Although the effect of hyperparameters on algorithmic outputs is well known in machine learning, the effects of hyperparameters on information systems that produce user or customer segments are relatively unexplored. This research investigates the effect of varying the number of user segments on the personification of user engagement data in a real analytics information system, employing the concept of persona. We increment the number of personas from 5 to 15 for a total of 330 personas and 33 persona generations. We then examine the effect of changing the hyperparameter on the gender, age, nationality, and combined gender-age-nationality representation of the user population. The results show that despite using the same data and algorithm, varying the number of personas strongly biases the information system’s personification of the user population. The hyperparameter selection for the 990 total personas results in an average deviation of 54.5% for gender, 42.9% for age, 28.9% for nationality, and 40.5% for gender-age-nationality. A repeated analysis of two other organizations shows similar results for all attributes. The deviation occurred for all organizations on all platforms for all attributes, as high as 90.9% in some cases. The results imply that decision makers using analytics information systems should be aware of the effect of hyperparameters on the set of user or customer segments they are exposed to. Organizations looking to effectively use persona analytics systems must be wary that altering the number of personas could substantially change the results, leading to drastically different interpretations about the actual user base.</p><p dir="ltr">Correction: Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels: <a href="https://dx.doi.org/10.1007/s10796-023-10403-8" target="_blank">https://dx.doi.org/10.1007/s10796-023-10403-8</a>, published online 19 May 2023.</p><h2>Other Information</h2><p dir="ltr">Published in: Information Systems Frontiers<br>License: <a href="https://creativecommons.org/licenses/by/4.0" target="_blank">https://creativecommons.org/licenses/by/4.0</a><br>See article on publisher's website: <a href="https://dx.doi.org/10.1007/s10796-023-10395-5" target="_blank">https://dx.doi.org/10.1007/s10796-023-10395-5</a></p>2023-04-22T03:00:00ZTextJournal contributioninfo:eu-repo/semantics/publishedVersiontextcontribution to journal10.1007/s10796-023-10395-5https://figshare.com/articles/journal_contribution/Finetuning_Analytics_Information_Systems_for_a_Better_Understanding_of_Users_Evidence_of_Personification_Bias_on_Multiple_Digital_Channels/24981711CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/249817112023-04-22T03:00:00Z |
| spellingShingle | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels Bernard J. Jansen (7434779) Information and computing sciences Data management and data science Information systems Machine learning Personas Hyperparameters Analytic bias Machine learning |
| status_str | publishedVersion |
| title | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| title_full | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| title_fullStr | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| title_full_unstemmed | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| title_short | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| title_sort | Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels |
| topic | Information and computing sciences Data management and data science Information systems Machine learning Personas Hyperparameters Analytic bias Machine learning |