An Integrated Perspective of User Evaluating Personalized Recommender Systems : Performance-Driven or User-Centric
Abstract
This study focused on user evaluation for personalized recommender systems with the integrated view of performance of the system and user attitude of recommender systems. Since users’ evaluations of recommender systems can be affected by recommendation outcomes and presentation methods, both system performances based on outcomes and user attitudes formed by the presentation methods should be considered when explaining users’ evaluations. However, an integrated view of system performance and user attitudes has not been applied to explain users’ evaluation of recommender systems. Thus, the goal of this study is to explain users’ evaluations of recommender systems under the integrated view of predictive features and explanation features at the same time.
Our findings suggest that social presence, both accuracy and noveltyhave impacts onuser satisfaction for recommender systems. Especially, predictive features including accuracy and novelty affected user satisfaction. Novelty as well as accuracy is one of the significant factors for user satisfaction while ecommender systems provided usual items users have experienced when systems provide serendipitous items. Likewise, explanation features with social presence and self-reference were important for user evaluation of personalized recommender systems. For explanation features, while social presence appears as one of important factors to user atisfaction of evaluating personalized recommendations, self-reference has no significant effect on user’s satisfaction for recommender systems when compared to the result of social presence. Self-referencing messages did not affect user satisfaction but the levels of self-referencing are different between low and high groups in the experiment.
Full Text:
PDFReferences
서길수, “과업의 특성과 매체 경험이 인지된 매체 풍요도와 사회적 존재성에 미치는 영향”, Asia Pacific Journal of Information Systems, 제8권, 제3호, pp. 119-134, 1998.
손재봉, 서용무, “협업필터링 시스템에서 Degree of March를 이용한 성능 향상”, Information System Review, 제8권, 제2호, pp. 139-154, 2006.
엄태영, 김우주, 박상언, “태그네트워크를 이용한 개인화 북마크 추천시스템”, 한국전자거래학회지, 제15권, 제4호, pp. 181-195, 2010.
이석준, 이희춘, “협업필터링 추천에서 대응평균 알고리즘의 예측성능에 관한 연구”,Information System Review, 제9권, 제1호, pp. 85-103, 2007.
이재식, 명훈식, “사례기반 추론을 이용한 인터넷 서점의 서적 추천시스템 개발”, 한국전자거래학회지, 제13권, 제4호, pp. 173-191, 2008.
Adomavicius, G. and Tuzhilin, A., “Personalization Technologies : A Processoriented Perspective,”Communications of the ACM, Vol. 48, No. 10, pp. 83-90, 2005.
Al-Natour, S., Benbasat, I., and Cenfetelli, R. T., “The Effects of Process and Outcome Similarity on Users’ valuations of Decision Aids,” Decision Sciences, Vol. 39, No. 2, pp. 175-211, 2008.
Arazy, O., Kumar, N., and Shapira, B., “A Theory-Driven Design Framework for Social Recommender Systems,”Journal of the Association for Information Systems, Vol. 11, No. 9, pp. 455-490, 2010.
Berlyne, D. E. (Ed.), Novelty, Complexity, and Interestingness, New York : Wiley, 2004.
Burke, R., “Hybrid Recommender Systems : Survey and Experiments,” User Modeling and User-Adapted Interaction, Vol. 12, No. 4, pp. 331-370, 2002.
Burnkrant, R. E. and Unnava, H. R., “Effects of Self-referencing on Persuasion,”Journal of Consumer Research, Vol. 22, No. 1, pp. 17-26, 1995.
Chin, W. W. (Ed.), “The partial least squares approach to structural equation modeling : Modern Methods for Business Research, Mahwah,” NJ, US : Lawrence Erlbaum Associates Publishers, 1998.
Choeh, J. Y. and Lee, H. J., “Mobile push personalization and user experience,” AI Communications, Vol. 21, No. 2-2, pp. 183-193, 2008.
Choi, J., Lee, H. J., and Kim, Y. C., “The Influence of Social Presence on Evaluating Personalized Recommender Systems,” Paper presented at the PACIS 2009 India, http://aisel.aisnet.org/pacis2009/49, 2009.
Choi, J., Lee, H. J., and Kim, Y. C., “The Influence of Social Presence on Costomer Intention to Reuse Online Recommender Systems : The Roles of Personalization and Product Type,” International Journal of Electronic Commerce, Vol. 16, No. 1, pp. 129-153, 2011.
Cyr, D., Hassanein, K., Head, M., and Ivanov, A., “The Role of Social Presence in Establishing Loyalty in e-Service Environments,” Interacting with Computers, Vol. 19, No. 1, pp. 43-56, 2007.
Davis, F. D., “Perceived Usefulness, Perceived Ease of Use and User Acceptance of Information Technology,” MIS Quarterly, Vol. 13, No. 3, pp. 319-340, 1989.
Escalas, J. E., “Self-Referencing and Persuasion : Narrative Transportation versus Analytical Elaboration,”Journal
of Consumer Research, Vol. 33, No. 4, pp. 421-429, 2007.
Fornell, C. and Larcker, D. F., “Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,” Journal of Marketing Research, Vol. 18, No. 3, pp. 39-50, 1981.
Fouss, F. and Saerens, M., “Evaluating performance of recommender systems : An experimental comparison,” Paper presented at the IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, 2008.
Gefen, D., Karahanna, E., and Straub, D. W., “Trust and TAM in Online Shopping : An Integrated Model,” MIS Quarterly, Vol. 27, No. 1, pp. 51-90, 2003.
Gefen, D. and Straub, D. W., “Consumer Trust in B2C e-Commerce and the Importance of Social Presence : Experiments in e-Products and e-Services,” Omega, Vol. 32, No. 6, pp. 407-424, 2004.
Häubl, G. and Murray, K. B., “Double Agent : Assessing the Role of Electronic Product Recommendation System,”Sloan Management Review, Vol. 47, No. 3, pp. 8-12, 2006.
Häubl, G. and Trifts, V., “Consumer Decision Making in Online Shopping Environments : The Effects of Interactive Decision Aids,” Marketing Science, Vol. 19, No. 1, pp. 4-21, 2000.
Hassanein, K. and Head, M., “The Impact of Infusing Social Presence in the Web Interface : An Investigation Across Different Products,” International Journal of Electronic Commerce, Vol. 10, No. 2, pp. 31-55, 2006.
Herlocker, J. L., Konstan, J. A., Terveen, L. G., and Riedl, J. T., “Evaluating Collaborative Filtering Recommender Systems,”ACM Transactions on Information Systems, Vol. 22, No. 1, pp. 5-53, 2004.
Ho, S. Y. and Kwok, S. H., “The Attraction of Personalized Service for Users in m-Commerce,” ACM SIGecom Exchanges, Vol. 3, No. 4, pp. 10-18, 2003.
Holbrook, M. B. (Ed.), “Introduction : The Esthetic Imperative in Consumer Research,” MI : Association for Consumer Research, 1981.
Karahanna, E. and Limayem, M., “E-Mail and V-Mail Usage : Generalizing Across Technologies,” Journal of Organizational Computing and Electronic Commerce,
Vol. 10, No. 1, pp. 49-66, 2000.
Komiak, S. Y. X. and Benbasat, I., “The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents,” MIS Quarterly, Vol. 30, No. 4, pp. 941-960, 2006.
Kramer, T., “The effect of measurement task transparency on preference construction and evaluations of personalized recommendations,” Journal of Marketing Research, Vol. 44, No. 2, pp. 224-233, 2007.
Kumar, N. and Benbasat, I., “The Influence of Recommendations and Consumer Reviews on Evaluations of Websites,” Information Systems Research, Vol. 17, No. 4, pp. 425-429, 2006.
Lee, H. J., Kim, J. W., and Park, S. J., “Understanding Collaborative Filtering Parameters for Personalized Recommendations in e-Commerce,” Electronic Commerce Research, Vol. 7, No. 3-4, pp. 45-70, 2007.
Liang, T., Lai, H., and Ku, Y., “Personalized Content Recommendation and User Satisfaction : Theoretical Synthesis and Empirical Findings,” Journal of Management Information Systems, Vol. 23, No. 3, pp. 45-70, 2007.
Linden, G., Smith, B., and York, J., “Amazon.com Recommendations : Itemto-item Collaborative Filtering,” IEEE Internet Computing, Vol. 7, No. 3, pp. 76-80, 2003.
Martin, R. and Hewstone, M., “Social-influence Processes of Control and Change : Conformity, Obedience to Authority and Innovation,” London : Sage, 2003.
Meyers-Levy, J. and Peracchio, L. A., “Moderators of the Impact of Self-Reference on Persuasion,” Journal of Consumer Research, Vol. 22, No. 4, pp. 408-423, 1996.
Moorman, C., “Organizational Market Information Processes : Cultural Antecedents and New Product Outcomes,”Journal of Marketing Research, Vol. 32, No. 3, pp. 318-335, 1995.
Oliver, R. L., Robertson, T. S., and Mitchell, D. J., “Imagine and Analyzing in Response to New Product Advertising,”Journal of Advertising, Vol. 22, No. 4, pp. 35-50, 1993.
Oliver, R. L. and Swan, J. E., “Equity and disconfirmation perceptions as influences on merchant and product satisfaction,” Journal of Consumer Research, Vol. 16, No. 3, pp. 372-383, 1989.
Osgood, C. E., Succi, G. J., and Tannenbaum, P. H., “The Measurement of Meaning,” IL : University of Illinois, 1957.
Pu, P. and Chen, L., “A User-Centric Evaluation Framework of Recommender Systems,”Paper presented at the In proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and their Interfaces (UCERSTI), 2010.
Resnick, P., Iacovou, N., Suchak, N., and Bergstrom, P., “GroupLens : An Open Architecture for Collaborative Filtering of Netnews,” Paper presented at the the 1994 ACM Conference on computersupported cooperative work, 1994.
Resnick, P. and Varian, H. R., “Recommender Systems,”Communications of the ACM, Vol. 40, No. 3, pp. 56-58, 1997.
Ricci, F., “Mobile Recommender Systems,” International Journal of Information Technology and Tourism, Vol. 12, No. 3, pp. 1-24, 2011.
Rogers, T. B., Kuiper, N. A., and Kirker, W., “Self-reference and the encoding of personal information,” Journal of Personality and Social Psychology, Vol. 35, No. 9, pp. 677-688, 1977.
Sarwar, B., Karypis, G., Konstan, J., and Riedl, J., “Item-based Collaborative Filtering Recommendation Algorithms,”Paper presented at the Proceedings of the 10th international conference on World Wide Web, Hong Kong, 2001.
Shani, G. and Gunawardana, A., “Recommender Systems and book : Evaluating Recommendation Systems : Springer,” 2011.
Sinha, R. and Swearingen, K., “Comparing Recommendations made by Online Systems and Friends,”Paper presented at the DELOS-NSF Workshop on Personalization and Recommender Systems in Digital ibraries, 2001.
Swearingen, K. and Sinha, R., “ACM SIGIR Workshop on Recommender Systems Beyond Algorithms : An HCI Perspective on Recommender Systems,” 2001.
Tam, K. Y. and Ho, S. Y., “Web Personalization As A Persuasion Strategy : An Elaboration Likelihood Model Perspective,” Information Systems Research, Vol. 16, No. 3, pp. 271-291, 2005.
Tam, K. Y. and Ho, S. Y., “Understanding the Impact of Web Personalization on User Information Processing and Decision Outcome,”MIS Quarterly, Vol. 30, No. 4, pp. 865-890, 2006.
Thirumalai, S. and Sinha, K. K., “Customization Strategies in Electronic Retailing : Implications of Customer Purchase Behavior,” Decision Sciences, Vol. 40, No. 1, pp. 5-36, 2009.
Unger, L., “Consumer Marketing Trends in the 1980s : When Growth Slows,” Journal of Consumer Research, Vol. 9, No. 2, pp. 69-73, 1981.
Wang, W. and Benbasat, I., “Recommendation Agents for Electronic Commerce : Effects of Explanation Facilities on Trusting Beliefs,” Journal of Management Information Systems, Vol. 23, No. 4, pp. 217-246, 2007.
Xiao, B. and Benbasat, I., “E-commerce Product Recommendation Agents : Use, Characteristics, and Impact,”MIS Quarterly, Vol. 31, No. 1, pp. 137-209, 2007.
Xu, D. J., “The Influence of Personalization in Affecting Consumer Attitudes Toward Mobile Advertising in China,”Journal of Computer Information Systems, Vol. 47, No. 2, pp. 9-19, 2006.
Refbacks
- There are currently no refbacks.