Publications

Asterisks (“*”) indicate equal contribution.
Please refer to my Google scholar page for the full list.

	Benchmarking Cognitive Biases in Large Language Models as Evaluators Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, Dongyeop Kang Findings of ACL, 2024  arXiv /  project page /  code /  data / Evaluated 16 large language models (LLMs) as automatic evaluators using preference ranking and introduced the Cognitive Bias Benchmark for LLMs as Evaluators (COBBLER), revealing significant cognitive biases and misalignment with human preferences, indicating limitations in using LLMs for automatic annotation.
	Consumer Engagement With AI-Powered Search Engines and Implications for the Future of Search Advertising Gabriel Garlough-Shah, Jong Inn Park, Shirley Anugrah Hayati, Dongyeop Kang, Jisu Huh Advertising Division of AEJMC, 2024 Explored consumers’ choice, motivations, and use behavior between AI-powered search engines (AIPSEs) and traditional search engines (TSEs), highlighting differences in motivational use and behavior on each by deploying a Chrome Extension to users.
	SelectLLM: Can LLMs Select Important Instructions to Annotate? Ritik Sachin Parkar*, Jaehyung Kim*, Jong Inn Park, Dongyeop Kang arXiv, 2024  arXiv /  code / Developed SelectLLM, a framework utilizing coreset-based clustering and large language models to enhance the selection of unlabeled instructions for improved instruction tuning performance.
	Under the surface: Tracking the artifactuality of llm-generated data D. Das*, K.D. Langis*, A. Martin*, J. Kim*, M. Lee*, Z.M. Kim*, S. Hayati, R. Owan, B. Hu, R. Parkar, R. Koo, J.I. Park, A. Tyagi, L. Ferland, S. Roy, V. Liu, D. Kang arXiv, 2024  arXiv /  project page /  code /  data / Explored the expanding role of large language models (LLMs) in generating artificial data, analyzing various types of LLM-generated text and their implications, revealing significant disparities compared to human data, especially in complex tasks, and emphasizing the need for ethical practices in data creation and addressing biases.