Evaluating Creative Short Story Generation in Humans and Large Language Models
arXiv preprint arXiv:2411.02316, 2024•arxiv.org
Storytelling is a fundamental aspect of human communication, relying heavily on creativity to
produce narratives that are novel, appropriate, and surprising. While large language models
(LLMs) have recently demonstrated the ability to generate high-quality stories, their creative
capabilities remain underexplored. Previous research has either focused on creativity tests
requiring short responses or primarily compared model performance in story generation to
that of professional writers. However, the question of whether LLMs exhibit creativity in …
produce narratives that are novel, appropriate, and surprising. While large language models
(LLMs) have recently demonstrated the ability to generate high-quality stories, their creative
capabilities remain underexplored. Previous research has either focused on creativity tests
requiring short responses or primarily compared model performance in story generation to
that of professional writers. However, the question of whether LLMs exhibit creativity in …
Storytelling is a fundamental aspect of human communication, relying heavily on creativity to produce narratives that are novel, appropriate, and surprising. While large language models (LLMs) have recently demonstrated the ability to generate high-quality stories, their creative capabilities remain underexplored. Previous research has either focused on creativity tests requiring short responses or primarily compared model performance in story generation to that of professional writers. However, the question of whether LLMs exhibit creativity in writing short stories on par with the average human remains unanswered. In this work, we conduct a systematic analysis of creativity in short story generation across LLMs and everyday people. Using a five-sentence creative story task, commonly employed in psychology to assess human creativity, we automatically evaluate model- and human-generated stories across several dimensions of creativity, including novelty, surprise, and diversity. Our findings reveal that while LLMs can generate stylistically complex stories, they tend to fall short in terms of creativity when compared to average human writers.
arxiv.org
Showing the best result for this search. See all results