As AI-generated text continues to evolve, distinguishing it from human-authored content has become increasingly difficult. This study examined whether non-expert readers could reliably differentiate between AI-generated poems and those written by well-known human poets. We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ2(1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ2(2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
Wow, that’s a horrendously bleak and depressing take. Of course you can’t put a price tag on that (why would you want to?), but you’re not seriously suggesting that human love has no emotional value, right?
A love letter from your partner, or the diary of a passed relative, or your child’s drawings? All of these things might be objectively worse than something a machine could produce. But would you feel the same when you received a love letter that’s just been printed off of ChatGPT? Humans are more than profit-producing machines, despite what capitalists want you to believe. And there is value in human interaction.