Ghostbuster: Detecting Text Ghostwritten by Large Language Models

Key Points:

  • Ghostbuster is a state-of-the-art method for detecting AI-generated text.
  • It works by finding the probability of generating each word in a document under several weaker language models.
  • Ghostbuster doesn’t need to know what model was used to generate a document, making it useful for detecting text potentially generated by unknown or black-box models.
  • When trained and tested on the same domain, Ghostbuster achieved 99.0 F1, outperforming other models.
  • Ghostbuster is robust to different prompt variants and performs well on text generated by Claude.


In a world where AI-generated text is causing a stir, Ghostbuster comes to the rescue like a superhero with its remarkable ability to detect AI-generated text. Large language models like ChatGPT may write impressively well, but they’ve become a problem, with students using them to ghostwrite assignments. Schools have even had to ban ChatGPT to preserve the sanctity of education. Talk about AI taking the easy way out!


But fear not, teachers and consumers! Ghostbuster is here to save the day. Existing tools often struggle with detecting AI-generated text that differs from what they were trained on. They can’t handle the complex features and end up misclassifying real human writing as AI-generated. Talk about an identity crisis!


Ghostbuster works its magic by calculating the probability of generating each word in a document under weaker language models. It then combines these probabilities to create a final classifier that can sniff out the AI-generated text. It doesn’t even need to know which model was used or the specific probability of generating the document. How mysterious!


When put to the test, Ghostbuster outperformed other models with an impressive 99.0 F1 score. It’s like winning a ghost-catching contest while everyone else is still fumbling with their proton packs. Ghostbuster is robust, versatile, and can even handle different prompt variants and text generated by Claude. It’s a true hero in the fight against AI-generated text.


So, if you find yourself suspicious of AI-generated text, don’t panic! Put Ghostbuster to the test and let it detect the phantoms of AI lurking in the shadows. And remember, with great AI comes great responsibility. Let’s use Ghostbuster wisely and avoid perpetuating algorithmic harms.



Prompt Engineering Guides



©2024 The Horizon