✔️ Transform your traffic in just 28 days. Click Here Now.
Thinking about using AI content for your website and SEO? The results of this case study will change your mind.
I am going to show you precisely how easy it is to detect content generated by several different AI’s using public tools.
Don’t forget that:
So I wanted to see how difficult it was to detect AI generated content. The process was very simple:
And just to spice things up a little bit, I also decided to include some of CNET’s Ai generated content (full case study) and some human spun content that I originally published in 2014.
The results weren’t what I was expecting!
What Will I Learn?
In the graphic below you can see exactly what happened from testing each of the top AI content generators against several detection methods (more details on each below).
You can click on any symbol in the table below to see the screenshot of the results for each individual test.
| OpenAi | GPTZero | Writer | OpenAI Detector | ContentAtScale | Originality | Copyscape | QueText | |
| CNET Ai Content | ||||||||
| My Post | ||||||||
| Frase | ||||||||
| Rytr | ||||||||
| Jasper | ||||||||
| Kafkai | ||||||||
| ArticleForge | ||||||||
| ChatGPT | ||||||||
| GPT2 | ||||||||
| Spun Content |
Straight off the bat, you can see that-
Failed most of the AI detector tests by a long way.
ChatGPT (often hailed as the “most advanced AI content generator”) performed the worst out of all the AI tools:
You can also see that the AI content published on CNET’s website was detected as 100% AI. Take a look at our full CNET Ai content detection case study to learn more.
But two AI content generators stood out from the rest…
They passed most of the AI content detection tools even though we used the quickest methods to generate the test content.
But here’s the huge problem…
Frase and Jasper were both cited for plagiarism by Quetext.
This essentially means that the AI content they produced isn’t unique.
In fact, none of the AI writing tools could pass the plagiarism test from Quetext at all. So if Quetext detects that the content is not unique, is likely that Google can as well.
Interestingly the only 2 pieces of content that passed every test was my own content along with some human written content that was spun on the paragraph, sentence and word level by a human.
And that makes sense because they were both written by actual humans.
I tested 8 of the most popular AI content creation tools available right now.
Below is a quick breakdown of each of the AI content creation tools we tested:
Jasper is the market leader in the AI content creation market.
They have a much more advanced AI writing system than any other tool I tested.
The legendary Jasper “boss mode” is designed to produce long-form content such as blog posts and articles.
It easily produced the best quality content out of all the other AI content tools I tested in the case study.
Jasper also crawls more of the web than most other AI content tools.
That means it generally has more content to use as context, which significantly increases the quality of its output.
Frase is a powerful tool that started out by helping writers create website content and optimise their content perfectly to rank higher in Google.
It has evolved significantly over the last few years, developing its own AI writer to produce content faster. They recently signed a deal with the Copyrytr group, which has a collection of AI tools including:
This has helped them develop their own AI content generation models faster.
Check out my complete Frase review to learn more about it.
Rytr is primarily an AI writing assistant that is best at coming up with different angles for copywriters, marketers and entrepreneurs.
Rytr is one of the most affordable AI writing tools with a cost-effective unlimited content writing plan. This has made it extremely popular over the last couple of years.
Kafkai is a decent AI content generator that specialises in producing SEO-optimised content for blogs and articles.
Kafkai uses natural language processing (NLP) and advanced AI models to produce long-form content that reads well.
ArticleForge is also primarily designed for long-form content.
It can produce SEO optimised 1500 word articles in less than 60 seconds – with the click of a button.
ArticleForge is popular because the user interface makes it easy to use. You type in your target keyword and ArticleForge will produce the article.
All sounds good, right?
The problem is the quality. The AI test results show that the content…
There is a lot of hype about ChatGPT at the moment.
The premise is simple – Ask ChatGPT to write something (or ask a question) and get back a well-worded piece of content in a number of seconds.
ChatGPT is currently free to use which is why it has gone viral all over the internet but it failed all of our detection testing.
GPT2 is another free AI content generator run by Watt AI.
It’s simple and fun to use but takes a lot of time if you want to produce anything decent.
GPT2 is best known for creating entertaining and amusing content from a story-driven perspective.
CNET launched a silent experiment by publishing finanical advice written by AI which has caused an awful lot of controversy.
However in this case study we were able to detect AI content in 87% of their AI created content:
While CNET haven’t released any specific details on what tools or processes they are using to create AI content, it seems they need a lot of improvement from a detection stand point.
In 2014 I published a tutorial that showed you how to fool duplicate content detection with human spun content.
The content in that tutorial was spun at the paragraph, sentence and word level by a human hand. Not only that but I actually shared the full article on my blog as an example, so I decided to take an output of that as a wild card test against AI content.
Learn how to disable Google’s AI overviews in the search results.
So what were the AI content detection tools we used in this case study?
Here is a quick breakdown of what they do and what makes them unique.
Originality was by far the best AI contect detection tool that we tested both in this case study and our CNET AI content case study.
That means their AI detector is built to catch out AI-written content and ensure it is completely plagiarism free.
Originality is good at what it does and is continuously being developed to stay up to date with modern AI writers.
Not a single piece of AI generated content was able to pass the Quetext test.
It uses advanced DeepSearch technology, which understands the context of the written content. It then checks it against billions of sources in its content index.
And although its not an AI content detection tool per say, it is the most advanced plagarism detection tool on the market. AI learns from the content you feed it, so duplication is expected at some stage.
The OpenAI team (the team that made ChatGPT) released a free tool on the 31st January 2022 to helo detect AI content:
This is the first official AI content detection tool as it is made by the OpenAI team in an effort to combat AI content being used where it shouldn’t be, like a thesis for example.
GPTZero is an advanced AI detector that was inspired to stop ChatGPT.
It uses advanced variables to track AI models and detects their digital fingerprints.
They are currently developing their technology for educators to stop students from taking advantage of AI for college and school assessments.
Writer is one of the most popular AI content detection tools around.
But Writer isn’t just an AI detector. It’s also an AI content writer.
The detector is designed to help people use AI-written content without being detected by search engines like Google.
OpenAI detector is a free-to-use public AI content detector.
It was initially developed for GPT2 AI content.
The team behind it have been working hard on it and have developed a simple-to-use tool that does a decent job detecting at now detecting GPT3.
Content at Scale is similar to Writer because the tool was designed to produce AI-written long-form content fast.
They have now added an effective AI detector to check your AI-written content before publishing it online. The detector is very good at detecting low-quality AI content.
Copyscape is about as simple as a plagiarism checker gets.
But just because it’s simple doesn’t mean it’s not effective. Copyscape has a full suite of plagiarism tools ready to detect copied content.
Can Google detect content written by AI content creators?
Short answer – Yes because if we can do it with publically available tools, so can Google who have invested more in AI than any other tech company.
Not to mention the fact Google published a paper in 2020 showing how they can use text generation models to detect low quality content.
A simillar test by Jon Gillham, comprehensively showed that AI content detectors could easily detect AI-written content as did Surfer.
It’s also likely that email spam filters (the biggest is opearated by Google) and social media networks will start to integrate AI content checkers in the future. T
his could mean that your emails go directly to spam or posts on popular social media sites get limited reach.
So are AI based content generators valuable at all?
Yes, if you use them properly.
AI generators are great for:
…for human writers.
From my testing, it’s clear to me that the best AI tools can come up with some amazing ideas and angles for content creation.
But when it comes to actual content writing for a website – stay away!
If Google decides to penalise sites using AI content in the future, it could completely destroy an entire website.
Don’t get caught up in the hype.
Protect your site and avoid using AI writers for website content or anything to do with SEO. We won’t be using it to create social updates or emails either!
Join My Facebook Group
I appreciate your work Mathew, I always have. I too have drunk the hype flavored cool-aid my damn self and I can see how people looking for that elusive ez-button would/could get sloppy using AI content for SEO. BUT… When done correctly, it can come up with some AWESOME (legit) content. As an SEO for 15 years+ that also owns a local retail shop; I figured out how to get chat-GPT to spit out 100% human written, plagiarized free content for each product we buy and sell! It’s not a silver bullet (damn close though)! Meaning, just create and then post. It still takes that human touch (and time) to format everything, tweak and rearrange some of the content etc. What is the “correct way” to do it? It’s all in the prompts. Here’s my list of 11 prompts I use for each product I want to write about from start to finish.- Enhanced Post Prompts (beginning to end) -1) I am a (Business Type) and want to attract search engine traffic from people looking to (Buy | Sell) (Main Keyword).Do you understand?2) can you create a list of LSI keywords associated with “(Main Keyword)”Do you understand?3) I want to write a long blog post that is SEO optimized using these LSI keywords throughout the content. Do you understand?4) this blog post is to be written in second person, from the perspective of (Business Name) making offers to (Buy | Sell) (Keyword) using the LSI keywords throughout the content in a natural way.Do you understand?5) My business name is ( ________ ), we have been a (business Type) in (City, State) for (amount of Time). We have five-star ratings with hundreds of satisfied customers.Do you understand?6) Create a blog outline with the above focus keyword and LSI keywords in mind.Do you understand?7) For the content, you will include the above information and LSI keywords written throughout the content with SEO optimized H2 headings for each topic point you came up with.Do you understand?8) Going forward, all of the above is to be used for your reference in writing a comprehensive, SEO optimized blog post.Do you understand?9) This content is to be written in second person, and in a conversational style of writing as it were coming from (Business Name). It will have a persuasive tone and appeal to do business with us.Do you understand?10) With the above instructions in mind, expand on the outline you provided. Do not include your instruction or reason for doing so.Do you understand?11) ContinueIT WORKS!!!Doing it this way, I have found the output to be damn near flawless each and every time! It adds that personal touch that (for me) has gotten buy AI detection with scores of 97% to 100% human every time! And… 100% unique, every time!Enjoy 😉
That is a very detailed process however does it pass ChatGPT’s own detection tool?
Hey MattThis is spot on… I have been using Jasper boss mode since it started. I now see more and more de-indexed pages from google for my client’s sites…. and I really think this new wave of chat interest will do 2 things… 1. We will see a proliferation of so-called “writing experts” pop up and tout their expertise in writing cheap content for small low-budget businesses, thinking they get a deal..only to find it penalizes their site in googles eyes and2. The massive increase in content on the web is going to be so overwhelming, that written content as we know it will not be a strong signal for search as it is now… these lame marketers will destroy that… and, as Google has already told us, they are now looking for E-E-A-T-S… specifically EXPERIENCE in the content that you are writing about… we will see a rise and sudden fall of these so-called experts pretty quickly
Indexing in general has got a lot harder but we’ll be publishing a case study about that soon!But yes I agree with your points, ultimately it’s a cat and mouse game and the cat is catching up to the mouse.
These AI detection tools can be tricked very easily, the question is what sort of AI detection does Google has?
I would assume they have some of the best resources on the planet for that given their investments
There is no such thing as duplicated content between different domains. Duplicate content exist only as internal thing around your domain. Period. And thats also a google statement.
I assure you when several high DR sites clone your entire site via proxy meaning they publish updates/changes just as quickly as you do – your rankings will suffer. We have had a couple of clients have this attack used on them, one in crypto and one in gaming.
Removing one or two punctuation or adding punctuation at a wrong place, most of the times makes 100% fake content to 100% real content. So getting around these publicly available AI detector is easy, not sure about Google though.
Yes the public tools are built with very limited budgets (often zero) and often by a one man band or a very small team. Google don’t have such bottlenecks.
Hi Matthew, I love this article! It was incredibly informative and comprehensive.You did a fantastic job of breaking down the different types of content AI can create and detail how Google can detect AI-generated content.It is important to note that while AI-generated content can be detected and penalized by Google, there are still some legitimate uses.For instance, businesses may use AI-created copy for internal documents or reports where accuracy is not paramount.Or, when creating website copy, businesses may appreciate the ability to quickly generate a large amount of content using an AI tool.Overall I believe that if used responsibly and with caution, AI-generated content can add value to specific processes. However, regarding SEO campaigns, AI-generated content should be avoided.Google has clarified that they disapprove of AI-generated content, which could lead to penalty action.Businesses must understand the implications of using AI-generated content and ensure they do not violate any guidelines. I appreciate the thoughtful insights you provided in this article on how businesses can use AI responsibly.This stimulates an exciting conversation about how we use technology in our marketing efforts, and I think we must discuss these topics openly.Thank you for such a valuable piece!
Yes there is lots of legitimate uses especially for producing anything that is not front facing content or part of email marketing.
AI gives you the bricks, you build the house.
Exactly!
Hi Matt,So, today I started my own case study to test onsite BASIC optimisation of a website (a few of them) only using the ChatGPT to do so. Here is my list:Keywords research – I’ve fed the AI the most information I can to spit out the best longtail keywords for a particular niche, then I am cross referencing that with data from other keyword tools to see the actual numbers of searches. I can, I think anyway, explain in detail to the AI, how to account for the search numbers locally, then asking it for the rankings…still trying to figure this one out.I will then input some text based on the niche, basically just a 300 to 600 word Homepage for a site I am working on, and ask it for the best headings (h1, h2) for the output and finally, Ill ask for the meta data. I am really looking forward to seeing how this all plays out.My last project, for the AI; I want to see if it can really build me a website from scratch. I need to fully understand how to input this all into the AI in order to teach it….if that’s even the right word.Thanks so much for this case study, I’ve been obsessed with it since it’s publication.
That’s a very interesting project are you doing it on a new domain or aged?
Thanks for the in-depth analysis. My main fear around AI content comes down to this… if I’m outsourcing content and I don’t have funds for investing in AI-detection tools, what do you suggest to make sure content writers are producing unique, accurate content that isn’t AI-generated?
Unfortunately managing writers and reviewing their work is a continous process that never ends. You should be using QueText at a minimum to check every piece of submitted work.
You explained AI content and what problems to face in the future but what about the spun content? Is it safe or not?
Human spun content when done properly is unlikely to be detectable at any point, however creating spun content to that calibre is labour intensive.
I am caught up in the AI ‘movement’–Ha!However, I was brought back down to earn by your surpurb review of all the AI players.Thank you.I have yet to commit to ChatGPT–or any tool, but your comparisons have given me direction!Incidentally, in one of the reviews you mentioned ‘sweet’ instead of suite–was that intentened?Thanks again for a very cogent discussion…you’ve opened my eyes to possibilities–and have given me a track to run on–instead of the wide open spaces!-Steve Lanning
Sweet, thanks for the headsup I fixed it!
Paraphrase bro, like everyone knows that. How else you make it unique…
Can you explain more please?
This is super frustrating. I am literally in Originality manually typing text and scanning it and it comes up AI. IT IS NOT AI — IT’S ME. As a marketing professional, I want to assure my clients that the copy I provide is human-generated since it is. However, every single “ai copy checker” I’ve used has identified my written copy as AI, even when I switch to a more casual voice. The writer.com and Originality are actually the worst, however, the ai detector at copyleaks.com seems to be much more sensitive to human content.Overall, this is terrible. This is already affecting my income as I have to waste time rewriting human content that’s being detected as ai. I really hope this is resolved soon. 🙁
Would you mind replying with a 300 word sample of your writing? Have you been in touch with either of the detection tools to report the false positive?
It was written by my own hand?
Hi Mathew, Thank you for the interesting AI test. You can run these Ai App content through another Ai App which will give them a more human touch. I don’t want to tell you, as it would be inhumane.Kind regardsBryn
You mean take the AI output and run it through Quillbot?
This is awesome Matthew! I’ve shared our own results with Originality.ai with some other SEOs and inevitably, many of them choose not to see it as a warning of the perils of AI content, but a challenge to game the system. One popular method to avoid AI detectors spotting the content as AI-generated is to use AI rewriting/spinning tools like Quillbot. I haven’t done the tests yet, but I feel like this will still be readily detected by QueText, and more importantly Google, since the content and context are still the same.
I would like to see the results of the test if you did! I showed you mine… 😛
Hey Matthew, great article. Have you done any single variable tests to rank an AI Generated article? Despite being flagged as AI by the detectors, does the article or post get ranked against competition?
Take a look at https://www.kevin-indig.com/how-well-does-ai-content-perform-in-seo/
AI definitely helpful, but we can’t use it in raw basis. I am still using with few ways around it. However it’s still involved 20 % of human work.
I think its good for planning, researching and creating content specifications for humans to execute.
It would be great to know if the content was used verbatim from these tools.If you were to edit and add your own original elements, how would the results look?
The content was copy and pasted verbatim with no editing or tweaks. I have now added links to the source content for you to download.
Great post. I’m glad someone addressed it in detail. What do you think about sales pages and landing pages? Especially when you’re using paid ads to drive traffic to them.
If organic traffic isn’t an issue – I don’t see why you couldn’t use it for paid campaigns. Perhaps Google could update their quality score in Adwords to reflect AI content on the landing page https://searchengineland.com/google-study-generative-models-redictors-page-quality-392460 but it would be easy to fix if they did
Thanks for sharing and you provide best information for SEO. result is not acceptebel but and i am very sad for result, but it’s help me to save my website. i never use AI-generated content.
Yes I wasn’t happy with the result either! I thought I could finally retire and have AI do all of these case studies for me =D
Excellent, Thanks for helping, I was not using AI Content, but other team and SEO’s buddies are using it and keep suggesting me to use it. Still, I was not using, the tool to create content from AI tools, so it will open other SEO’s Eyes if the want to take long-term benefits from SEO and Google.
And I’m sure they are probably seeing that content rank at the moment!
Great post! Thank you for the case study. I was little concerned about AI content. But I’m glad that I have taken great decision to not to put AI content on my website. I agree with you that Google can easily detect that. Chatgpt is useful for generating content ideas and outlines as you said. Thank you for the solution.
No problem glad to help!
Are you now saying AI writing tools can’t be used as an assistant.
What I’m saying is what is published above in the article. I don’t see any reference to saying “AI writing tools can’t be used as an assistant.”
Thanks for a great review! Do you think using AI-generated meta tags for improving CTR is ok with search engines (assuming this is short-form content and it’s harder to know if it was AI)?
That depends on how aggresive Google choose to be in the future https://searchengineland.com/google-study-generative-models-redictors-page-quality-392460
Seeing spun content here with all green checks. And ChatGPT with all red crosses made me cringe so hard. My eyes are tearing up right now And to even lower the level of ignorance and competency of your knowledge in any of this, that makes me cringe even more, is seeing Jasper and all the other crap in here praised more than ChatGPT which is GPT3.5’ish, while Jasper is using GPT3 api to provide you with those articles. You litterly said my Suzuki is better than your Ferrari hahahah, unsubscribe button has never been clicked so fast. Matt, you lost it brother, accept your defeat.
All I did was take the output of tools and use it as an input for other tools and screenshotted the results.I’m sorry that made you emotional but the data is the data.If you don’t understand why human spun content doesn’t get detected as AI content, it’s because human spun content is in fact, written by a human. And yes tools like Jasper are BASED on GPT3, however they have trained the model with their own action specific data sets which greatly impacts the output.You didn’t unsubscribe…?
Excellent article, Matthew and team. As an SEO, I have dabbled over the years with some of these tools. Most of them, I didn’t use for any length of time as I wasn’t happy with the results. Jasper is one that did stick for longer but even before reading this article, I felt it wasn’t the right route. I am referring to the AI route, in general.As an author and media producer, using non-human content doesn’t sit easily with me. One thing that we did use consistently was voice to text, originally with Dragon Naturally Speaking. I very much liked writing like this and then going back in and refining the work. It is quite a spontaneous way to write and easier than being on a the keyboard all the time.As for spun content, I hate this with a passion so yes, I was surprised initially to read the results. But I guess it does make sense. As a human is doing the spinning.Bring back good old fashioned writing for a purpose with creativity and proper research, that’s what I say!! 😉 Google does recognise it, I have had many experiences of this.
I think we agree with our outlook! You just can’t beat engaging enterating content backed with research and data.
Very interesting research Matt, and timely too! Without AI detection I had visions of the web turning into a giant spam bot.
Arguably, it’s already a giant spam bot =D
I believe the fundamental problem is that people are lazy. Its a lot of work putting a decent, well-written and informative web article together. But yes, I think there is a lot of hype at the moment and in the future there will be a lot of “publishers” that regret cutting corners.
Yes we saw exactly the same thing in the golden era of auto generating spun content about 10 years ago except spun content was only applicable in the SEO arena whereas ChatGPT is applicable in every industry.
You didn’t clarify what sort of articles you wrote, and how you instructed the AI to write those articles. It would have been useful to actually view some of the articles. If you just give the AI one word, or perhaps one sentence then the final work is more likely to be ‘risky’ than if you give paragprah level instructions. My 1500 word AI article gained 98% Human from Writer… and the only plagarism QueText found was the (my) URL I copied/pasted the text from (into their tool).I would strongly imagine that Google will leave margins for error, in penalising AI work (because if something is only 2% AI written – do you really penalise the 98% human written)?
I have updated the article with links to all the source content for your reference and a link to https://searchengineland.com/google-study-generative-models-redictors-page-quality-392460
This came at exactly the right moment. In the UK, the Good Morning TV programme was talking about AI in education and said that the UK may have to go back to just exams as students are getting A+ grades by using it for GCSE and A Level course work. My son had to write a short essay this morning for college so I suggested he used AI to see what happened. He did and when he read it out was told it was easily the best in the class. He then told them how he had done it. I am just starting a new blog and have thought that I would need to use Ai just to keep up with everyone else but this article has changed my view.So as a thought, could I use AI to give me the bones (plus a little flesh) of a post and then Spin it as I, like you, did a number of years ago? Would this make it unique enough to get past AI and plagiarism detectors? Would this be a way forward to at least speed up the process?I would be really interested in what you thought about this.
Well since publishing OpenAI have released their own detection tool https://platform.openai.com/ai-text-classifier which we have now updated the case study with.It’s likely any students that were using Open AI to produce their work secretly are under pressure right now.I think using AI to create outlines and find common questions is the best use case. Any actual writing for whatever reason should be done by a human hand.
Very comprehensive, and thanks for that post. I have tested some of the data I generated with chatGpt and it’s 99.9% fake. I had misgivings about using it and I’m glad I read this post.Will use it for ideas but not for full text. Thanks again.
No problem Dave!
Thanks Matthew, this is something that definitely needed to be said given the amount of hype and all the new tools hitting our inboxes of late (MarkeingBlocks 2.0 anyone?) I was curious about this, “Content at Scale is similar to Writer because the tool was designed to produce AI-written long-form content fast.” and why you didn’t test their content generated against the detection tools? Might be interesting to see how unique their own content would be.Also curious if a combination of AI content + a top shelf spinner would work, a first draft and editor as it were?Thanks again.
I will add them to the testing!Yes perhaps a process that embraced an AI written outline, that was then written by a human writer would work. Spinning of course is optional depending on how and where you want to use that content.
Ai content is against Google guidelines but what about human-edited Ai content? Google is against automatically generated content which I guess copying and pasting Ai generated content directly on your website without editing or fact check.What about this tweet by Danny Sullivan – https://twitter.com/dannysullivan/status/1589681355868504064?s=20&t=ZrOVzgU5B_ciSBalduNSag
There are some nice uses of AI generated content out there, for example instead of reading 100 comments/review from users – AI can write a quick summary from that.Its unlikely the wider SEO community will use it that way though.Also please take a look at https://searchengineland.com/google-study-generative-models-redictors-page-quality-392460
Very interesting read. For now I am convinced by your arguments and tests. Let’s see in the near future if AI detection tools will still work. OpenAI of course thinks they will not be able to very soon. Will be exciting 😉
You should run some of your own content through the tools to see what they find
Thank God you posted this. Hopefully people will actually listen to you! It’s just that, all hype!
I think its a pivotal change in history that will challenge many industries, but its not ripe for use in SEO.
Thank you.Excellent article. So many people are always looking for the easy way. Well I always say there is no easy way. Go through the process and create good quality content that grabs the end-users attention and give them some benefit and answers their question(s)That’s the secret,Jeff Goldsteinlmtw.us
That is exactly right Jeff!
Great study. Makes me feel relieved in a way but only time will tell. 🙂
SEO is always a cat and mouse game but we are repeatng histroy except before it was duplicate content, then auto spun content, then auto scraped and spun content and now it’s AI content.
Google have subsequently issued a clarification that they are not against AI content, just poor content written purely for search engineshttps://mobile.twitter.com/dannysullivan/status/1589681355868504064?t=QRmZkIj8SwOl8TkGYCAnOA&s=19
I’m sure the SEO community won’t use AI to generate poor content written purely for search engines, nor will writers try to pass off AI content as their own.
I agree with Matt, some human content when written specifically for search begin rankings can become and seen as AI even if its written by human.So in my view, don’t over optimize your content in eyes of SERP because it will look like its by AI, that’s what Google is trying to do here. Also, if AI can write human focused work then its good.