Wikipedia:WikiProject AI Cleanup/How the AI noticeboard works

This is rough guide to how the AI noticeboard works, outlining the rough life cycle of a case. It also contains advice for anyone that wishes to help out; there is more located at WP:AICGUIDE.

Cases (i.e. threads) usually stay on AINB for a few weeks, sometimes more. At time of writing, there are usually ~140 cases on AINB at any one time and only a handful or two of people working there, so if you've made a report please be patient while someone makes their way towards addressing it.

Filing a case

• advice on filing, briefly explain the system
• explanation of case statuses

Analysis

The goal of analysis is to determine whether someone has habitually misused LLMs, and the time period of usage (including a rough start date), in order to meet the evidential threshold for presumptive removal. Anything before 30 November 2022 (the release of ChatGPT) is extremely unlikely to be LLM-generated. Below is a rough guide to doing this in ascending order of how time-consuming the method is, formatted as a bolded list for your own amusement.

• Edit filter logs and paste checks: input their username into this group of edit filters, you can download and use User:Suffusion of Yellow/FilterDebugger so you don't have to scan each diff for what triggered the filter. This is often very useful for answering whether there has been LLM misuse and getting a very rough idea of the time period, though on its own is usually insufficient. You may want to remove the search filter and check for other edit filter hits like adding non-existent templates or citing Wikipedia. Also use the tag filters at Special:Contributions to see whether they have triggered Paste Check or LLM Paste Check.
• Blatant WP:AISIGNS: scan some creations for red-linked categories, templates, and "See also" entries; for markdown, chatbot communication, and bolded lists etc. It can be most effective to view the first revision of their creations using XTools (go to their contributions page, scroll to the bottom and click "Articles created", click on the timestamps). You can also do an EIA of the user with Gnomingstuff to see if they've tagged any of their articles in the past.
• Subtle WP:AISIGNS: scan some of their creations more closely for AI signs (unfortunately you just have to learn them). A lot of these also show up in human writing sometimes; it is the presence of several signs at the same time (i.e. in the same article/edit, across several articles/edits) that skyrockets the probability it is AI-generated. Unfortunately, the examples at WP:AISIGNS have become somewhat outdated with respect to recent models; if you're struggling you can use Pangram (see WP:PANGRAM) or GPTZero (not withstanding WP:GPTZERO) as another 'sign'/data point. There are also signs of human writing to look out for.
• Check for other indicators: check whether any of the links in the article appear to be hallucinated (i.e. use Link dispenser or user:Fermiboson/sourcecheck to find 404s and do a site search or check the Wayback Machine to see whether they previously existed/worked), check for incorrect ISBNs (can use ISBN Search) etc.
• Review creations for WP:V: i.e. for instances where the sources don't support the text, implying the existence of hallucinations. This is often the most reliable way to determine whether an article is AI-generated, but also the most time-consuming. You can usually expect an AI-generated article to have over 1⁄3 of its claims fail verification (a rough estimate based on experience). Start with sentences that have AI signs. Especially if the author has done some cleanup, starting at the bottom and working upward will often get faster results.

User conduct

WP:AICGUIDE#Engaging with editors suspected of LLM use goes into more detail with respect to advice for engaging with the 'subject' of a case. A summary: don't bite the newcomers, and don't waste time and energy bickering. It is important to remember that being reported to a high-traffic noticeboard, for an issue potentially related to competency, is practically always a terrible experience, especially for newcomers. Other editors need to always stay conscious of this and aim to mitigate it where possible.

While the noticeboard is primarily for cleanup, it can handle conduct issues as well and regularly dishes out blocks for persistent LLM misuse or lack of communication past multiple warnings (you can use {{AINBA}}, but do so responsibly).

AI-generated creations

Several means are available to delete an LLM article; it's important to choose thoughtfully among them. They are listed in approximate order of AINB usefulness. Please read the associated policy or guideline page before starting to use any of these.

WP:LLMPROD is a specialized PROD which will cause the article to be deleted in 5-7 days. It cannot be used if there have been substantial contributions by other editors. We disregard bots, sockpuppets of the same case, taggings, and edits whose only purpose was to partly delete the LLM text when determining substantial contributions. Note that once the conditions to use LLMPROD have been met, it is not necessary to show that each individual article is LLM. (This makes it possible to handle cases with thousands of creations in a feasible amount of time.) Unlike other forms of PROD, an LLMPROD tag can be restored if removed without discussion or remediation.

WP:Draftification or WP:Userfication is an alternative to LLMPROD if the editor is possibly cooperative. It is best practice to tag draftified articles as AI-generated ({{Template:AI generated}}), as otherwise any editor who discovers the draft may return it to mainspace. Since "draftification is not a backdoor to deletion" this should not be done with very old articles or ones whose author is inactive or blocked. Recommend to the author that articles drafted for LLM should be submitted to AfC, not simply returned to mainspace. The bitter truth is that most cleanup attempts fail. Do also tag drafts with {{Ai-generated}} to make sure other people unaware of why it was draftified don't move it back to mainspace. Even if it is moved back to mainspace, it will show up in CAT:AIG for someone to eventually cleanup, or if the tag is removed it'll show up in Special:AbuseFilter/1407 (which tracks tag removals).

WP:Articles for deletion can be used when an article is impossible to LLMPROD but contains so much LLM text that salvage appears impossible. However, AfD is oriented toward notability and will sometimes keep an LLM article on a notable subject. If this happens there's nothing left to do but tag or fix it. Note that you cannot apply any other form of deletion to an article that is currently at AfD or has been closed as Keep there.

WP:G15 speedy deletion can be used for extremely blatant LLM: chatbot comments to the user, multiple badly broken citations, or certain unequivocal technical signs. It can not be based on stylistic cues, no matter how blatant. In practice most LLM articles are not eligible for G15. Occasionally other speedy deletion criteria can be used, such as promotion, copyvio, or ban/block evasion.

WP:Proposed deletion seldom offers advantages over LLPROD.

When none of these methods are applicable, the three remaining choices are to fix the article (or find a volunteer to do so--WikiProjects may be able to help), reduce it to a stub, or tag it AI-generated.

Tracking pages and cleanup of ordinary edits

Making a tracker

Once a more-or-less exact start date has been established, a tracking page for cleanup (example) can be created using the "Manage Case" button at the top of the case report. The tracker-creation script offers the opportunity to filter the articles before creating the tracker. Common practice is to filter out creation and retargeting of redirects, undo and manual undo, and Twinkle edits. It is also generally helpful to filter out small edits, though unfortunately some editors tend to introduce questionable text with smaller edits than usual: it may be best to spot-check a few small edits before deciding on a cutoff. The current default when filtering by size is 50 bytes.

Limits. The tracker breaks down at roughly 1200 articles due to template expansion limits. The following steps can help. First, aggressively filter out small edits, going to 100 bytes or even higher. Second, do a pass filtering out non-creations, apply LLMPROD, and only after the LLMPRODs have been executed (so the deleted articles will no longer appear in the tracker) remake the tracker. If all else fails, make multiple tracker pages, but this will disable many useful functions of AINB-helper and should be a last resort. The tracker also breaks if a single tracked article has more than roughly 4000 edits. Source editing to remove those edits is the simplest solution.

Cases with multiple editors. If you are making a tracker for a sock-puppet case, list all of the editors in the tracker construction form, separated by "|" symbols. This will make a single combined tracker which identifies which account added each group of edits. It can only accommodate around 50 names; so far we have not needed more.

Recreating a tracker. The "Recreate tracker" button will make a new tracker, preserving each article's status, but losing any custom comments associated with it. This can be helpful when a new sockpuppet is found, or filtering needs to be changed, or the start date is pushed back. If it is critical to preserve the comments, you can instead create new trackers without publishing them, and then use "Copy text to clipboard" in the tracker script and paste that text onto the end of the case. When multiple trackers exist, any article found in more than one will have its buttons disabled and will need to be updated by source edits. This method is inferior unless preserving existing comments is critical.

Doing the cleanup

The various situations and methods for removing AI-generated content are outlined (very roughly) below, formatted into an unnecessary table for your own amusement.

Situations and methods for presumptive removal of AI-generated content on Wikipedia

Situation	Method
The presumably-LLM edits are the latest edits to the article	Revert/restore an earlier version of the article, use WP:RDP if the old version has deprecated infobox parameters (can see this using "Preview")
The presumably-LLM edits are not the latest edits to the article, but the edits by other editors after it are all minor and 'insignificant' as it were	Restore an earlier version
The presumably-LLM edits are not the latest edits to the article, and the later edits are ‘significant’ (i.e. PAG-compliant additions of text or otherwise time-consuming edits). None of the conditions outlined in the below cell apply	Option 1: manually restore an earlier version, copy-paste the later edits in if possible. Option 2: use Who Wrote That? (WWT) or User:MSK/saucership to highlight content by the LLM-using editor that is still in the article. Edit the current version (in a new tab if using WWT) and remove all content that was highlighted, while replacing it with content from an older revision that was overwritten if necessary
The presumably-LLM edits are not the latest edits to the article, and the later edits are ‘significant’. One of the following also applies: The LLM-generated addition is an important aspect of the topic Edits by others altered the LLM-generated content too much such that it can’t just be removed The version before the LLM edits is of bad quality	Usually these articles just get tagged with {{AI-generated}} because they are too time-consuming or require expertise (i.e. it’s left to local editors or future editors at AINB) To clean it up you need to preserve the content in some form, reviewing the text and the sources for WP:V and reworking the content to be PAG-compliant. Sometimes you can remove most of it and just keep the basic facts. On a rare occasion, an article's content was entirely replaced by lengthy AI-generated content a while ago, and it isn't reasonably possible to preserve later edits; here there's unfortunately not many other options but to restore the pre-LLM version in whole.

The following is what the tracker statuses roughly mean:

• Completed = cleanup has been done, AI-generated content has been removed from the article (whether by you or someone else)
• Unnecessary = no cleanup needed
• Tagged = article is tagged w {{AI-generated}}, it is tracked in Category:Articles containing suspected AI-generated texts and Special:AbuseFilter/1407 tracks removals (i.e. it is not AINB's problem anymore)
• Ongoing = cleanup is in progress, the article is LLMPRODed and will likely be deleted in 5–7 days, or it has been queued for some other form of deletion (AfD, PROD, speedy deletion)

The Bulk Edit button offers an option to mark all deleted articles as Completed. This is an efficient way to mark large batches of completed LLMPRODs, and also lets you more easily spot articles where the LLMPROD has been declined.