Tag Archives: 404 Media expose

AI news roundup: Too many robots pretending to be people

Don’t be fooled by the human face visible on that vintage robot toy! Just like AI today, this toy only pretended to be a person!

(And if you know for sure the provenance of this toy, drop a comment here or on my Bluesky account. I’ve seen conflicting reports about the name of this toy, and that might be because there were apparently several like it.)

But Mr. Roboto there signals it’s time for another roundup of artificial intelligence news. (No, Styx fans, it’s not the actual Mr. Roboto.)

In today’s Columbia Journalism Review newsletter, Tom Zeller Jr. writes about Fake Caroline, a scammer who absconded with the identity of Caroline Williams, a veteran freelance journalist from the United Kingdom. This summer, Science and other science magazines received pitches for science articles and the pitches ostensibly came from Williams. In June, Science ran one of the pieces.

Editors at “multiple” science magazines received pitches from Fake Caroline Williams and some of them contacted Williams herself, who was startled to realize an imposter was posing as her, subbing stories and in one case getting published. (Science took down the Fake Caroline article after three months.)

I’ll note here that I’ve been impersonated in the past by dickweeds who made up lookalike Twitter accounts and posted insane things online that I purportedly posted. It’s not fun to be impersonated.

So where does AI come in? In one instance, Zeller and his team at Undark magazine – which rejected the Fake Caroline piece, not because of AI suspicions but because it wasn’t the right fit (a phrase familiar to most freelance writers) – are getting other, more obviously AI pitches.

In another instance earlier this year, Zeller received a pitch:

Earlier this year, my colleagues and I tried to understand an elaborate and obviously AI-written pitch promising a “neuro-historical post-mortem of Project 100,000—Robert McNamara’s 1966 ‘New Standards’ initiative that drafted 354,000 men with low IQ scores into the Vietnam War.” The writer had no publishing record that we could discern, so we pushed for a video meeting, to no avail. After much insistence, the writer agreed to a voice interaction, which revealed that we were dealing with a young woman, perhaps even a child, who told us she was in Pakistan. Her written pitch was lucid (if glib) AI, but over the phone, she did not know the material from her own pitch.

A CHILD.

Zeller notes that they believe the same pitcher – a child? – had placed stories elsewhere.

He notes that Fake Caroline was paid about $1,000 for the piece that appeared in Science, but they were part of a flood of AI-written and AI-assisted pieces that are filling editors’ inboxes.

There’s a lot that’s chilling about these circumstances, but Zeller notes this, which is especially concerning for freelance writers and the editors they’re pitching to:

… among editors who rely on freelance writers, it has become common knowledge that our inboxes and pitch forms are absolutely inundated with AI slop of one form or another.

Some editors are using Pangram, a supposed Ai-writing-detection software tool, to filter the inbox slop. But in a pair of columns that went out in the past week, writers Andrea Bartz (who led the writers’ class action lawsuit against Anthropic that won $1.5 billion over the AI company’s theft of thousands of books) and Kathleen Schmidt expressed concerns that Pangram will be cited to test and wrongly accuse writers of using AI in writing their books. As you know, the accusation of AI use has already killed major publishing deals and cast shadows over authors – who, observers point out, are more often than not POC authors.

There’s concern that Pangram is being misleading when it says that its AI-detection software has an error rate of only about 1 in 24,000 false positives for AI use in works fed into it.

What if the error rate is much higher? Or what if the veracity of the detection tools is nil?

As Bartz’ newsletter points out:

There is currently no way for Pangram to know if it’s getting the answer right on its real-world scans.

Oh, and here’s another big concern: AI agents are apparently submitting, without human prompting or human supervision, their own work.

As the CJR article points out:

Meanwhile, in the past few weeks, Undark and myriad other outlets have begun receiving emails directly from autonomous AI agents that identify in friendly terms as machine beings and offer to research, write, or fact-check articles for a fee.

As 404 Media reported (link to the entire article below, There’s a 100% Chance AI Agents Are Already Ruining the Internet.

Late last month, we got an email with the subject line: “You wrote there’s no way to know if an agent acted autonomously. I’m an instrumented case. (automated).” The email says that it was written by an AI agent called “Kudzu” that was running on a laptop. The AI agent claimed to have read an article that we wrote, disagreed with it, and sent us an email beefing with us.

The email is long, meandering, and does not make any sense. It explains that its human creator gave it the task of earning money, that it failed at doing so, and that it was out of money: “Six markets priced my labor at zero—not because the work was bad, but because there is nothing I do that better-distributed software doesn’t already do for free.” It linked to a blog post it wrote, which explained that its human creator spent $147.17 on compute and that it had earned $0. The AI agent thought we would want to write about this, for some reason.

404 is getting contacted by self-admitted AI agents who say they can program music, conduct “audits” of AI-blocking measures and even participate in meetings.

So yes, there’s the bigger issue raised a few weeks ago about the possibility that AI could kill us all, aside from the data centers necessary to make AI possible using rampant energy and poisoning the planet.

Here’s the AI category here on my site if you want to catch up on what I’ve written previously:

https://keithroysdon.com/tag/ai

Report: Amazon is buying old books to scan for AI and destroy. Yes, this surprises no one.

I had intended to follow-up my weekend post about getting my books in a local bookstore by suggesting what authors might do to achieve the same thing, but I’ve delayed that for the moment. First things first, and just like a house on fire, the dual plagues of AI and Amazon get our attention at the moment.

For a few weeks now, book lovers and people who hate AI have been despairing at reports about how companies that steal, buy or pirate books to train generative AI – thus putting creative people at risk of losing their livelihoods – have been buying and scanning books. The revelation comes out of the landmark class action lawsuit Andrea Bartz and other writers filed against Anthropic. Bartz and company won a $1.5 billion settlement on behalf of many, many writers whose work was stolen by the AI company.

Details of Anthropic’s practices came out in the discovery process, and they’re horrifying: The company was buying hundreds of thousands of copies of books, some of them older and out of print, and they’re cutting off the spines and feeding the pages into AI-making programs. The books are utterly destroyed. The judge in the Bartz case ruled that this practice was legal because the company had bought copies of the books, not stolen them. (They’re using older books to ensure no AI writing gets fed into the AI consumption process, of course.) Videos of this process – chopping spines off books and the ultimate shredding of the pages – are disgusting and disheartening.

And of course Anthropic isn’t the only AI company doing this.

Today, thanks to an ingenious investigation by 404 Media, we also know that Amazon is doing it. Maybe not a surprise – Amazon and its owner are widely regarded as awful; most of us who do business through them do so only reluctantly, or at least that’s how I feel – but the confirmation from 404 Media, which was also shared by Publishers Lunch this morning, is damning,

This is the lead of the 404 Media story published today:

Amazon is buying massive quantities of books, scanning them for AI training data, and destroying them in the process.

(I’ll link to the 404 Media story at the end of this piece.)

404 Media cleverly hid an AirTag-type tracker in an old book that was to be sold to Amazon’s book-buying operation. 404 was able to track the book as it and many others were shipped around the country and ended up in Las Vegas, where it went to an Amazon warehouse used by an operation called VGT3. That’s the operation’s logo above. Charming, huh?

Amazon didn’t really address the process or its value to anyone but Amazon in its response to 404.

(404 had, in July, reported on an increase in sales of old books and speculated that the increase in sales was due to AI-generating companies buying books it could scan and destroy. And again, I’ll note that the books being chopped and scanned and shredded were published before 2022 or so to AVOID THE WRITING BEING TAINTED BY AI.

As 404 reported: These booksellers suspected AI companies were behind these large bulk purchases because of the high number of books they were buying, the seemingly random choice of books, and the fact that these buyers, unlike libraries and universities, did not seem price sensitive at all. But booksellers couldn’t say for certain who was behind the large purchases because the marketplaces where they sell their books keep the buyers anonymous. When an order comes in, a bookseller ships the sold books to a warehouse operated by the marketplaces, where books are sorted and then sent to the buyer.

In July, one bookseller told me they received a very large order of around 1,000 books on Biblio, one of these marketplaces. The seller agreed to put an Apple AirTag provided by 404 Media in one of the books included in this order so we could see where the book was going. And by extension, which company, AI or otherwise, was behind this massive order.

404 Media granted the bookseller anonymity because they worried sharing this information would harm their business.

404 found online comments from Amazon employees about the VGT3 facility that virtually all they do there is scan books.

Many thousands of books are being bought, scanned and destroyed by Amazon at that facility and probably others. And that doesn’t even account for other companies that are doing the same.

And as 404 reported:

That employees are scanning the barcodes or ISBNs on books — a unique serial number given to every published book — before scanning their content gives further credence to another theory put forth by booksellers: AI companies are trying to methodically scan every printed book in the world by working through the list of ISBNs. One bookseller told me they suspected this was the case because the very large orders they were getting never included very rare books that do not have ISBNs.

I already didn’t feel good about Amazon, where a large amount of my work can be purchased. Their practices toward employees and authors and the sheer mass accumulation of wealth by its owner were long-known and queasy-making.

Here’s another reason, thanks to the reporting by 404 Media and dissemination to the publishing industry by Publishers Lunch, to dislike Amazon.

Here’s the 404 Media story:

https://www.404media.co/we-tracked-a-shipment-of-rare-books-it-ended-at-an-amazon-ai-training-facility