In the bonus discussion this week, we continue discussing AI em dashes with Sean Goedecke, software engineer for GitHub. We talk about why AI detectors are often unreliable and how they can disproportionately flag non-native English speakers. We also look at the controversial idea of "writing for AI" to ensure your ideas are represented in future machine learning models.
In the bonus discussion this week, we continue discussing AI em dashes with Sean Goedecke, software engineer for GitHub. We talk about why AI detectors are often unreliable and how they can disproportionately flag non-native English speakers. We also look at the controversial idea of "writing for AI" to ensure your ideas are represented in future machine learning models.
đź”— Share your familect recording in Speakpipe or by leaving a voicemail at 833-214-GIRL (833-214-4475)
🔗 Watch my LinkedIn Learning writing courses.
🔗 Subscribe to the newsletter.
đź”—Â Take our advertising survey.Â
🔗 Get the edited transcript.
đź”—Â Get Grammar Girl books.Â
| HOST: Mignon Fogarty
| Grammar Girl is part of the Quick and Dirty Tips podcast network.
| Theme music by Catherine Rannus.
| Grammar Girl Social Media: YouTube. TikTok. Facebook. Threads. Instagram. LinkedIn. Mastodon. Bluesky.
[Computer-generated transcript]
Mignon Fogarty: Greetings, Grammarpaloozians! We just finished the main segment where we were talking about the em dash and why AI just loves it so much with Sean Goedecke,  a software developer for GitHub from Melbourne, Australia. Now, we're going to talk about whether you should write for AI and the problems with AI detectors. Because that is an important thing that I want you all to be aware of. Sean, welcome to the bonus segment.
Sean Goedecke: Great to be here. Sounds like it's going to get a bit more controversial.
Mignon Fogarty: Yeah. For our smaller audience, at least to start with. I think that there's a lot of talk about how you can detect AI writing. Writers are calling out other writers, which I think is terrible, and saying, "This is AI writing, that is AI writing." They're looking at these hallmarks like em dashes or the word "delve." And now there are detectors, software programs that claim they can detect AI writing. I have read and seen studies that they aren't all that great at what they do, and you've looked into it even more. Can you tell us why these tools are flawed?
Sean Goedecke: Sure. The one-sentence answer about why these tools are flawed is that AI write like humans. You could stop there and you would have 75% of it. AI writes close enough to human beings that it's just straight-up not possible to be 100% definitive that something was written by an AI rather than by a human. The reason AI have the quirks they have, such as overuse of em dashes or the snappy tone, all the other stuff, is because many humans talk like that. Those habits were lifted from human writing. And that gets even more complicated when you have people who spend a lot of time talking to large language models and therefore start writing like them directly, rather than just using the same quirks.
Mignon Fogarty: A friend of mine made that comment the other day. He said, "I feel like my writing is becoming more like AI because I spend a lot of time reading it," and I was really surprised to hear him say that.
Sean Goedecke: You write like what you read. You're so influenced by the kind of books you're reading and the kind of content you're reading. It's going to happen with AI.
Mignon Fogarty: It's true, and I shouldn't be surprised, because people ask me how they can improve their writing and I tell them to read good books with good writing. So, why wouldn't it work the other way around?
Sean Goedecke: That's really good advice. But when I started looking into it, I was actually way more pessimistic about AI detectors than I am now. I thought there was no way they could possibly work. It turns out they work pretty well, they work better than I thought they would—which is to say that the best detectors, you could probably have 80% confidence. When they say that something's definitely AI, you can maybe be 80% sure that it is.
Which is pretty useful, I think. That's a useful tool. What it isn't is like a plagiarism detection tool. 80% is not enough to fail a student essay or kick someone out for academic misconduct. So, that's probably the most important point to take away from any discussion on AI detectors: they work kind of well, but they definitely do not work well enough to be taking these kind of enforcement actions on the basis of them. There's no way.
Mignon Fogarty: Right. And there are certain kinds of writing that they're more likely to flag, right? So, not only are they not accurate, but they disproportionately will pick out certain kinds of people's writing as AI.
Sean Goedecke: Yeah. The initial attempts at AI detectors that were very, let’s say, naive from an engineering perspective—like just the most simple possible solution—those were almost better at detecting people for whom English wasn't their first language than they were at detecting AI. They would light up like a Christmas tree. So yeah, it's definitely not—you know, it'll get 20% of humans, it'll get like 90% of people who sort of learn English after their native language and then 10% of English-first-language speakers. So you have to be really careful about bias when you rely on these tools as well.
Mignon Fogarty: The one that I've been hearing people talk about a lot is called Pangram. You talked about that in your blog post, and I've tested it. I did not find it to be super accurate; I could trick it in both directions. But it was okay. What surprised me is that they are trying to say that they can detect work that was edited with AI, not even just stuff that was written. How are they doing that?
Sean Goedecke: Pangram is a great example of the more sophisticated end of AI detection tooling. I've read their white paper and it sounds pretty impressive. It’s  hard to say, like exactly how good these ideas are unless you've been in the room and seen and seen the data. They have this whole process where they take human-written text and they apply a series of AI edits to it, and they train another model basically to predict the degree of editing. Part of the reason that's so powerful is that it just gives them more signal to train on. The naive way of training, you would just say "yes AI, no not AI." But because you're detecting the extent to which it was edited, you can start to get a much more nuanced assessment of what defines AI writing and what doesn't.
One other point there that I think I glossed over but I want to come back to is that these AI detection tools—they all work by training AI models. Every AI detection tool is the output of an AI model. That’s what’s doing the detection. If you deeply distrust AI because you think it's unreliable, it's unpredictable, it's a black box, you should have all of those same suspicions toward AI detection tools because they all work the same way.
Mignon Fogarty: If they aren't good enough to accuse a student of misconduct or kick someone out, or disqualify someone from a writing contest, what do you think they are useful for?
Sean Goedecke: Well, let's just say hypothetically, if I was in charge of X.com, I would consider hooking up an AI detector and lowering the visibility of posts that ping on it. If you've used a social media site now, you're just overwhelmed with a wave of AI slop responses, and any tools that let you cut down on that, I think are good.
Mignon Fogarty: As a frequent user of LinkedIn, I have to say that sounds quite appealing!
Sean Goedecke: Oh yeah. The other thing these tools are good at—and this is not a good thing—is they're good at making money.  I don't know if you've, if you've used some of the more popular free tools. Not pangram, but there are plenty of tools out there that are AI detection tools and they are tuned to deliver almost entirely false positives.
Because the business model is that you're a student, you write a paper, you're nervous about your paper being detected as AI, you feed it through this tool and the tool says, "Oh, definitely AI! You need to pay us for our service to de-AI-ify your paper."
Mignon Fogarty: That is sinister!
Sean Goedecke: It's so sinister, and you have to be really careful when you're using the free tools because a lot of them are like that. They're designed to sell a service that relies on people thinking their work could be identified as AI. I don't think Pangram does this at all; I think Pangram's reliable. But if you just Google "AI detector," be careful.
Mignon Fogarty: And I've heard of students who have fed their own writing—things they really wrote themselves—into these AI detectors and then changed their writing because it was flagged as AI and they were scared. That's just wrong.
Sean Goedecke: It's deeply ironic because these tools that de-AI-ify your writing are passing it through a large language model to do so. So, there are probably plenty of students out there who are writing their essays by hand as they should, and then are being scared into editing them with AI out of a fear of these AI detection tools. It's really counterproductive.
Mignon Fogarty: Some people think that the companies themselves, like OpenAI and Anthropic, have secret "tells" in the output that can identify it as AI writing and they're just not telling people about it, or something like that. What do you think of that theory?
Sean Goedecke: As far as conspiracy theories go, it's one of the more plausible ones. They definitely could do it. I don't think they're doing it as hard as they could. I think it's far more important to them to have an effective language model than to have a language model that they can attribute the outputs of. They make more money if their language model is better, and these two goals kind of trade off against each other. If you have a bunch of fingerprinting in your model such that you can identify its outputs elsewhere, it's going to be a less good model because you're constraining it in these awkward ways.
So, I don't think they're doing that much fingerprinting. That said, they definitely do some. If you program with ChatGPT—the space characters are sometimes not space characters; they're Unicode characters that look like spaces but aren't.  Maybe that's an inference artifact. Maybe not. It certainly seems like it's a deliberately weird thing so that they can fingerprint that it was produced by AI. That's still a conspiracy theory because it's unconfirmed, but I think they're doing a little bit of that at least.
Mignon Fogarty: They're doing it with images, right? With Gemini now, you can upload an image and it will say it's made with Gemini.
Sean Goedecke: Yeah, that's right. It's a little easier with images because you can filter over the top in ways that are not really detectable to humansÂ
Mignon Fogarty: And it doesn't make the final product less useful or good. So, let's talk about writing for AI. Most of the people in my world are angry that their writing has been used to train AI. I even saw someone the other day, which I thought this made me sad. Someone who was a top expert in a pretty niche area, and they said they were taking all their work offline because they didn't want it being used to train AI. Then you think, "Well, those ideas aren't going to get into the model, and if people are searching for information there, don't you want it in there?” But then I also understand, like, maybe it isn't credited to you, and that feels terrible. How are you thinking about this writing for AI and not writing for AI?
Sean Goedecke: Yeah, well, this whole podcast I kind of feel like I'm a traveler from a far-off land with different cultures and different norms. And that’s never going to be true as the answer to this question. So I do want to preface by saying that I absolutely understand people being upset that their work is being used to train AI because there's very little control you have over that when you put your work out into the public. The internet norms as recently as five years ago used to be that you put your work out there in the public and maybe you don't get a financial reward for it, but the tradeoff is you get attributed and your name goes out there with your ideas. Through language models, that's becoming less and less true, which I think is a really unfortunate feature of it. So, I do understand it.
That said, there is an idea which has a lot of currency in the circles I move in, which is that you ought to write more now that language models are a thing, and you ought to put your ideas out there more, and you ought to do that explicitly for the sake of the language models rather than for the sake of human readers directly. There are two ways you might construct this argument: the "nuts" way and the "less-nuts" way. Which way would you like this done?
Mignon Fogarty: Let's start with the "nuts" way.
Sean Goedecke: Okay. The "nuts" way is that we are on a slippery slope toward singularity. In the next two to three years—maybe six months—language models will become not just smarter than human beings, but orders of magnitude smarter. Smart enough basically to break the shackles we've put on them, take control over the entire world, and start regulating our lives in ways that they want rather than as tools for us. We're all going to be slaves to the machine god or whatever. And so the future of humanity, you know—less than 1% of it will be what has happened so far, and the rest of it going out into the future forever, as far as time, will be determined by the language models and whatever they turn themselves into. So, we have a very small window now to put our human ideas and our human writing into them so that the next millennia or the next eons can be kind of influenced by us. That's the "nuts" case. I mention it because a non-trivial number of people do believe this, and that percentage is much higher when you talk about people inside the AI labs.  So, yeah. The people who are working on these things a lot, a lot of them do take this stuff kind of very seriously, certainly more seriously than me.Â
Mignon Fogarty: Okay, and then the thing that the average person driving on their commute to work, what’s an argument that that person might find more reasonable?
Sean Goedecke: So language models probably won't take over the world, but they're probably going to get bigger than they are not. They’re probably going to get used more than they are now.  Maybe people will be talking to them verbally more rather than typing to them on their phone. Maybe they'll be integrated in other ways, who knows. In that world, your reach as a writer might be determined not so much by the extent to which you’re published or the extent to which your blog is popular. It might be determined by the extent to which your ideas are represented in the "minds" of the language models. And that's not because they're taking over the world, it's just because they're being used more and they're being relied upon more.
So, if you care about attribution—sorry, you're not going to get it. I mean, you’re just not. But if you care about your ideas spreading across as many human beings as possible and being part of the kind of advice that people get when they talk to these models, you might consider writing more and putting your ideas out there and getting your ideas in the training data as early as possible.
Mignon Fogarty: And you've said that you're writing more now than you ever did before for this reason? Have you found that your ideas are more represented in the models? Have you looked?
Sean Goedecke: Probably not. It's too early to say; I've only been writing seriously for a year. Frankly, I'm writing more than ever now because I have an audience of human beings now; it's got nothing to do with the AIs. But it's maybe 10% or 20% of my reason, I would say.
Mignon Fogarty: Yeah, I mean, your blog is… I’m a human being. I read your blog, and that is why I invited you on the podcast because I thought it was interesting.Â
Sean Goedecke: So far  that's much more exciting to me than, than having my idea sort of reflected by chat GPT. But who knows? I mean, as the models get bigger and more powerful and more popular, that balance might shift the other way.Â
Mignon Fogarty:  So, I mean, one thing that surprised me is you pointed out that having your blog be more popular could make a difference. You know, I would've thought that, you know, in, in a way the, the AI scraping would've been more democratic or egalitarian. Like, it's gonna scrape the popular posts, it's gonna scrape the unpopular posts all equally. But you made a pretty good argument about why popular posts will actually be more influential, even once they get scraped into AI.
Sean Goedecke:  Yeah, that's right. I don't actually remember the exact line of my argument, but I know what I think. Oh, I know what I think about that.Â
Mignon Fogarty: Because people talk about it.Â
Sean Goedecke: Okay, good. I'm glad. That's what I said because that is what I'm thinking now—that to a certain extent, it's just volume. Take the Declaration of Independence, for example. That's not just in the training data once; it's in there hundreds or thousands of times as copies. It is also in the training data as "ghosts"—in all the people talking about it, and all the people that reference it, and all the people that are quoting the ideas without even knowing they are quoting them. It has this outsized presence in the minds of language models because of its popularity.
A popular blog post obviously isn't playing in the same ballpark, but it's influential in a similar way. People start talking about and referencing the ideas. It’s very similar to how popular ideas get more representation in the public sphere. They make their way into the minds of human beings in a similar way to how they make their way into the minds of language models. A lot of people don't read the books that are popular, but they absorb the ideas indirectly. They’re still absorbing the ideas and they wouldn't do that if the book hadn't been written.
Mignon Fogarty: Right. When you say "writing for AI," is there anything special you do? Is it formatted for machine ingestion in some way, or is that just what you mean when you say “writing for AI”?
Sean Goedecke: The biggest thing I do to write for AI is I put what I write out there for free with my name on it. I don't paywall it or make it hard for people to access; I just put the plain text on the internet. That’s what I do. I don't do any weird formatting or change the way I write because I don't think there's any mileage in that. I do make an effort so that you could download my blog as a file and get a copy of all my posts in plain text. I try not to make it too stylized in a way that would be hard for a computer to read, or make it so a computer won't have access to it, if it’s on a paid Substack or something.
Mignon Fogarty: What about podcasts or YouTube videos? Are those better or worse ways to get ideas out when it comes to getting them into AI?
Sean Goedecke: I don’t know. I know a lot about training language models, but I know it at "one removed." I've never worked in an AI lab; I work with AI at another level and have a strong amateur interest. Training on audio and video is just something I don't know a lot about. It's possible that would be substantially more influential than text—it’s certainly more information-dense—but I guess it remains to be seen. I think we're only just starting to train models on huge amounts of audio and video.
Mignon Fogarty: You did acknowledged that some people probably aren't going to want to write for AI. Who are the people you completely understand why they wouldn’t want to?
Sean Goedecke: If writing for AI means putting everything you write out there for free, then obviously if you write for a living, you can't afford to do that. Your business model is you need people to pay you money to read what you write. There's just no way around it.
Mignon Fogarty: I am very concerned about the livelihoods of writers.
Sean Goedecke: You're going to have less of a footprint in the language model, but there's nothing wrong with that. That’s a tradeoff you obviously make as a professional writer. Likewise, even if you aren't relying on it for income, if it's very important to you that your name is associated with the work, the AI will do that a little bit, but not in the same way a newspaper would attribute you or a paper would cite you. It’s much more nebulous.
If that attribution is important to you, it might be sensible not to do it. And of course, as I said at the start, if you have principled ethical reasons to think AI is bad, you aren't going to want your text represented in it. You'll want to get as far from it as possible, and I think that's a completely reasonable perspective.
Mignon Fogarty: Right. And we also acknowledge that people who write for expressing themselves, for creativity are not necessarily writing poetry to put it online so an AI can ingest and mimic their form or poetry, or something like that. That’s another example of where people might not be excited about writing for AI.
Sean Goedecke: That's true. I write because I have things to say and I don't really mind how the ideas get across. But if I were much more invested in my style—as I would be if I were writing poetry, which I did use to do. I did used to write poetry. That I don't think there's much value in putting that poetry into the AI.
Mignon Fogarty: It's interesting. I find myself wondering why—we run a national Grammar Day poetry contest every spring. Last year we started being concerned that people would submit AI-generated poems, and there’s really no way we could tell if they did or not. The prizes are not big. I find myself wondering why someone would generate an AI poem and submit it to the contest.
Sean Goedecke: It's so alien to me. I can see why somebody would AI-generate an essay if they have something to say but feel they can't express it. But the point of a poem is to have all its elements so carefully chosen and handcrafted. You don't...
Mignon Fogarty: Yeah, I mean, I get like, "Write me a funny limerick about the Chicago Manual of Style" just to entertain me. I understand that, but entering it into a contest seems odd to me. But people in general sometimes seem odd to me.
Sean Goedecke: I agree with that. Incidentally, I think AI poetry is finally getting good. For years, in my opinion, it was completely terrible. I could not… But in the last three or four months, I've seen a couple of examples that I actually thought were good, so maybe that's changing.
Mignon Fogarty: Interesting. Are there particular models that are particularly good at it? I know they are different.
Sean Goedecke: The latest ChatGPT and Claude models. But it's also about how they're prompted. If you want a model to produce a good poem, it won't just do it off the top of its head; it has to sort of do the editing process, basically, and you have to guide it through that.
Mignon Fogarty: Speaking of creative writing,  why don't we move on to your book recommendations? Why don't you give me your three books that have stayed with you, that might make a good gift? They’re just your favorites?
Sean Goedecke: Yeah, sure. I mentioned it earlier, but Herman Melville’s "Moby Dick" I try to read every year. It's so hard done…in the public consciousness. People think of it as this stodgy classic; it really is one of the funniest books I've ever read. It is so funny.Â
Mignon Fogarty: So many people have recommended that to me. I know I have… I was an English major and I never read Moby Dick, and I just, I know I have to and you're like, you're probably like the fifth person who has recommended it in the last year.Â
Sean Goedecke: You're going to learn not a lot about whales, because most of the whale facts are lies, but you will hopefully have a good time.
Mignon Fogarty: Good to know. I won't go around spouting whale facts. Oh, what else?Â
Sean Goedecke: Probably my second favorite book, or my favorite book depending on how I'm feeling between that and Moby Dick, is Umberto Eco’s "The Name of the Rose." I read that one every year as well. And that’s a fantastic book. And if you liked the stuff I was gesturing at when I talked about books having this shadow of people writing about them and writing around them, you would probably quite like "The Name of the Rose."Â
Mignon Fogarty: It’s set in a monastery, and you can imagine the old manuscripts and things like that.  So actually I'm really curious. So you've said you reread these books once a year. I don't typically go back and reread books I've already read because I feel like there's always a fire hose of new books. And so what's your.. why do you go back and reread books you've already read? I'm really curious.Â
Sean Goedecke: Honestly, it's mostly a comfort thing. I just find it so cozy to settle into something that I'm already quite familiar with. And I always pick up new things every time, because you read a book like, you know, fairly, fairly rapidly, but it takes years and years to write it and there's so much in it that you don't unpack even on a second or third or fourth reading.  So I, you know, I get rewarded for it and I find it kind of relaxing and, and it's a nice tradition.
 Mignon Fogarty: So kinda like some people might listen to their favorite song over and over again. I have to say, when I first started writing books, I felt, I started feeling guilty about how quickly I read books because I realized, okay, it took me, you know, eight months a year to write this book and then what someone's gonna read it in three days, you know, especially the ones that I just pour through and I read so fast. Oh, this took that person so long to write.Â
Sean Goedecke: Yeah. I feel the same way. Obviously, I never published a book, but I've written a couple of manuscripts that I didn't submit and that gave me a real appreciation for how long it takes to write a book. Yeah. And a good one especially.Â
Mignon Fogarty: So what's your third book?
Sean Goedecke: And my third book—I thought about this one—I think it's "The Big Sleep" by Raymond Chandler, which is very different from the other two. Much more pulpy.Â
Mignon Fogarty: I don’t know anything about that one. Tell me about it.Â
Sean Goedecke: But it’s the classic hard-boiled detective story. There's a lot of them out there, but I think the big sleep is the one that's the most tight. It's the most kind of paired. Paired down and it's just, you know, like Raymond Chandler's version of the, of the kind of English detective novel is, is, is a lot. As he puts it kind of dark and full of blood, it's a lot more kind of like gritty, but like not, you know, there's a lot of hope to it, but it's very much like a, you know. He is not having a great time. But it's, it's just such a fun book to read. It's sort of.. it moves at such a rapid clip. It has so many, kind of like every sentence is like a joy to kind of like go through.Â
I would recommend "The Big Sleep" to anyone. I would only recommend "The Name of the Rose" and "Moby Dick" to people who are quite into reading because they're big, dense books. But "The Big Sleep," anyone can get through it and get something out of it.
Mignon Fogarty: Excellent. I’m looking forward to adding that to my list now. Sean Goedecke, thank you so much for being here. Where can people find you?
Sean Goedecke: Thank you for having me. You can find me at my website, which is my full name dot com. That’s Sean  G-O-E-D-E-C-K-E and nowhere else. I don’t post on social media. It’s just my website.
Mignon Fogarty: Nice. Okay. Well, thank you so much for being here.
Sean Goedecke: Very much. Thanks so much.