attaching moral qualms to fictional characters

While my family’s experiments with GPT-3 based engines are disappointing from a creative point of view, I got a kick out of some of the results.

People of the Internet (the worst kind of people) given access to GPT-3 worked hard to make the AI say things that are not considered prosocial by the people of OpenAI. Most of this work involves tricking the AI not to use the Assistant personality it’s trained to present. Programmers at OpenAI in turn have trained GPT to be resistant to various kinds of user manipulation.

This has resulted in GPT trying to present a more prosocial reality…about fiction. We asked ChatGPT to summarize the first chapter of The Shadow of the Torturer, a novel by Gene Wolfe. In the summary the AI repeatedly makes remarks about the main character Severian’s discomfort with torture.

The Book of the New Sun’s fictional world has torturer as a professional occupation. There’s a torturer’s guild and a bunch of bureaucracy to go with it. Severian is a torturer by occupation. For him, torture is a somewhat banal experience. He never reflects on its morality, nor does anyone else in the series. But GPT attempts to ascribe its own morality (which is also mine) to Severian.

Severian must be the hero of the novel, so the AI can only attribute heroic words and thoughts to him. The engine does not know what morality means. It only knows which words often follow each other in the English language. The output may only be useful for characters whose morals we agree with.

This is interesting to me in a creative tool sense. It’s possible to use GPT as inspiration for writing prompts, as long as the user looks up everything it declares to be true. For example, I could ask it to list objects in an abandoned warehouse. It would likely come up with some plausible ones and a marmoset, or something equally as preposterous. (The trouble is other things I might have to research)

Again, this engine is derivative by design. I don’t expect to get anything truly creative out of it. For writing prompts my first line of use is various thesauri by Angela Ackerman and Becca Puglisi.

If novelists jump to using using this model to generate their stories, the stories will not only sound derivative, but the characterization will be muted. Good people will have fewer messy flaws than they do in real life. No amoral protagonists, or ones that have morals different from those agreed upon by the scientists at OpenAI!

By reading the same things over and over not out of choice (comfort reads are a different phenomenon that I enjoy with no guilt) but because they’re what’s most easily accessible, we’re more likely to live without challenging ourselves or examining our world views. I find torture horrific, and Wolfe has some sensibilities similar to mine. He sets up an alien society where Severian is uninterested in analyzing his day job. Violence is less remarkable in the The Shadow of the Torturer. But a person in a modern society may have qualms about owning a gun or buying from a shady business, although these activities are common in adjacent cultures.

I find that characters with morality systems different from mine force me to take a look at my values and why I hold them. What would be my values if I held them in a significantly different society? Exposure to different moralities makes us grow as human beings. The magic of a novel is access to infinite ways of life. With this exposure, maybe we can become better people.

The previous generation of AI text output

Lately I’ve been reading articles about corporations experimenting with chatbots that use GPT-3. GPT-3 is an artificial intelligence with the ability to generate a short text article from a text prompt given to it.

Mr. Saturn and his brother and I have played around with various implementations of GPT-2, the previous iteration of the technology. We found the output gloriously absurd and inspirational. Mr. Saturn’s brother tried prompting the engine with “Mr. President, I am writing to you today to…” and we got the following output, which I will attempt to analyze:

Mr. President, I am writing to you today to request an immediate briefing regarding concerns raised by my staff regarding a forthcoming event organized by the U.S. Embassy at the Russian Consulate in Chicago.

GPT-2 assumed that the letter to the President was to a government president, not a corporate president, and it was the President of the United States of America. When I was growing up in the US, one of the school activities everyone had to do in elementary school was write a letter to the President. So there is probably a large amount of letters from people to the US President. I am not sure if this is also a common practice in other English-speaking countries which have a President as head of state.

However, the letter’s opening paragraph suggests that it was written by an adult in the political sphere, as most schoolchildren do not have their own staff. But the idea in my brain is that the President of the United States primarily receives letters from two kinds of people: schoolchildren and bureaucrats. What else do those two populations have in common, besides writing letters to high-ranking politicians? I am not a good enough comedian or philosopher to elaborate on that.

I must note that this text was generated in 2019. At that point, the President of the United States was Donald Trump. Russia had not yet invaded Ukraine. And why would anyone be worried about disrupted events in Chicago in particular? I’m not sure. When I think of disrupted events, I think of Washington DC or Los Angeles thanks to the bubble I grew up in. However, Chicago is a big city, so I’ll allow it.

I understand that the U.S. Embassy will host several prominent political figure-slaying clown bands at its upcoming June 22 event at the Embassy’s Washington Park facility. The Embassy, however, is concerned about the event going forward because of concerns raised by our employees and the local community about the possibility of political violence from the various clown bands.

The fact is that such an event is not in our best interests.

Several political figure-slaying clown bands! I am not the most well-listened person in rock music, so the only clown band that comes to mind is Insane Clown Posse. I am delighted by the idea that ICP would come to a US Embassy to represent us to another nation. Russia is apparently famous for clowns, so I suppose there’s a connection there. Also, I love the political/legalese language of this sentence. It fits the tone of the letter about diplomatic event security.

The Embassy supports the right of individuals to express themselves freely in their neighborhoods. However, given our concerns regarding such clown bands and other political performers in Chicago, the Embassy cannot stand by idly while local residents engage in dangerous acts of violence on behalf of political clowns. The Embassy’s understanding is that the upcoming visit to the U.S. by representatives of “Mr. Trump” will likely include an event scheduled with the participation of the aforementioned clowns, who represent our political leader. In the event of such an event taking place, the Embassy believes it would be helpful to learn more about these clowns and hear from one of their agents.

In 2019, “Mr. Trump” (I love the scare quotes here), was the President and would not send representatives to the United States, as most of his representatives already lived here. However, GPT-2 didn’t know who the President of the United States was, or even what the job of US President was. It only knew what words tended to follow each other in political correspondence at.

However, the phrase “the aforementioned clowns, who represent our political leader” mimics a lot of dialogue by Americans who feel that way about people who vote differently from them. This was especially true of people who disliked the cronies of Mr. Trump.

Sincerely yours,

John Podesta.

What a wonderful punchline. For those who are unfamiliar with him, John Podesta is an American political consultant from Chicago. If I were looking for the name of a person who might write a political letter to the President, John Podesta seems plausible enough. The irony here is that Podesta was affiliated with Clinton, Trump’s opponent, during her presidential run in 2016. The text was generated in 2019, and my friends and I had a good laugh at the time at trying to imagine these concerns leading to this letter being written1.

In any case, this letter, the fruit of machine learning, gave me such joy because the form of the writing was close to what I anticipated a letter to the President from a political operative to look like, but the contents sound outlandish. The beauty of GPT-2 is that it forces us humans to acknowledge and define why its assertions are so funny or disturbing. In that, we can learn a lot of lessons, such as “many humans are afraid of clowns” and “most people think politicians, especially ones mentioned in the same conversation as the word Trump, are not serious and perhaps even clowns.”

Unfortunately, our experiments with GPT-3, GPT-2’s successor, are so far nowhere near as hilarious or enlightening. I’ve found the output boring and derivative.

1. Today, Podesta serves as an advisor to President Biden of the United States. Given the events of 6 January 2020 at the Capitol, the letter would make more sense if it were to be written more recently. And now an image of Juggalos defacing a bust of Zachary Taylor lives forever in my mind.