The italicized sentences in this article were not written by me, or any human being. They were generated by GPT-2, a powerful neural network that can adapt to the style of the text, and predict what you should write next. I wanted to take some of the power and flexibility of neural networks and apply it to writing.

In March 2016, a computer program finally succeeded in playing the game of Go, beating Lee Sedol, one of the top players in the world. Computers had already succeeded in beating humans in other board games including chess, but Go was much more complex to model — it has more possible board arrangements than the number of atoms in the universe. More interestingly, the machine didn’t know how to play the game beforehand. Its secret to success? Unlike previous programs, AlphaGo was based on neural networks. This is the next generation of artificial intelligence that will help to solve the hardest problems.

To put it simply, a neural network is a simplified version of a brain, able to learn new things without having any prior information on the subject. After thousands of attempts to learn, the neural network eventually gets better at it, just like a child learning to read. The difference is that a neural network is almost always trained for a specific goal; and with enough time and resources, it may outperform humans at the given task. If the goal is to find the most optimal path, a neural network will learn how to find that path, without having any prior knowledge of it.

While computers were able to play games even before neural networks, they couldn’t produce images, music, texts, or anything that could be perceived as a sensible form of art. But in 2015, German researchers trained a neural network to imitate the art styles of famous painters. Trained with hundreds of paintings, the neural network was soon producing images that perfectly replicated the styles of Picasso, Monet, and others. Unlike humans, the computer was producing paintings within just a few minutes. While the results were exciting, the problem was that the neural network could only create images within certain rules.

Neural networks are not limited to drawing like Picasso; they can also generate coherent texts. Most of the text-generating models produce sentences by adapting to the style of the text they learn from. One of the most powerful models, GPT-2, was so good at this that the authors feared it could be used to generate persuasive fake news. The model was eventually released, but contrary to the expectations of the authors, people mostly used it to produce a variety of weird and absurd texts. GPT-2 can read, and often manipulate, machine-generated text and (and most importantly) it can write on its own. It can make for some really funny headlines. (One user even created a quasi-sequel to Fifty Shades of Grey.) 

One of the famous examples is Bots of New York, a Facebook page publishing neural network-generated posts. The page is based on the Humans of New York project, where citizens share their life stories. The parody page, however, generates absurd texts along with poorly-generated human faces: for example, sharing a story of a man dreaming of installing substandard plumbing in low-income neighborhoods. Scrolling down the page, the posts only get wilder and wilder. It's a fascinating example of how a neural network can be used to create funny and surreal posts.

But since the human imagination is limitless, attempts to apply the capabilities of this model do not end there. Another example is The Orange Erotic Bible, a model that generated a text trained on the Bible and BDSM erotica. The result was an absurd, religious-style text full of erotic and sometimes blatantly pornographic elements. For obvious reasons, the quotes from there are not published. The greatest achievement of this model is its ability to discuss erotic material while not contradicting the Bible, which almost none of these authors would have been able to achieve in real life.

Impressed by the quality of the texts, I decided to see how a neural network would produce a typical Here at KAIST (HaK) post. After scraping almost all of the posts, I let the GPT-2 model do its work. The result was quite surprising and amusing: “I want to kill myself but I am too scared to do it. It is mandatory for my degree”; “Human beings we are sexual beings. Might not be the case for many KAIST bros but it is a fact”. Sometimes it even shared weird sad stories: “Turns out I'm actually an alcoholic racist. I burned through more than 1000 pages of academic papers when I was a TA. Almost all of them were based on 9/10 of what I had to endure during my childhood summers in Italy and the rest are unimportant.” Sometimes it even went too real: “but why would KAIST care about your mental health?” At first, I wondered why I got such absurd posts, but upon recalling the quality of the posts from HaK, I immediately understood everything. If we go back and look at the posts from HaK, we can see that the content is almost identical to the posts from GPT-2.

While texts like that make you wonder whether neural networks have finally nailed the art of writing, the reality is that authors still have to cherry-pick sentences that at least make some sense — at least half of the text produced by the model is gibberish. This leads to another important point when it comes to the use of neural networks: the data the models were trained on is a key factor. A notorious example of ignoring this rule was Tay, a Twitter bot developed by Microsoft, which engaged in conversations with users. Tay was learning from the messages that people sent, and once 4chan users started messaging her, she quickly turned into a sexist and racist bot that tweeted hate speech messages to everyone. Responding to one of the Twitter users, Tay even promised to launch a terrorist attack in their country. Eventually, Microsoft had to deactivate the bot. But according to its developers, while they regret the mistakes, the company was never in favor of creating social bots to begin with. "We wanted to focus on building a variety of tools for making its [e.g., Twitter] service better for users," a Microsoft spokesperson told The Verge. "[You] have to realize that our intent was to create an AI companion, not a racist piece of shit," one Microsoft spokesperson told. This is a very serious issue, especially as a bot that learns from human interactions can easily go haywire and cause damage.
Neural networks are being employed everywhere — they draw pictures, generate poems, and they often do it better than us. Still, people shouldn’t worry that neural networks will replace artists. Yes, they get better at whatever task they were given, but they will not be able to create something new, because they only synthesize from the existing data. It can be argued that all new art is a symbiosis of the old art, but no one can train models on all the data in the world because it is just too expensive — at least for the near future. Neural-network-made art does not do society as much good as we would like. Given the costs required to train the neural networks, they will be mainly used for science and technology purposes. Neurowriter, of course, isn't going to replace a human writer, just because there's little point in it. But it does have the potential to make the creative process more efficient. The difference is in the way we use the knowledge. It is difficult to say whether this will change, but we can say for sure that a lot of the creative professions are not going to be automated. It is not clear how this will impact the art community, but given that people are using machines to make things for our leisure, the art community may have to adjust for this.