Back in the day, a conspiracy theorist/linguist named Noam Chomsky did some important research to show that ‘Finite State Grammars’ could not model natural language. He argued that computers cannot account for the ‘richness’ of natural human language. This is one of the eternal debates between the social science people and the math people. Computer people attempt to create modular tools to understand language and linguists tell them that their work is essentially useless.
Before Noam Chomsky, people believed that FSG’s (a Markov Model) could be used to model human language. Chomsky believed that Markov Models weren’t adequate since they did not take into account the underlying structure of the grammar. A FSG, by definition, can’t model anything that requires ‘long-distance dependencies’. The general idea of a n-gram Markov Model to model a language works like this:
1. Get a bunch of n-grams.
2. Put probabilities to these n-grams
3. Generate sentence based on the n-grams.
This ends up working pretty well for certain cases. For example, some cool academic people have modeled Shakespeare around this simple idea. If you take a bunch of 3-word or 4-word combinations from the corpus of Shakespeare’s work, you can create novel Shakespeare sentences. For example:
1. Sweet prince, Falstaff shall die.
2. This shall forbid it should be branded, if renown made it empty.
3. What! I will go seek the traitor Gloucester.
4. Will you not tell me who I am?
These are all computer-generated Shakespeare strings based around the idea of an n-gram model. What linguists like Chomsky say is that the language model is always going to reflect the corpus of n-grams. Since we can never have a good enough corpus, and since we can’t model long-distance relationships that occur in language by using an n-gram model, this stuff is limited in scope.
We can do the same shit for rap though. Get a bunch of n-grams, put some probabilities to it and create novel rap lines that rhyme. Here are some examples:
one cup of drank poing while you popping noise //
sucker far for this life it so erotic joys //
huh believe that thats that sport its called his boys//
producers know the haters they be roastin charming toys //
long black with rims so fat and formed is proved//
this gun that gets harder than people starving dude //
yeah verse two hi haters im just walkin nude //
lookin as i stick to themselves and storing food //
And so on. Sometimes the bars are good and sometimes they’re stupid and really bad. But some results are pretty funny. See a partial randomized results board here: