If You Paypal Me $7.00 I Will Shit-Talk Your ‘Track’ On This Stupid Blog

Here are 5 ways (in a DOPE listicle) I will shit-talk your song on here if you send me $7.00 @ liban.aliyusuf [at] gmail.com

  1. If your Syllable Per-word average is over ~1.35 I will assume you are a lame nerd that has attached himself to rap music in a superficial way to assuage your intellectual, artistic insecurity through rap music.  It means that your music is a way you can ‘write dope lyrics’ and prove that rap is a medium that has ‘smart’ elements.  You are probably a person that thinks Immortal Technique ‘tells the truth’ about the government.  You listen to Talib Kweli interviews and post it on you FaceBook wall.
  2. Your rhymes are mostly Perfect Rhymes yet you believe you went through ‘hella hard work’ to construct your songs that nobody has yet listened to.  You haven’t went thought hard enough about how to construct weird rhymes and you think that you don’t rhyme in a ‘conventional’ way.   You believe that nobody else ‘can you what you do’.  I will, in an objective manner, prove that simple computational techniques exist to create rhymes that match your ‘tight rhymes’.  They will be ‘slant rhymes’ that a computer has created and, if you are being fair will realize that a stupid computer algorithm has created rhymes as good as what you have written.
  3. You focus very hard on end-rhymes and miss the COMPLEXITY of assonance and other things like that.  You probably have many ‘freestyles’ to important beats like A Milli.  Making your imprint on famous beats is important to you and hitting ‘hella dope’ end-rhymes is your way of creating an important freestyle that sticks in a listener’s head that they hopefully ‘share’ on some social media site.
  4. I will shit-talk the fact that your ‘white’ score is so high or that you G-Drop so often.  This will prove that rap isn’t a thing you ‘need’ to ‘get out’ of a shitty situation but something you do to rebel against immigrant parents or racist parents in your life.  You have a good enough education or are far enough removed from the ‘rap game’ that your involvement in the rap game is something sinister and insincere.  You associate with intangible cultures that don’t exist or that don’t have your best interests in mind.
  5. You have enough Youtube comments that I can prove your fanbase consists of lames that associate with ‘real hip hop’, a thing that doesn’t exist that you use to justify your involvement in a culture that, AT IT’S BEST, promoted violence, sex and drugs.  You want it to be about something greater.  To be about a ‘real’ cause that inspires kids to be educated.  You are fighting a losing battle.  You hate reality.

Email me your tracks at liban.aliyusuf @ gmail.com and I will shit-talk your best song to the best of my abilities.  PAY ME!

Misspelled Words On Youtube Comments As A Metric

I believe that people who comments on Youtube videos, good or bad, provide a solid, measurable way to understand content.   Social media data mining may or may not be a bullshit thing to study but I think if the results (especially ones on either extremes) make sense and pass the ‘eye test’, it’s probably something worth exploring further.  One thing I think makes sense to look at, especially for videos from ‘street’ rappers, is to see if the people commenting type in some sort of unique, measurable way.

Ebonics is a rule-governed language that can sometimes be studied on paper.  For example, something like G-Dropping can be looked at on text.  Another rule of ‘Ebonics’ has to do with word-initial fricatives.  This is a fancy way to say that words like {This} get pronounced {dis} in spoken language.  And sometimes, because this is an example where the spelling of the word goes with the general rules of sound and stuff, people will actually write {dis}.

We can go to the Youtube videos of several artists and look at the distribution of [This,The] with [dis, da].  It turns out, of course, that it’s an excellent way to separate, on extreme ends, different styles and fanbases.  Fans of Pastor Troy speak way differently than Atmosphere fans.

Artist /d/ /th/ Ratio
Soulja Slim 15.52 98.71 6.36
Playa Fly 17.1 108.89 6.37
Guerilla Maab 12.59 103.9 8.25
Beanie Sigel 10.68 88.97 8.33
E.S.G. 10.68 88.97 8.33
Pastor Troy 11.99 110.9 9.25
Yo Gotti 8.26 90.62 10.97
Kane & Abel 11.24 126.63 11.26
Eightball & MJG 9.37 112.73 12.03
Koopsta Knicca 8.62 104.44 12.12
South Park Mexican 7.05 87.76 12.44
Gangsta Boo 6.51 82.6 12.68
Mia X 8.03 106.84 13.31
Continue reading

Visualizing Some Rhyming Things

The new Earl verse is good.  Some slowed-down, descriptive, ‘dark’ multisyllable rhyming from the boy Earl.  Anyways, I put up a visualization online to look at his new verse and used it in a side-by-side comparison with my rhyme dictionary.  So, for each of the big rhyme schemes he’s carrying in this song, I put up what the rhyme dictionary would match with it.  It works really well I think.  So, for the rhyme scheme Earl uses {coldest speaking, ho this decent, hold and squeezin’, home and peacin’}, the RapMetrics rhyme dictionary matches up these rhymes:

only teasing
smokin leanin
sewin’ shearin’
souls of beings
own appealing
older swedish
most concrete thing
modes of pleading
though perceiving
oceaneering
ocean meeting
own policing
coachman ceased his
control of peking
covert feeling

Pretty good rhyme matching.  Anyways, please go to the link to check out what I mean (because WordPress won’t let me do the things on here cause WordPress sucks).

Whippin’ is a cool ‘trap’ verb Pt. 1

I like the Haiti Boyz.  Where New York drug rappers focus on the mafioso expensive spaghetti part of the drug trade, a lot of the cool new DC Youtube rappers exist almost exclusively in the cooking/bagging part of ‘the game’.  The actual ‘shitty’ part that requires hard work.  The reward seems to be in doing this type of work and I identify with it for that reason I think.  The mafia fetish raps was probably always a way for rappers to prove that they were beyond the unglamorous parts of the ugly shit they were involved in.  It’s only one step removed from ‘going legit’.  Or maybe a way to associate with both crime culture and [things associated with high society].

The hook for the song above by the Haiti Boyz goes through a bunch of metaphors where Pooh-Loe brags about his ‘whippin’ abilities.  Stretching x amount of cocaine into y amount of crack (where x>=y).  I really like it.  Anyways, it seems that ‘whippin’ isn’t a very common term in the rap corpus.  It seems odd that, with all the rappers out there rapping about the drug trade, very few are explaining the process of ‘stretching a deuce to an 8′.  Isn’t that important?  It could be that it’s an uninteresting part of the process (doubt it) or that the amount of first-hand experiences doing that are minimal .  Could also just be that it’s not a very good search term.  But maybe it’s evidence that Rick Ross really is one of the best fictional rap storytellers:

  1. whippin keys in the back thats how i stack dough
  2. everyday them hammers bang, whippin yay like annie mae 
  3. uhh and we ship it from haiti, baby im whippin them babies

And of course the hook for BMF where he says “whippin’ work, haleluja”.  Probably the most famous, quotable instance of the term ‘whippin’.  An interesting choice because the whole song is based on the idea of being a drug kingpin and not ‘one of the suckers’ in the kitchen.

The next step with this little project is going through the songs and finding terms relating to the term ‘whippin’ as a drug term and not as a car-related verb (“whippin the benz”) or as a thrashing (“pistol whipping”).  For example, words like “cake”, “keys”, “babies” should be highly correlated to “whippin”.  Once we get a good enough baseline of these terms I think we can put a empirical ‘trap’ score to songs which could be fun.  For example, Waka isn’t a trap rapper.  Neither is Chief Keef.

Who The Hell Is Iggy Azalea?

Who is Iggy Azalea?  Complex tells us she’s a young rapper lady that we should be ‘on the lookout for’.   We learn in this interview that she grew up in a small town in Australia and ‘totally hated school’ and used rap as an ‘escape’ instead of socializing with her peers.  While I was playing around with my new classification tool, some of Iggy’s songs tripped up the system.  It’s okay that the classification is wrong but what kinds of things is a young Australian girl saying to classify her rap style as distinctly ‘black’.  I decided to find out, and here is a partial list of some of the words from My World that tripped up the script.

bitches
g
gangstas
em
mill
stack
money
realest
mines
sparkle
glare

The heavy ‘synth’ in this song makes it hard to make out some of the words but just reading the lyrics it seems that the ‘Iggy’ persona is just a combination of rap memes with nothing distinctly ‘Australian’ in the music.

The NYTimes Said Nice Things About RapMetrics

Whereas “Decoded” and RapGenius at least encourage the exegesis of lyrics, elsewhere online, rap lyrics are treated more like a form of structured data. RapMetrics, for instance, is a fascinating project by a chemical-engineering student named Liban Ali Yusuf; it is both a rhyming dictionary and a source of more-provocative analytical projects. In one, Yusuf sifted through lyrics by white and black artists and generated a list of “white” words and “black” words. A few “black” words: Houston, mink, Impala. A few “white” words: rotting, nuggets, ninja (which: ugh). On the less sophisticated end of things, far away from RapMetrics (which is a truly impressive project) we have the circa 2008 meme of presenting rap lyrics in graphical form. For example, in one tongue-in-cheek graph designed to show the proportional relationship described by Notorious B.I.G., Puff Daddy and Ma$e on “Mo’ Money, Mo’ Problems,” the X axis is labeled “money” and the Y axis is labeled “problems.”

Link

The author Will Staley (a cool dude imo) has problems with rap lyric analysis as a whole but he was pretty cool about this project.  To be fair, it does take some of the fun and mystery away from things when the internet grabs a hold of something and deconstructs it immediately without giving the thing a chance to exhale.

Bill Labov

One of the cooler academics out there is a dope dude by the name of Bill Labov.  While other academics were going to remote pockets of the Amazon to figure out phonological shifts and other stuffs that nobody’s checking for, Bill Labov was doing some of the most important sociolinguistic work in America, fighting for a change in how people thought about language.  In 1971, he wrote a column in the The Atlantic and showed that, if we strip the everyday dialog of some middle-class and lower-class subjects to it’s bare-bone ideas, the bias we have to verbose language and stuff would show.

He talked to two subjects: Larry (lower-class black dude) and Chas  (middle-class black dude).  The questions were framed around mystic beliefs and their answers were recorded.

JL: What happens to you after you die? Do you know?

LARRY: Yeah, I know. (What?) After they put you in the ground, your body turns into–ah–bones, an’ shit.

JL: What happens to your spirit?

LARRY: Your spirit–soon as you die, your spirit leaves you. (And where does the spirit go?) Well, it all depends. (On what?) You know, like some people say if you re good an’ shit, your spirit goin’ t’heaven…’n’ if you bad, your spirit goin’ to hell. Well, bullshit! Your spirit goin’ to hell anyway, good or bad.

JL: Why?

LARRY: Why? I’ll tell you why. Cause. you see, doesn’ nobody really know that it’s a God, y’know. ’cause, I mean I have seen black gods, pink gods, white gods, all color gods. and don t nobody know it’s really a God. An’ when they be sayin’ if you good, you goin’ t’heaven, thas bullshit. ’cause you ain’t goin’ to no heaven, ’cause it ain’t no heaven for you to go to.

When Larry gets called out for saying there’s no heaven even though there’s a hell, he tap dances around it quickly and without hesitation:

JL: Well, if there’s no heaven, how could there be a hell?

LARRY: I mean–ye-eah. Well, let me tell you, it ain’t no hell, ’cause this is hell right here, y’know! (This is hell?) Yeah, this is hell right here!

When Chas is interviewed, he qualifies all his statements and manages to say very little:

CR: Do you know of anything that someone can do, to have someone who has passed on visit him in a dream?

CHAS: Well, I even heard my parents say that there is such a thing as something in dreams, some things like that, and sometimes dreams do come true. I have personally never had a dream come true. I’ve never dreamt that somebody was dying and they actually died (Mhm), or that I was going to have ten dollars the next day and somehow I got ten dollars in my pocket. (Mhm.) I don’t particularly believe in that, I don’t think it’s true. I do feel, though, that there is such a thing as–ah–witchcraft. I do feel that in certain cultures there is such a thing as witchcraft, or some sort of science of witchcraft; I don’t think that it’s just a matter of believing hard enough that there is such a thing as witchcraft. I do believe that there is such a thing that a person can put himself in a state of mind (Mhm), or that–er–something could be given them to intoxicate them in a certain–to a certain frame of mind–that–that could actually be considered witchcraft.

I ran both their passages through my race classification tool and Chas scored way higher on the ‘white’ side than Larry.

Subject Black Score White Score
Larry 0.33 0.26
Chas 0.65 26.28

While Larry’s scores don’t indicate much, Chas’ high ‘white’ score indicates that the rap database as a baseline for verbose middle-class language is a good one. So while the rap database isn’t so good as a corpus for regular speech from lower-class (black) citizens, the ‘white’ scores work pretty well. Obviously we need way more data to say so but it’s a good enough start.

Rap Language Model II

Back in the day, a conspiracy theorist/linguist named Noam Chomsky did some important research to show that ‘Finite State Grammars’ could not model natural language.  He argued that computers cannot account for the ‘richness’ of natural human language.  This is one of the eternal debates between the social science people and the math people.  Computer people attempt to create modular tools to understand language and linguists tell them that their work is essentially useless.

Before Noam Chomsky, people believed that FSG’s (a Markov Model) could be used to model human language.  Chomsky believed that Markov Models weren’t adequate since they did not take into account the underlying structure of the grammar.  A FSG, by definition, can’t model anything that requires ‘long-distance dependencies’. The general idea of a n-gram Markov Model to model a language works like this:

1. Get a bunch of n-grams.
2. Put probabilities to these n-grams
3. Generate sentence based on the n-grams.

This ends up working pretty well for certain cases. For example, some cool academic people have modeled Shakespeare around this simple idea. If you take a bunch of 3-word or 4-word combinations from the corpus of Shakespeare’s work, you can create novel Shakespeare sentences. For example:

1. Sweet prince, Falstaff shall die.
2. This shall forbid it should be branded, if renown made it empty.
3. What! I will go seek the traitor Gloucester.
4. Will you not tell me who I am?

These are all computer-generated Shakespeare strings based around the idea of an n-gram model. What linguists like Chomsky say is that the language model is always going to reflect the corpus of n-grams. Since we can never have a good enough corpus, and since we can’t model long-distance relationships that occur in language by using an n-gram model, this stuff is limited in scope.

We can do the same shit for rap though. Get a bunch of n-grams, put some probabilities to it and create novel rap lines that rhyme. Here are some examples:

1.

one cup of drank poing while you popping noise //
sucker far for this life it so erotic joys //
huh believe that thats that sport its called his boys//
producers know the haters they be roastin charming toys //

2.

long black with rims so fat and formed is proved//
this gun that gets harder than people starving dude //
yeah verse two hi haters im just walkin nude //
lookin as i stick to themselves and storing food //

And so on. Sometimes the bars are good and sometimes they’re stupid and really bad. But some results are pretty funny. See a partial randomized results board here:

http://rapmetrics.com/ROBOTBOARD.php

Trouble and Waka Flocka

One of the cooler songs from 2011.  Trouble is an interesting rapper that probably deserves a ‘thinkpiece’ to describe his ‘anti-rap’ style.  It’s a lot easier to write about a group like Clipse bucking rap motifs (“And I leave it to y’all, to freestyle and battle and shit“) because they still adhere to general rap writing principles.  Pusha T and Malice rhyme at the end of their bars and they have clever metaphors so it’s cool that they hates rappers and consider the word ‘rapper’ a pejorative.  There is some good writing about Trouble though and the author of that piece does a good job of putting Trouble’s work into some sort of historical context.

Stylistically, from an ‘anti-rap’ perspective, Waka and Trouble are a lot alike. Flocka has managed to cross over to the mainstream on the back of some great singles, one classic album and a few great mixtapes.  Obviously, making good music is the most important thing but I think that the two fanbases, of Trouble and Waka, are quite different because Waka is so ubiquitous.  We can measure this by looking at how people are discussing each of their works on the internet.

We can do this by measuring how ‘street’ the comments are versus how ‘backpacker’ the comments are.  I introduced this in the last post.  It would make sense that Waka’s latest efforts would be a lot less ‘street’ than Trouble.  We can do this by simply looking at their respective Youtube pages and getting the data we need.

Artist Street Score
Trouble 48.14
Waka Flocka 4.00

Waka’s score, because it’s so low relative to Trouble indicates that Waka has crossed over more so into the mainstream.  It’s not fair to compare Trouble’s debut mixtape to Flocka 4 years into his career as an established artist.  A fair comparison would be to see how Flocka’s debut mixtape Salute Me or Shoot Me compares to Trouble’s December 17th.

It turns out that Salute Me or Shoot Me has a ‘street score’ of 31.79 which is a lot closer to Trouble’s score of 48.14 and says that the path to mainstream, for guys like Trouble and Flocka, goes through capturing the ‘streets’.  Or at least the people who post Youtube comments from the streets.

It’s like Jay-Z said:

first I snatched the streets, then I snatched the charts. First I had they ear, now I have they heart.

mining youtube comments to learn about rapper fanbases

Most of the work I’ve done from the statistical side so far has been based around the simple idea that looking at lyrics data can help get some sort of insight into the music of any given rapper.  The artists’ work, through this lens, is the only thing that matters.  This way, any conclusions made about a rapper is inferred from the actual lyrics and nothing else.

Obviously this is a severely limited type of analysis.  Certain guys can market themselves as lyrical when really, empirically, nothing they do is that impressive.  Conversely, certain rappers considered ‘simple’ or ‘garbage’ for their lyrical content are in fact empirically complex.  The perception of the fans in this way is just as important as the lyrical content.

Gauging artist perception from the fan/listener P.O.V. is an awkward task.  How do you get good, representative data?  How do you express this data in a way that makes sense?  Luckily, there’s already a good foundation for this type of stuff.  Sentimental Analysis is a field in natural language processing that tries to put objective measures to the attitude of the commenter.  This way, a review can be analyzed statistically to determine the feelings towards the subject on some sort of scale; negative or positive, for example.

Reviews, especially by music critics, aren’t all that great as a resource for a few reasons.  The simplest one is that there just isn’t a lot of great rap writing to mine.  Most rap websites serve as content farms and it’s far more efficient, from a web economic standpoint, to post 10 new songs a day than it is to write in-depth reviews that nobody will read or care about.  The more important factor though is that a ‘star-rating’ sentimental analysis might miss the point.  It’s just way less interesting to know that a troll-wave Riff Raff is a ‘2-star’ rapper than it is to understand what his fanbase consists of.

A ‘star-rating’ system for a movie might affect who watches it but it doesn’t tell us much about why the people who do like it like it or where they’re coming from when they do.  Generally speaking, rap fanbases seem fractured and exclusive.  A’ street’ rapper might be called dumb by ‘backpackers’ and a ‘backpacker’ might be called a lame nerd by everyone else.  To me, the backpacker/street divide is one that deserves some empirical analysis.

To do this, I mined Youtube comments from about 500 different rappers.  I attached certain keywords with each side of the ‘backpacker’/’street’ stuff.  Words like “lyricism”, “real hip hop”,”nigga”, “fucks wit” were strong indicators.  For each artist, I scanned 10 relevant videos with a high volume of comments.  Then put a score to it.

Some obvious problems arise from this methodology:

  1. Are the videos representative of the artists’ total body of work?
  2. Are Youtube commenters representative of an artists’ listenership?
  3. Are the keywords themselves good indicators?

Clearly a good study would try to account for these problems and the other 100 problems that arise from this but fuck it.

So, first, we have two distinct artist types we want to separate.  Pure ‘backpacker’ rappers and pure ‘street’ rappers.  Table 1 shows each categories’ top 20 artists.  It passes the initial eye test.  There doesn’t seem to be any artist that definitely doesn’t belong.  Also, if that list of 40 rappers was given to a rap listener and they were told to put these artists into two distinct categories, it would probably resemble the lists below.

Rank Street Backpacker
1 Richie Rich Jehst
2 Mr. Serv-On Emanon
3 Ahmad Aceyalone
4 C-Murder Lords of the Underground
5 Koopsta Knicca Rah Digga
6 Suga Free Chubb Rock
7 Boyz N Da Hood One.Be.Lo
8 Lil Boosie Talib Kweli
9 B.G. Afu-Ra
10 Yo Gotti Grand Puba
11 C-Bo Kool Moe Dee
12 Yung Ro Das EFX
13 Soulja Slim Crucial Conflict
14 E.S.G. Cru
15 Guerilla Maab A Tribe Called Quest
16 Mac Dre The D.O.C.
17 Lil’ Keke Poor Righteous Teachers
18 Z-Ro Guru
19 Dayton Family Organized Konfusion
20 Kane & Abel Jeru the Damaja

As a quick and dirty tool to seperate styles, this works pretty well for extreme cases it seems.  It becomes apparent that Youtube comments have a direct relation to the type of music and that certain keywords serve as good tests to learn something about an artists’ fanbase.  I’m surer there are modular tools that could be made to look at other genres in the same way but that’s dependent on those genres having some kind of binary divide in fans, I think.

At this point, without really listening to an artist you could learn something.  For example, somebody online suggested that this group Clear Soul Forces was the best new group he’s heard in years.  It seems derivative of NY Premo scratching sneakerhead soft stuff but I wasn’t sure.  Clearly they’re not ‘game changers’ by any means.  They scored safely into the ‘backpacker’ range.  ALGORITHMIC HATE.

***************

If you want to play around with some of the data, use the link below:

http://www.rapmetrics.com/yt6.php