Stuff Smart People Like

For smart people, by smart people
Contributing Authors

Recent Tweets @smartppllike

We are honored to bring Hope to the lives
of our nation's heroes and their families:





"no sacrifice forgotten, nor need unmet"


Hope For The Warriors ®

Posts tagged "statistics"

jeffrubinjeffrubin:

“Throughout this whole rap of 44 bars, there are 496 syllables, and of those syllables, 215 are rhymed. That means that 43% of his syllables are rhymed, which is one of the highest rates you will find for any rapper. For instance, as quoted in this article here, Camron has a rate of .41 rhymes per syllable, Eminem has a rate of .38 rhymes per syllable.”

- An in-depth statistical analysis of why MF Doom may be the most technically accomplished rapper

Paul’s always insightful analysis finds itself focused on the topic of Bracketology. Also, there’s a little shout out to us there. Heyo!

Chart Attack: Percentage of Ph.D.’s in Academic Jobs by Graduation

Well, now I feel better…

-Ed

(via How Many Ph.D.’s Actually Get to Become College Professors? - Jordan Weissmann - The Atlantic)

Chart Attack: Lower Unemployment Rate for Bachelor’s Degree Holders

It’s hard to argue with the implications of this graph.

Yadda, Yadda, “correlation does not equal causation” Yadda.  

(via A Case for College: The Unemployment Rate for Bachelor’s-Degree Holders Is 3.7 Percent - Derek Thompson - The Atlantic)

Chart Attack: Do shorter sentences = dumber speeches?

Here’s a very interesting counterpoint to all the “Presidential speeches are getting dumber” arguments. Changes in the language and construction of the speech may not represent anything but a stylistic change on the part of the speech writer.

(via The State of The Guardian’s SOTU Infographic Is…Dumber | Popular Science)

Chart Attack: Correlation between Autism Diagnosis and Organic Food Sales

Maybe the best example of correlation not being equal to causation that I have ever seen (r = .99!). Also, reminds me of a classic XKCD comic.

(via Correlation between autism diagnosis and organic food sales - Boing Boing)

Kind of an ambitious effort by atheists/humanists/secularists to get some data about like minded individuals across the world.  Let’s you see the numbers according to country and broken down by various demographic variables.

Chart Attack: A Decline in the U.S. Prison Population?

Three data points do not a trend make. But let’s hope.

(via US prison population on the decline)

Chart Attack: Video Game Spending per Capita vs. Gun-Related murders per 100,000

Not sure why they chose to look at only 10 countries, but the U.S. is definitely an outlier with regard to one of these variables…

(via Ten-country comparison suggests there’s little or no link between video games and gun murders)

Chart Attack: Consumer Demand Curves

A treasure trove of interesting charts that track average consumer spending by age from HSdent. I had a hard time picking a favorite. But I thought this one was interesting. What’s driving the spike? Mid-life crises? Or having kids in band?

Students aren’t the only ones cheating—some professors are, too. Uri Simonsohn is out to bust them.

(via The Data Vigilante - Christopher Shea - The Atlantic)

statwars

It’s a stat! For the Forty-Force episode, the fellows welcome guest panelist, Paul to discuss the statistical particulars that you can’t learn from a Jedi… Plus the Smart Suggestions abound, as always.

Download the .Mp3


Smart Suggestions:

Main Segment: 

Quantifying Star Wars

Read More

Here’s the link

This is a great collection of the various ways you can edit (or depending on your political perspective, fudge) your charts to get them to suggest something they don’t actually suggest.

My favorite example is changing the order of magnitude of the x value, which to me is flat out fudging. Here’s the example from the post: 

The trend line makes it look like the numbers have gone up and then slightly down but have never gone under the original mark from the beginning of 2011. So the conclusion you are supposed to draw is that things haven’t gotten any better, for a time they were worse, and are probably now just as bad or worse as when we started the year.

However, if you ignore the trend line and instead look at the numbers they tell a different story. To see what I mean, compare the number and point from Jan (first point) to the number and point for Nov (last point). It’s like an optical illusion. For a bigger effect compare the number and point for Mar (third point) to the number and point for Nov. Whoa! 8.6% is higher on the graph than 8.8%. That’s pretty magical. Here’s what it looks like graphed out with the proper units across all data points (again from the post):

This graph still suggests that things got worse for a while but if you want to start comparing differences in magnitude the highest the rate had been in this time period was .2 percentage points above the baseline %. At the end things are .4 percentage points better than the baseline.

Two different graphs, two very different stories.* Check out the full article for more.

Also, further reading (a classic!).

*My personal take on this — and I am a bleeding heart liberal — is that these numbers don’t represent much of anything but random fluctuation. That is, there might be some evidence of a slight downward trend but it’s really only driven by the last data point (i.e., more data points, probably a year’s worth are needed). So what puzzles me is that if they were going to fudge the numbers anyway why not just leave the last data point off and keep the x values the same rather than changing the magnitude of the x values?


A Student emailed me today and I spent so much time on the response I thought I should post it somewhere. Below is a modified version of the email as well as a somewhat more modified version of my response. Enjoy! (ed note: PSY 236 is an introductory personality psychology course).

Hi Dr. Witt,

What was the name of the band you showed us in class today? There were like 30 people in it all in one room doing different things (popping balloons, sweeping etc). they were awesome!

Also, random question - my friends & I were talking about testing taking. In high school, we were always the first ones to finish tests in class and did well on them. Now in college, we still perform well on the tests/exams but we’re often the last ones to finish. Presumably our Intelligence has increased, but why do we take longer to finish tests? Not sure if you’d know, but I thought I would ask.

Thanks!

PSY 236 Student

Hi PSY 236 Student,

Glad to hear that you liked the band. Their name is Efterklang.

As far as the test question is concerned. I think the simplest solution is that you are being made aware of something called “restriction of range”. Be prepared for me to nerd out a bit here as I attempt to illustrate:

Not all the people who attended your High school went to college (based on some estimates I’d say about 25% of them did). So now that you are in college you aren’t surrounded by the same people that you were in high school. You’re surrounded by the top 25% of those people (from your high school and from other high schools). Thus the range of ability (intelligence, test performance) has been “restricted”. 

Let’s do a quick simulation to demonstrate what I mean.

Imagine that you had 100 random people (all high school students). You know each person’s average test score across all of the tests they took in high school (expressed as a percentage) and you also know the average time it took them to complete all the tests they took in high school. If you plotted this (see graph below) you would see that there is a strong negative association between the two variables in high school students (~ r = -.75). That is, the less time it takes a person to take a test on average, the better their test scores tend to be (this is what you observed when you were in high school).

test scores and time taken for 100 random high school kids

Okay, now let’s focus on the top 25 scorers (just as only the top 25% of HS students get into college). Without doing anything (e.g., collecting information about their performance in college courses) we can look at the relationship between their test scores and time it takes to take a test (see graph below). You can see that the relationship between test scores and test time is much weaker in these high performers (~ r = -.35). In fact, you and your friends might be those two dots in the top right corner. That is, people who score well on the exams but take longer. Does this make sense?

top 25 high school students performance versus time taken

The range gets restricted more and more as you move through your education. Many people go to grade/elementary school. Most of them go to High school. Some of those people don’t make it through high school. Then a smaller number of people who complete high school go to college. Some of those people don’t make it through college. Then a much smaller percentage of college graduates go to graduate or professional school. The range gets smaller and smaller and as a result so does the association to the point where it might be no correlation or it might actually change from a negative to a positive association.

In graduate school many of my colleagues (myself included) had trouble with the transition because they were no longer the smartest kids in the class. This was so common that I coined a term for it. I called it “the superman effect.” When superman was on earth he was superman (our sun gave him special powers), but when we was on Krypton (his home planet) he was just like everyone else (their sun was red and kept their powers at bay). So to go from Earth, where he spent most of his life and had super powers. to a planet that had a red sun (such as Krypton, though he couldn’t do that because it had exploded) would have been a humbling experience.

My first experience with a graded assignment in graduate school was quite the wake-up call. I received a 92% on a statistics exam. I was pretty happy that I had done so well. However, I later found out the class average was 94%. I definitely wasn’t on Earth anymore…

Probably more of an answer than you wanted, aren’t you glad you asked?

Dr. Witt

An imagined love letter from a 12 year old girl to Nate Silver

An excerpt: 

I wonder if when you get up in the morning you open your kitchen cabinet and go, I’m feeling 18.5% Rice Chex and 27.9% Frosted Mini-Wheats and 32% one of those whole-grain Kashi cereals which have photos of smiling multicultural people on the boxes, as if smiling multicultural people were a new form of fibre. And then I wonder if you think, But I’m really feeling 58.3% like having a cupcake for breakfast, but then your mom says, “I don’t care if you’re a fancy statistician with a Times blog and Seattle green-architect eyeglass frames, you still need something heart-healthy to start your day,” Read more: http://www.newyorker.com/humor/2012/11/19/121119sh_shouts_rudnick#ixzz2C2qCgs1N

(via Paul Rudnick: My Crush on Nate Silver : The New Yorker)