With the State of the Union address tonight, The Guardian plotted the Flesh-Kincaid grade levels for past addresses. Each circle represents a state of the union and is sized by the number of words used. Color is used to provide separation between presidents. For example, Obama’s state of the union last year was around the eighth-grade level, and in contrast, James Madison’s 1815 address had a reading level of 25.3.

My guess is this has to do with changes in how we write and talk more than anything else. Lee Drutman and Dan Drinkard for the Sunlight Foundation ran a more rigorous analysis on Congressional records back in May, and the declining trend is similar.

12 Comments

  • Lovely graphic, but I’m not so fond of The Guardian’s companion text.

    I find it interesting that The Guardian’s chart title includes “The state of our union…” when, in fact, a UK paper is not part of the American presidents’ union.

    I also think their labeling of presidents “in order of reading level” is a bit misleading. (The label makes it seem as though most recent US presidents cannot read beyond an 8th grade level.) What they mean to say is “by reading level used in their speeches.”

  • I wonder how much of this is an artifact of words that were once common and ordinary falling out of common use, and therefore being considered a signal of extraordinary vocabulary?

  • Dave Dunkman February 12, 2013 at 4:43 pm

    @MattieF As far as I can tell, whether the word is archaic doesn’t enter into this measure. Flesch-Kincaid is based on syllables/word (are old words longer?), but more relevantly, on words/sentence. I think the major decrease in Reading Level has to be driven by the change in punctuation norms over that period. The Declaration of Independence, for example, uses 10 semi-colons and ends with a 150+ word sentence.

  • There is another way to think about this, and it puts a different perspective on the changes in speech verbiage. What if you graphed the linguistic accessibility of State of the Union speeches instead of reading level? You would find that over time, the State of the Union speeches have become more accessible to more people. It is not as if in 1815, when James Madison gave his speech, that everyone could understand it. But almost everyone can understand Obama’s new speech. That’s a good thing.

    • Excellent point, Karl. James Madison’s SOTU also wasn’t broadcast live to millions of people, which suggests to me that it was written for an entirely different audience than the one tuning in to Obama’s speech last night. I’m not sure if this explains the relatively smooth trend in the graphic, but I suspect you’re on to something.

  • It might be important to note that up until Woodrow Wilson they were delivered in written form to Congress (as noted on the graphic). Now they are speeches given as much, if not more, for the general population. That is to some degree comparing apples to oranges. We would use different wording in a written document to Congress pre 1913 than in oral speeches given to the public (and Congress) in the more recent decades. I wonder if you compare the SOTUs to other presidential speeches given the public at that time how those would compare in reading levels.

    I also wonder if the chart reflects reading level of the time period of each SOTU? Would a reading level of the average citizen in, say 1840, be the same as the reading level in 1980? Just some thoughts that I had.

  • I think it would be interesting to correlate this with the audience reading comprehension level over time.

    Perhaps comprehension was higher back then, or perhaps people understand more now… it would be interesting to find out.

    • I agree — that would be an interesting representation. But I suspect we’d find the opposite. Educational attainment levels and literacy rates are quite a bit higher now than they were in the 1800s…

      • I’ve seen evidence of higher levels of competence in years past. Fewer people may have attained higher levels of formal educations but, in many respects, standards were higher in the past. Nowadays, more people may have access to education but may not be attaining similar quality… an example of one of the problems of using simplistic views/models from data.

        Perhaps, a slightly clearer example would be how life expectancy keeps increasing. People use these “numbers” to suggest that we are so much healthier, etc., but what’s happening is that we are simply eliminating many causes of ill-health, like childhood disease and previously more fatal “older age” diseases. Of course, these are good things. But, the implication in many policy or political situations is that we are so much better off than we were before. On “average,” this may be true, however, we haven’t seen significant changes in the peak ages one lives too. People aren’t now living regularly to 120, which would indicate a true difference in life expectancy…

        Some things to think about…

        Back to the comprehension “question”…

        I recall several years ago (early 1990s) a letter from the president of the university where I used to work. He’d include a letter on some topic every month that was distributed with paychecks. In one particular month’s letter, he included a sample of 10 questions from “a 1895 graduation exam.” The complete exam was given to freshmen entering Harvard in the late 1980s. They scored something like 45% on the exam. Even if you removed questions that involved well-known facts at the time of the original test, and other “unreasonable” questions, the Harvard students still scored under 60% [mean/medium, I can’t recall exactly].

        Anyway, this exam was actually given to middle school students in rural Missouri (to qualify to continue to high school). I recall that the mean/medium(?) score at the time was 75%.

        The president used that example to ponder the state of education…

  • I just read James Madison’s State of the Union speech from December 5, 1815. If that is considered grade level 25, then the level of education in this country is truely appalling. I am sorry, but that speech should not have a level of 25 +. It is basically what a competent high school student (which is not saying much) would have written if they wished to put a more eloquent spin on things. The sole issue here is the unfortunate falling out of usage of words that were more precise, or had a certain aesthetic to them. It cannot be a matter of reading comprehension. I fail to see how such a speech would be inaccessible to much of the populace, and if it is, then certainly most of the so-called rise in educational attainment in this country has been a hoax.

  • The Flesh-Kincaid reading level gauge is based on sentence and word length. That sort of measurement is like measuring the bottom flare of bell-bottom jeans over time. Just as jeans have become more straight and tight all the way to the bottom, language styles have dropped a lot of the flowery phrases and run-on sentences; think Hemingway.

    What is more important is the content of the speech. Obama’s Marxist demagoguery is elementary school level no matter how long his sentences are.