read

Renouncing My Klout


For the second year in the row, I’m on (Thomas B. Fordham Institute president) Michael Petrilli’s list of “The Top Twitter Feeds in Education Policy.”

Truth be told, it’s only the second year because last year my friend José Vilson asked why there were so few women and people of color on Petrilli’s list and volunteered my name as someone who might be missing. Thus I was added to the list after it was initially published.

And I’m only on the list this year because I didn’t delete my Klout account – one of the metrics Petrilli uses to determine eligibility – soon enough.

Confession: I noticed Petrilli tweet a week or so ago that he was in the middle of prepping this year’s list; and it reminded me that I needed to delete my Klout account. I’ve never cared about my Klout score and I’ve never used the account, but Klout has, without my consent, created an account and a score for me. Thanks, technology industry!

You actually have to log in – even if you’ve never signed up for Klout – to request the company delete your account. I just did this last night for Hack Education’s Twitter account – an account that is, for all intents and purposes, an RSS bot. And I did this for the Klout account linked to @audreywatters. But apparently not in time to disqualify me from Petrilli’s list.

The Fault in our Algorithms


Naming "the top" is a power play, no doubt. But Klout is an incredibly flawed way to rank the “Top Twitter Feeds in Education Policy” in part because the score doesn’t simply reflect Twitter “influence.” (Whatever “influence” might be. More on that below.) The company encourages users to link their Facebook, Google+, LinkedIn, Foursquare, and Instagram accounts as well as their Twitter accounts and uses data from all these services to calculate and to boost Klout scores. (It also uses Wikipedia and Bing search result data to determine the score.)

I don’t have a Facebook or LinkedIn account. There's no Wikipedia entry for "Audrey Watters." So my Klout score, I imagine, is lower for it.

I say “I imagine” because it’s not clear how the Klout score is actually derived. The company says it uses “more than 400 signals from eight different networks to update your Klout Score every day” and uses “machine learning models” to make sense of all the social media data it sucks up. For what it's worth, however, several years ago someone reverse-engineered the Klout score and argued that about 94% of the differences in people's scores could be accounted for by the number of their Twitter followers. Surely, it’s tweaked the algorithm since then. Surely.

But we don’t know. It’s a black box, the company’s “secret sauce.”

Of course, complaints about Klout aren’t new. Science fiction author John Scalzi has said that he quit Klout because “I suspect the service is in fact a little bit socially evil.” Fellow SF writer Charles Stross has also described Klout as “evil” – and quite possibly illegal (as data collection without consent violates UK privacy laws).

Ideology and Ranking


But even if we did know the algorithm that drives the Klout score, I’d still want to ask questions about the meaning of the measurement and the weight that the number – any ranking system, really – carries. Why, if nothing else, are we so obsessed with ranking?

What purposes does Klout serve? Whose purposes does Klout serve? Why is Michael Petrilli or Forbes or Rick Hess or any of the other popular list-makers interested in a ranking or rating system for those in education?

See, this isn’t simply about “influence”; it’s about ideology.

I’m in the middle of writing a chapter for Teaching Machines that examines the histories of “intelligence” and ed-tech – intelligence testing, artificial intelligence, “intelligent tutoring systems.” Much like “influence,” “intelligence” is something difficult to define let alone quantify. And yet we do.

We can debate, as philosophers have for ages, the meaning of these terms – “intelligence,” “influence.” But more importantly, we should ask: why do these characteristics matter? To whom do they matter? And once there’s a practice in place that has defined these terms and has designed measurement tools to assess them and a scale to rank them, we should ask what purposes these designations serve. I don't mean what sorts of perks do you get with your Klout score or your IQ; I mean for us to consider how might these ranking systems reinscribe hierarchy and inequality, all the while purporting to offer an “objective” tool that reflects ability.

Sorta like "science," but not.

So yes, I’ve been thinking a lot lately about the power that comes with crafting definitions, with promoting standards, with devising measurement systems – and the role that technology and algorithms will increasingly play here.

Whose interests do these definitions and standards and measurements and algorithms serve? What sorts of (often unexamined) legacies do these practices carry forward?

From the OED:

psychometry: from the Greek ψῡχο- psycho- + -µετρια measuring – literally “soul-” or “mind-measuring.”

1. The (alleged) faculty of divining, from physical contact or proximity only, the qualities or properties of an object, or of persons or things that have been in contact with it.

The first reported use of this word was 1854 – J. R. Buchanan’s “lectures on the neurological system of anthropology” in which he wrote “The influence of Psychometry will be highly valuable ‥. in the selection from candidates for appointments to important offices.”

2. The measurement of the duration and intensity of mental states or processes.

The first reported use for this definition was 1879 – Frances Galton who wrote “Psychometry ‥. means the art of imposing measurement and number upon operations of the mind, as in the practice of determining the reaction-time of different persons.”

As Mark Garrison writes in his book A Measure of Failure: The Political Origins of Standardized Testing, “Standardized testing – or the theory and practice known as 'psychometrics' – … is not a form of measurement. Psychometrics is best understood as the development of tools for vertical classification and the production of social value.”

Psychometry claims to measure the mind. Klout claims to measure online influence. But look at the OED. Look at those definitions: influence and intelligence. Psychometry and Klout. I'm fascinated how they seem to dovetail so neatly in today's education politics and how readily they become a sort of "disciplinary power" that maintains the functioning of schools, economies, and other hierarchical systems. Who "measures up"?

Audrey Watters


Published

Hack Education

The History of the Future of Education Technology

Back to Archives