Test Scores, Profits, and Page Views are Measurements, Not Primary GoalsPosted: January 23, 2012 | Author: ThomasPowell | Filed under: soapbox | Tags: business, education, metrics | Leave a comment »
I’m a little disturbed by at least a couple of news story themes that I’ve heard recently: “Can Eliminating [X program] Improve Test Scores?” and “Students to be Moved to Schools to Improve [the school’s] Test Scores.” The former concept sounds like a fishing expedition to find a way to take money from “non-essential” programs. The latter concept sounds like a bureaucratic shell game.
What if you substituted “Earnings Per Share” or “Page Views” in the place of “Standardized Test Scores” and “public company X” for “school X”?
Measurements are just one instantaneous interpretation of the health of the mission — whether business or education. In either case, the numbers can be gamed in a given year… shuffling above-average students to underperforming schools, teaching to the standardized test, deferring decisions on actions negative to the bottom line, expediting actions positive to the bottom line, etc…
How does any of this improve the health of the business or the overall education of the student?
When you make your measurement your ultimate goal, you neglect the actual mission the organization was founded on–providing value to your customers. You never take the risk of scrutinizing the measurement to improve it, because doing so might set you back against your goal. In never changing how you measure your organization, you develop an ever-widening blind spot for any trouble that’s brewing.
Have you sacrificed a child’s long term development–eliminating music, art, and physical education–just so that their test scores this year will be better than last year?
Have you turned away customers for good just to save money on an offshore customer support call center?
Have you made your site the most awful in the world, just so every visitor generates 50 page views instead of just 1?
While it may not seem all that relevant, this thinking around measurements (called “vanity metrics” in the book) were inspired by The Lean Startup (affiliate link). It seems that a lot of energy in many endeavors is wasted in coming up with ways to see “how good we’re doing” instead of understanding if we’re truly “doing good” at all. I, personally, think it’s an eye-opening read that may challenge your habits and the habits of your organization.