Unanswerable Questions in Educational Accountability

The debates surrounding the preferred structure of accountability systems in the education sector revolve around a few important questions.  As Figlio and Loeb (2011) point out, these discussions often focus on the level of accountability, whether or not to use extrinsic rewards, and the measurement of agent behavior.  While the merits to the various answers to these questions should be considered in depth, it is impossible to find optimal solutions to each of these problems.  In fact, it appears that the very quest to find the answers to these problems within the system of residentially assigned public schooling actually increases the number of remaining questions.

What Level?

First, at what level should we design accountability systems to operate at?  Scholars discuss the most preferred level for accountability systems, but any level is sure to result in principal-agent problems somewhere.  After all, the education sector is fraught with several complicated principal-agent relationships.  These relationships are intertwined between various actors such as parents, children, teachers, unions, school boards, principals, superintendents, mayors, and other representatives all the way up to the national level.  Since it would be extremely complex and costly to implement accountability systems for every single relationship, it is important that we at least discuss the merits of a couple of important types.

It may appear wise to set accountability standards at the school level.  If we do so, the leader of the school can make decisions within the school in order to create a successful institution.  Similarly, in the private industry, customers review the quality of different brands, and sometimes even review the quality of locations set up at specific geographic locations.  Thanks to historical customer reviews being available at our fingertips today, customers can determine if a specific fast food restaurant location is high or low quality.  However, it does not seem very useful to collect that information at the employee (or teacher) level.

Nevertheless, what about the school leader relationship with the teacher?  How is the school leader to know whether a specific teacher is high or low quality?  The principal needs a way to measure teacher quality other than how the given teacher treats the school leader.  Thus, an accountability measure is at least necessary at the teacher level.

A Problem with Measures

The problem here is that even the best econometric measures of teacher quality rely on a few important assumptions.  The often-praised measure of “value-added” requires that increases in the observed measure is desirable, and that maximizing the measure is preferable.

Furthermore, proponents of value-added measures often forget that the model assumes exogenous sorting of students and teachers.  As Derek Neal (2008) pointed out, the model assumes that school leaders are not doing their jobs at all.  After all, the job of the school principal is to find teachers that will help their specific types of students and to match students and teachers effectively.  Since (we hope) principals assign students to teachers that will best serve them, the value-added model fails the exogenous sorting assumption.  If the model fails one of its most important assumptions, how could we conscientiously assign rewards or punishments to teachers based on them?

Monetary Rewards and Opportunity Costs

The next question is whether we should give explicit rewards and/or punishments based on the measures.  For example, some scholars have proposed to use monetary incentives for teachers, or merit pay, in the educational setting.  Sure, monetary incentives allow the principal to shape the actions of the agent (or the teacher).  However, how does the school leader identify the optimal level of the measure, and, perhaps even more importantly, how do they determine if the additional costs incurred are worth the additional benefits?

This is a common theme in discussions of accountability systems: even bright scholars will forget the basic, but important, economic lesson of opportunity costs.  If a monetary incentive raises test scores, many people will argue that the proposed policy is ideal, especially since spending, in general, has not correlated with any improvement in educational outcomes that we care about (Hanushek, 1996).

However, those same people will completely disregard the other side of the equation by not asking the more important question: “Is the observed result worth the cost?  And what other programs could the same resources be allocated towards?”  In addition, do we want to maximize test scores, or do we simply want to reach a certain level and then focus on something else?  Surely, if we allocate 100% of our educational resources towards test scores, many children will suffer from decreased creativity levels and harmed social skills (Neal, 2010).

In addition, if we allocate a billion dollars to each student and find that they have magnificent test score gains in every subject, would the investment be worth it?  What if long-term outcomes also increased?  Suppose that we somehow knew, with certainty, that the billion-dollar investment, per student, doubled their lifetime earnings (an increase of a little over a million dollars, on average).  Obviously, this would be a horrible investment.  Instead, the investor should just give each student two million dollars.  The student would be better off (by one million dollars), and the investor would spend 99.8% less.

How to Fix These Problems

In short, it seems that an attempt to answer any of these accountability questions results in many more questions.  In this sense, answers to accountability problems within the system of residentially assigned public schooling do not reduce the number of remaining questions, but increase them exponentially.  Due to the complexity of relationships and diverse goals within societies, accountability systems cannot function within the public sector of schooling.  Perhaps most importantly, the very attempt to control the various mechanisms of these principal-agent relationships is likely to cause much unintended harm.  Instead, the schooling industry would perform much better in the complex, yet simple system of voluntary exchange.  All of the complicated relationships, goals, and desires would be efficiently communicated through the spontaneous formation of a single metric: price.


Image Source: http://dosminutos.complot.net/wp-content/uploads/2015/04/Problema.jpg


One thought on “Unanswerable Questions in Educational Accountability

  1. This was an amazing article and right on the money. Treating schooling using capitalist pressures in the marketplace is much better than trying to use noncompetivie means. Looks like there is no real way to improve education through incentives that are politically driven. Look at the cheating teachers found guilty in Atlanta. Keep up this every difficult topic its needs this emphasis. Good work.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s