5

This TED presentation says that the New York Department of Education is using a secret algorithm to evaluate teachers:

(..) teachers were being scored with a complex, secret algorithm called the "value-added model (..)

I have looked for this algorithm and there is some criticism surrounding it (Huffington Post):

An algorithm is an automatic way to determine results. No need to have those pesky human factors like judgment, emotion, discernment, understanding, empathy, sympathy, critical reasoning involved.

While using or not an algorithm for evaluation is clearly debatable, I am interested mostly in the not being public part. This sounds strange as teachers do not know what they should improve if they are penalized by it.

Question: Why was the "value-added model" algorithm kept secret?

  • 2
    @KDog it is a policy of public employee evaluations. – user9389 Jul 26 '18 at 15:57
  • 2
    And who owns the IP on the algo? – K Dog Jul 26 '18 at 16:00
  • 8
    @KDog In general, there is no real IP on algorithms (although implementations can be copyrighted). It's probably a secret to prevent teachers from attempting to game the algorithm artificially inflate their score. – user11249 Jul 26 '18 at 16:46
  • 6
    I dont know enough details to know if/why this specifically would have been done. But one reason why you would hide the details of an imperfect algorithm is to prevent people from "gaming the system" or "playing to the test". Basically if there is no knowledge of what is analyzed, people have to act normally and just hope that the magic black-box doesnt overlook their work. Thats all they can do. If they know the algorithm, they can focus on only things that the algorithm grades while ignoring other things, even if those other things are actually really important. – Tal Jul 26 '18 at 17:43
  • 4
    @KDog It seems clear that this is a public policy question. According to the article, the New York Department of Education used this approach to score teachers. So the policy question is, "why was it a secret?". – indigochild Jul 26 '18 at 20:18
4

Algorithm is kept secret because it's developed by a private company and as such proprietary (source).

As far as I know this is the system used by NY Board of Education, however I was not able to confirm it yet. But if it is then it answers your question: it is a product on which many fortunes depend.

  • Same reason why we can't (or "couldn't," if that has changed) look at source code in voting machines to make sure they are honestly tabulating votes. – PoloHoleSet Aug 1 '18 at 17:20
  • @PoloHoleSet - Well, in some countries there is a requirement to keep votes on paper, too (in case of machines printouts). But from experiences in voting in my home country I can tell you that doesn't matter. Stalin said that who's counting the votes matters, not who votes. You wouldn't believe what's going on there. One can also asks Scots - their independence vote was as fine a racket as possible... – user10424 Aug 2 '18 at 10:39
  • Believe me, I'm not claiming that the USA way for stuff like this is the desirable mode, at all. – PoloHoleSet Aug 2 '18 at 15:33
1

One reason to keep scoring algorithms like this secret is to reduce the ability for people to "game" your system by performing poorly but scoring highly. By keeping an algorithm like this a secret, it hopefully encourages people to simply perform their best and hope the algorithm reflects that.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .