Monday, February 15, 2010

Is PECOTA Over?

Check out this tweet. (Hat tip, BCB.)

I've been thinking the same thing as I get my various iterations of the PECOTA spreadsheets. Something isn't right here.

Here's what we know:

1. BP has been hemorrhaging talent, the latest being Joe Sheehan.

2. Nate Silver is gone, and Nate Silver, more than anyone else, is responsible for PECOTA.

3. BP has insinuated that they have attempted to further automate PECOTA, transitioning it from a bunch of somewhat disorganized spreasheets that Nate could make sense of to a more user friendly system.

I'm not going to speculate on exactly what is going wrong, but it's easy imagine several ways in which PECOTA could have gone wrong given these facts. It's been all over the map to this point, and I'm skeptical this year as well.

9 comments:

Jon said...

I'm curious, what exactly did you "use" Pecota for anyway?

PaulNoonan said...

Occasionally for FB. Mostly just for evaluating the teams I'm watching.

Jon said...

Couldnt you just look at their actual stats (current and past years) for that?

PaulNoonan said...

Those are not the same things. So no.

Jon said...

duh, i'm aware that pecota projects stats forward, and thus is not the same as looking at past stats. but in order to evaluate a team your watching, do you really need some super advanced projection system (that in reality barely does better than a weighted average of past 3 years)? or maybe just some common sense and looking at past performance on your own. i honestly dont see much point to pecota unless your a GM or playing fantasy baseball.

PaulNoonan said...

I didn't really understand your question. It seemed like asking why I would read a novel when I already know the alphabet.

To counter:

1. I subscribe to BP anyway because I like reading about baseball (though they have admittedly gone downhill a bit). If you do this, PECOTA is "free".

2. I find PECOTA interesting.

3. I also read Hardball Times and the Zips projections and indeed, the weighted average of the past 3 years in the form of the Marcel Projections.

4. I occasionally do some writing here and the projection systems in general make for some good fodder.

5. When someone is way off of their projection I find it interesting to try and find out why.

6. PECOTA was one of the first advanced baseball things I really discovered (especially on the 'net" and so I have some nostalgia for it.

7. In 2007 when PECOTA projected the White Sox to drop from 90 to 72 wins and a bunch of players and radio people got all whiny about it, and then they went out and won 72 games...that was fun.

Jon said...

I'll try to clarify, though your analogy of alphabet-novel is a bit far fetched, considering pecota at its best was maybe 5% better than marcel (just looking at past stats). i'm not knocking BP subscription, i have half of one too, or interest in pecota projections i guess. i'm just wondering whats the big deal if they're messed up, since they dont add much value. i dont do anythign with them other than occasionally making over/under win bets in vegas (i made money betting on the 2007 white sox and 2008 rays too), but you can get those types of projections from lots of places. and we both know that 2007 projection, as funny as it was, was incredibly lucky to be that accurate (i think those chisox were bad in totally different ways than the pecota prediction expected).

PaulNoonan said...

Agreed, the Sox thing was lucky.

I don't think it is that important in the grand scheme. Mainly I think it speaks to BP's current state, and whether they can remain relevant with just Golstein/Goldman/Karhl and Carrol (with Davenport allegedly behind the scenes) and assorted others. I was skeptical when Nate left. I think he drove a good deal of their most insightful work.

And if you are playing fantasy this year, you can probably do just as well with Marcel.

PaulNoonan said...

I also think they trade on PECOTA. It's very rare that I hear BP discussed without PECOTA being mentioned.