Sunday, April 12, 2020

How the predictions panned out

First, my own predictions:

On March 29, 2020 I predicted 300 deaths (+/- 50) in Georgia by Easter.  On April 4, 2020 I updated that to 400 +/- 50.  The actual death toll in Georgia as of Easter midday is 433.  My original estimate was based on the data we all had at that time.  I simply used a linear regression model and then a fudge factor of four standard deviations.  The death curve, unfortunately, was not linear and so I underestimated by about 24 percent, though my updated estimate was within my margin of error.

How did others do?

Around that same time the Georgia Municipal Association (GMA) gave a presentation to every Mayor in the state.  Their model predicted a range of values for three months from the date of the presentation, which was, I believe, March 25, 2020.  The low figure if shelter in place was done, was 6,000 deaths by the end of June.  If no action was taken, the figure was 27,000 deaths.  A little less than a month into their timeline and we are still under 500 deaths in Georgia.

The IHME model, widely used nationally, has changed several times as more data becomes available, but on March 29th, their predicted death toll for Georgia for Easter was 558 deaths.  Their prediction as of today, for today, is 552 deaths.  They are about 22 percent over the actual number.

So it appears that I under estimated the death toll by about the same amount that IHME over estimated it.  GMA's numbers can't really be tested yet, but look way over estimated so far.

My April 4 estimate was much, much closer than the IHME's for the same time frame.  They missed by about 80; I missed by 33, though I gave myself +/- 50 as a range.  By contrast, the IHME model has a +/- of nearly 200!  My estimate is both more accurate and has much less wide of a range.

I'm not smarter than IHME and I have access to less data than they do, so how did I come in with the better estimate for Georgia?  Skepticism, rational thought, paying no attention to news hysteria, and admittedly, some luck.

My basis for decision making from the first has been to follow the deaths.  Forget the number of cases; look only at the deaths.  Forget modeling, because complex iterative models have unexamined assumptions that can lead you astray.  Look at real world data, graph it to see the shape, and use rational thought, not panic, to make decisions.

In the end, I will admit to being somewhat too optimistic with my initial prediction, but considering the fact that I'm just a guy who likes numbers, not a statistician, not an epidemiologist, I think my analysis is at least as good as some of the so called "experts."

No comments:

Blog Archive