Here's the link to which I'm reacting: https://arxiv.org/abs/2004.04734
I didn't bother to read the entire
paper, but the link is above if you're so inclined. The abstract
alone answers a couple of questions:
What is the confidence interval for the IHME model? Turns out it's 95 percent, which is interesting and about what I expected.
How good is the model? Not very, even with all the updates. Here's the damning sentence from the abstract - "...the true number of next day deaths fell outside the IHME
prediction intervals as much as 70% of the time, in comparison to the expected
value of 5%."
So
if the stated confidence interval is 95 percent, even with a huge +/-
margin, the results should fall within those margins about 95 percent of
the time. Instead, the actual results are outside the generous margins
70 percent of the time. That translates to a 30 percent confidence
interval, which is absolutely ridiculous. Standard statistical work is
usually done at either 99, 97, 95 or in really uncertain cases, 90
percent confidence intervals. And work at 90 percent CI is usually
pretty speculative. 30 percent is unheard of.
I'll
admit that the model looked good five or six weeks ago because it was
"scientific" lending it credence in the swirling uncertainty we were all
operating under. However, tracking actual deaths in Georgia and
comparing to the model, even as it's updated time and time again, it
just doesn't pan out long term. Now somebody presumably smart (since
it's Cornell University) has done the math and found that the stated
accuracy is nowhere near the actual accuracy.
This
is a damning indictment of depending too much on one model, failing to
take in a variety of opinions and predictions and then using some
judgment to make a decision. It looks like our national leaders
especially, just grabbed the model and used it to guide ALL decisions.
Worse, it looks like they're still using it, even after it's been proven
inaccurate.
Part
of the problem as I see it is the extremely variable nature of the
data. New York City is not Atlanta. Atlanta is not Blairsville.
Blairsville is not Dougherty County. Wildly different rates happening
should be compared and contrasted, not lumped together and averaged out
to some impossible one size fits all response.
And
by the way, Georgia's death rate is still climbing, not decreasing.
That's the real data speaking, not Kemp or his advisors, and not a
prediction.
No comments:
Post a Comment