Nate Silver Under Fire
The election analyst is out on a limb with Trump.
Nate Silver made a name for himself in the field of baseball analytics (“SABRmetrics”) and then became a household name among election nerds when he started blogging at DailyKos and then his own FiveThirtyEight site during the 2008 election. That ultimHeately led him to the larger spotlight at the New York Times (2010-2012) and Disney/ABC until he was let go during a round of massive layoffs last April. He has subsequently taken his show to SubStack.
After showing a massive swing blueward after President Biden dropped out of the race (which Silver had been actively encouraging for months), giving Kamala Harris a 60% shot at winning the election, he has been something of an outlier in showing no convention bounce for Harris and actually predicting that Trump was actually in the lead. I’ve mentioned this before but, since I’m not a paid subscriber to his site, haven’t followed up since I’m not privy to the analytics.
In yesterday’s Open Forum, @charontwo posted a link to an Adam Silverman thread on Bluesky Social drawing a line between Russian influence operations, Peter Thiel’s investment in political betting platform called Polymarket, Silver’s joining said market, and Silver’s prediction flip.
The evidence beyond the post hoc ergo propter hoc is, to put it generously, thin. But Silverman notes,
Remember, Silver’s model is proprietary, so there’s no way to replicate (stress test for accuracy) his results. Without being able to replicate them, the model cannot be verified. This is the opposite of social, behavioral, or any other type of science or application of scientific methodology.
He notes political scientist Rachel Bitecofer makes the same point, going so far as to call it “unethical in terms of research standards” and “anti-theoretical.”
But, of course, Silver isn’t an academic social scientist but a stats nerd trying to make a living getting people to pay for the results of his proprietary model. It’s absurd to expect him to make it freely available.
A Salon roundup of the pile-on (“Nate Silver faces backlash for pro-Trump model skewing“) adds:
Silver left the ABC-owned company and launched his own Substack, the Silver Bulletin, last year. Since then, he’s become increasingly critical of the FiveThirtyEight projection and its behind-the-scenes assumptions and adjustments.
“There’s a fine line between an “objective” statistical model and “just some dude’s opinion,” Silver wrote in mid-July, a month after launching his own competing election model. “But the 538 model falls somewhere on the wrong side of this line, in my view.”
Some social media users have denounced his complaints as not based in math, especially as he makes similar adjustments to his data. Silver has adopted the FiveThirtyEight system of weighting polls differently, ostensibly based on reliability.
First off, as already noted, Silver didn’t “leave” ABC, he was let go. And, as he’s repeatedly pointed out (apparently to no avail), while the company retained the rights to the FiveThirtyEight brand, he retained the right to the FiveThirtyEight model, which was his own creation. So, he didn’t “adopt the FiveThirtyEight system,” he took it with them—and the site operating under that name now uses a completely different system that’s mostly driven by non-polling factors.
He’s facing criticism for allegedly favoring junk polls over respected pollsters.
“Patriot Polling is literally run by two right wing high school students that is ranked 240th on FiveThirtyEight,” former pollster Adam Carlson noted on X, asking why that poll was weighted more highly than a YouGov poll, which they called “an internationally respected pollster that is ranked 4th on FiveThirtyEight.”
Some users believe that Silver’s methods of weighting polls are dubious, especially as his swing state calculated “polling averages” move in the opposite direction as recently released swing state polls.
Silver’s reaction to all this on the Platform Formerly Known as Twitter has been characteristically abrasive. But basically he argues that he’s applying a model that has been shown to be effective for quite a long time and trusting his process. Further, he argues that others are simply not weighting the significant skew presented by the Electoral College highly enough and that Harris’ showing in recent swing state polls has in fact moved in the wrong direction.
That seems plausible to me. And, indeed, the latest NYT/Sienna poll (which, Silver notes, is the highest-rated poll in the FiveThirtyEight model) shows Trump with a slight national lead and neck-and-neck in the swing states.
Silverman’s counter is, to say the least, bold:
Is Thiel or Silver taking RU funding? Thiel probably has unsanctioned RU money in his investment fund. But not like Benny was being paid. Here we have the two different revenue & influence to establish reflexive control streams move in the same direction until they overlap, merge, and become one.
The purpose of all of this is demoralization of Democratic turnout; exhaustion of Democrats, Democratic leaning voters in having to deal with this bouncing around the information domain; normalization of Polymarket as a legit political forecaster; amplification of Trump as the likely winner.
Drive positive sentiment for Harris down in the info domain & up for Trump. All of this is exactly how Prigozhin’s network operated in the Sahel & Madagascar under contract to the GRU prior to Putin extrajudicially executing him. It’s also the first part of the subversion process they used.
Silver is a gambler. Indeed, his latest book is about that subject. So, I’m inclined to believe that he’s pushing his chips to the middle betting on his model.
Is it possible that he’s instead pushing his reputation on the line to push Russian propaganda? I . . . suppose? But the payout would have to be pretty big, given that being wildly wrong on this election would presumably be the end of his career as a political odds savant.
I could not let a post referencing gambling; Lawyers, Guns, and Money (Silverman); and the Russians go by without a nod to Warren Zevon, who presaged this all many years ago:
Well, I went home with the waitress
The way I always do
How was I to know
She was with the Russians, tooI was gambling in Havana
I took a little risk
Send lawyers, guns and money
Dad, get me out of thisI’m the innocent bystander
Somehow I got stuck
Between the rock and the hard place
And I’m down on my luck
And I’m down on my luck
And I’m down on my luckNow I’m hiding in Honduras
I’m a desperate man
Send lawyers, guns and money
The shit has hit the fan
My growing concern with Silver is not, at the moment, Russian money (but I will confess it would not shock me if some tie was shown given what we have seen of late).
My concern is that his current business model requires agitation on Twitter to drive eyeballs in ways that was not the case with his gigs at the NYT and ABC. As I have noted, he basically said this on the Ezra Klein Show about a month ago.
Moreover, Silver has always had a bit of the “Libertarian Bro” persona which seems of late to have dialed up to 11, and, worse, he seems to admire the Tech Bro Billionaires and is increasingly emulating them.
I still have some confidence in his model, but we will see how that plays out. I am increasingly seeing him as that which he has claimed not to be: a pundit.
One other thing: he has an open disdain for academics, but it seems to me that he kind of wants to be one both in terms of his modeling, but his books seem to attempt a bit of that whiff of academic popular press styling.
Random thoughts:
1. As with everything in this cycle, we’re off the map. Given the dynamics of Harris’s entry into the race, I’m not surprised that there wasn’t a large convention “bounce.” I don’t know if that is a good or a bad thing.
2. Looking at https://polymarket.com/elections it’s worth noting that his model currently has Harris at a 69% chance of winning the popular vote. If Silver’s right this will mark the third time this new century that we’ll have a popular vote and electoral vote split (three of the last seven Presidential elections).
3. I am genuinely interested in if/how Silver’s model accounts for ground game and party infrastructure. In a GOTV election, that definitely makes a difference.
4. Looking at 538 in 2020 https://projects.fivethirtyeight.com/2020-election-forecast/ (when Silver was still there), this same period in September marked a moment when the race also tightened (though the model still predicted Biden). Looking specifically at one swing state (Georgia – https://projects.fivethirtyeight.com/2020-election-forecast/) you’ll also see that Trump was, at this time, given a 68% chance of winning. So even in his old model, there was definitely room for movement (Biden only emerged a the leader in late October).
@Steven L. Taylor:
I think more than a little bit of that can be credited to his time as an Undergraduate at University of Chicago. Speaking from experience, “the place where fun goes to die” has a bit of a track record of producing this type of reaction.
https://mstdn.social/@maxkennerly/113102055979309410
@Steven L. Taylor: Yes, I agree that the need to drive clicks is a legitimate concern. He could do that all cycle and then adjust in the last week to get the ultimate prediction right.
I am not a statistics nerd, but when the New 538 head, G. Elliot Morris, was at the Economist doing his election nerding I found his voice interesting and his criticism of Nate Silver remarkably calm, rational and, to me, persuasive. Similarly his comments on his model at the Economist seemed logical.
We will see now that 538 is his baby how it turns out this election but I would not immediately discount 538 just because Nate Silver took Silver’s 538 model and G. Elliot Morris has created an updated version of Morris’ to bring to 538.
Thanks for this analysis. I can’t blame him for (seemingly) going in the pundit direction – hey, dude’s gotta make a living, right?
OTOH, many thanks for reminding everyone of the prescient genius of Mr. Z. Now that that earwig’s stuck in my head, time to dig the rest of the lexicon out. Followed by a collection of Zappa and Lehrer.
Last minute ETA. The stacks of his book piled up on the shelves seems cheesy and hucksterish to me, but I’m neither a successful author nor a successful savant, so it’s probably just sour grapes on my part.
While Silver has produced impressive results in the past, the fact that his model isn’t public means that we don’t know how it’s been adjusted. And given that his income now is heavily dependent on his hit rate, it’s fair to wonder if he has adjusted it to improve that rather than his accuracy.
Another consideration is that he considers himself a professional poker player. From what I’ve seen there are only three types of this: the person that runs the house and takes a cut from every pot but doesn’t otherwise participate; someone who is losing big money over time and needs income from somewhere else to compensate: someone who is essentially running a con on suckers. This latter avoids other skilled players or at most works with them to lure the suckers into thinking they are close to winning so they will stay in the game longer so as to lose more. He is obviously not the first so he must be one of the other two.
@MarkedMan:
Yup. David Karpf, GWU political professor and poker player, did an extended thread of Silver’s latest book on Bluesky and Nate’s use of poker and venture capitalists as models for behavior. I recommend it but the tl;dr version is that Nate has seemingly gotten high on his own supply and his fascination (and addiction?) to gambling is distorting his views.
To MarkedMan’s point, here is a 5-post excerpt:
TBH, I liked this bit to underscore the shallowness of Nate’s thinking:
@MarkedMan:
@SKI!:
Ayup. Grandma (who taught me math with a deck of cards) ran card rooms for the old Asian gentlemen in the 30s-40s . She insisted that the only way to successfully gamble was to shear the sheep. From her stories, she loved having know-it-alls like him at the table.
@MarkedMan:
Because much of election modeling is fortune cookie punditry, but the business model depends on panicky Democrats and other poll fetishists believing Silver and Co. are doing objective math.
Impressive? Depends. Results-wise, meh. Silver nailed one election a dozen years ago. Since then, the conventional wisdom shaped by Silver and others like him has been 50/50 at best in forecasting outcomes.
Their polling-modeling-punditry consensus was off in 2016. In 2020, it generally missed Biden’s Sun Belt surprises and Rethuglikkklans’ down-ballot resilience. Didn’t see Dems sweeping the Jan 5, 2021 Georgia Senate runoffs—oopsy daisy. And promises of “Red Wave 2022!!11!!” were a notorious belly flop. Post-2022, Dem strength in races has been wildly underestimated.
A lot of people make bank pretending they can predict the future; the past decade of actual election results shows polling and modeling are not so more reliable than astrology, a Magic 8-Ball, or a coin toss.
But the click-generating hustle is definitely impressive. If people want to throw money at Nate Silver and the rest for their guesswork and statistical model hocus pocus, good for them.
It’s a clever scam. On Election Eve, tweak your model’s assumptions to land on “This person has a 53% chance, that one 47%.” Then when you miss the mark: “You fools! I didn’t say 0%. You just don’t understand probability!” So no matter what, you’re always right. Clever.
tl;dr: Politics junkies have a tough time sitting with uncertainty and are desperately reading tea leaves, trying to figure out now what’s going happen in November. No earthly being knows.
I was a Nate Silver defender for a long time, against liberals who disparaged him. But his faults became too hard to deny in 2022, when his averages–which ended up strongly underestimating the performance of Dems–were filled with questionable pollsters. When criticized about it before the election, his response was that if Dems were concerned about Republican-aligned pollsters flooding the averages, then they should produce their own to counter it.
He also defended Trafalgar as one of the most accurate pollsters of recent times–a conclusion that requires myopia of the highest order. In 2020 they had Trump leading in five states that Biden went on to win. In any case, the one consistent thing about Trafalgar is that they always produce results more favorable to Republicans than most other pollsters. When a pollster behaves that way, there will invariably be cycles in which it appears more accurate than other pollsters, simply if polls in general are off in the other direction (as was the case in 2020). It doesn’t imply Trafalgar has any special insight; at best it’s a case of a broken clock being right twice a day.
Silver’s main problem is that he’s too reliant on polls altogether in his analysis, as polls are not the be-all, end-all of making a sensible prediction. I’m not one of those people who claim the entire polling enterprise is broken. Even the polling misses in recent years have been overstated in popular imagination–relatively small errors overall that look worse because the elections are decided by tiny margins. But there are still reasons why these errors happen, and Silver has a bad habit of not considering other factors (such as what’s happening on the ground) that might sway it one way or the other. It’s why, for example, his 2020 averages showed Biden doing better in Florida than Georgia, whereas some other analysts (such as Larry Sabato) correctly predicted Biden would win GA but lose FL.
“given that being wildly wrong on this election would presumably be the end of his career as a political odds savant.”
Seriously? When was the last time any right-leaning pundit found his career ended by being wrong, no matter how wrong, no matter how many times?
@Kylopod:
The mind boggles. That isn’t the answer of someone who is actually trying to be predictive or accurate.
@Matt Bernius:
How pray tell are you judging “Silver being right” when he presents a probabalistic outcomes.
The problem here is the general innumeracy of virtually everyone in understanding prabilistic analysis and treating probability outcomes evaluation as “predictions” in a rather on/off fashion. Of course this is a general human cognition fault…
Well given the cliquish partisan tribalism here, I have zero expectations of anything other than rather thin gruel of logical fallacy filled ad hominem attacks on Silver for not being a cheerleader for Own Tribe and for the sin of having a model that warns of a high danger of a disappointing outcome (and puts specific probalistic weighting that right or wrong has the merit of being geneally explicit).
But now Mr Silver is Not on The Team so…
@SKI!: I had a friend who had a seat on the Chicago Board of Trade in the 80’s and 90’s. Made tremendous amounts of money (which as near as I could tell, he put all up his nose). He visited me one weekend when I lived in New Orleans and in between sets at various Blues clubs as we bar hopped around town, I grilled him on what he actually did, and how he made money. He told story after story about getting one over on other traders. And finally, several hours in, I said, “Jerry, I just don’t get it. You’re describing a zero sum game. If one trader is up, another trader is down. But you’ve all been there for years, so how come you’re all not broke?” “Oh”, he said. “Sometimes I get one of them and it’s great. And sometimes they get me and I immediately start thinking about getting them back. But we make our money from all the idiot amateurs who thinks they can beat us. They don’t have a chance.”
@Lounsbury:
I’m not sure why you think pointing out the inherently unfalsifiable nature of Silver’s model is a point in its favor…
@Lounsbury:
Awwww, poor innocent martyr Nate Silver. Such a victim of those who are not cheerleaders for perpetually wrong punditry. Waaaaaaaa cry waaaaaaa. How will he and his self-satisfied performatively nonpartisan fanboys ever survive these earth-shattering attacks? Poor babies.
Anyway…
@Steven L. Taylor:
Nate Silver has, over several years, started developing a bad case of Scott Adams disease: having become a celebrity he has begun to imagine himself a “Great Man” and is increasingly upset he doesn’t receive the obedience he believes that being a “Great Man” entitles him to. This often leaves the sufferer prone to right-wing radicalization as right wing extremists are often quiet happy to cater to that sense of entitlement.
Now Silver’s radicalization is relatively minimal at this point, but his persona definitely seems to be trending in that direction…
Polls offer nothing but fodder for “news” and pundits to generate clickbait heat. Direction does give some indication, but seemingly only that they all are reverting to the middle as their starting point for the last 60 days of the election. All we can say is the Harris bump was pumped up but is now dissipating. That is not good for momentum pieces the “news” wants to write.
What remains is whether the Democrat-aligned media can do a bit of CPR on the numbers after the jolt of the debate and still hope stay close enough not to be wiped out by the actual election outcome. After all, the “Vibes” campaign is about getting people to vote with the crowd rather than with unemotional thought. Hence the assault on Silver for daring a dissenting opinion on the polling data.
@Lounsbury:
Ok, I get what you are saying, and I also think we are hitting the challenge of discussing prediction using easily parsable language.
You are entirely right that as in 2016, Silver is laying out probabilities. And that in 51% of those probabilities Trump wins and in 49% Harris wins. And that no one knows which of the many potential models will match whatever happens later this year.
At the same time, for better or worse Silver has gained a lot of cultural capital for “accurately predicting the outcome of elections.” To my knowledge, I have not seen Silver ever suggest when things went well (see Obama for example) that his model didn’t actually suggest who would win the election. It’s typically when the model “failed,” see Clinton, that he chooses to advance the idea that the model wasn’t wrong, it just a different scenario happened.
I am sure you will think this is a dodge, but it seems to me that Mr Silver tends to want to (understandably) speak out of both sides of his mouth.
@Matt Bernius:
OK, I’ll step up and defend at least past iterations of Nate Silver. In some Presidential election 538 got every state right (nearly every state?) and he published a column pointing out that this was problematic for the model. Given the odds they gave in each state they should have netted some bad predictions. He seemed genuinely puzzled as to what had happened and how they should adjust their model. They got too any right to just be lucky.
Modern day Nate I don’t follow closely enough to judge.
@MarkedMan:
TY for the correction! I really appreciate it. I will try to find that note.
@MarkedMan: I don’t see how it was problematic that the model got every state right. Surprising would be the better term, IMO. Let’s say you have ten states, and in each Candidate X has a 51% probability of winning based on the model. If Candidate X won all ten states it would be extremely surprising. It still would be the single most likely scenario of all unique scenarios. The thing to remember is that there are ten scenarios in which Candidate X wins nine states and only one scenario in which Candidate X wins all ten states. I don’t see extreme accuracy as a problem with the model.
@Matt Bernius: The fact that most you do not understand in reality (see “predicting election” you just used, showing exactly that) probabilistic analysis is not the fault of Silver – he rather does try again and again to highlight what probabilistic analysis does. How you read him…. well the untrained will focus on the an English discourse and treat him as Predictions like an Oracle.
(I shall not claim myself to be particularly good at such myself, however years of graduate training in such in context of econometrics, at least made me aware of the ease of misunderstanding probabilities and the ease at which one slips into what is in the end default processing, erroneous but default)
In the end his sin for most of you is in reality not being a Cheerleader for the Team and thus provoking the tribal response.
In any case the recent polling from multiple sources rather indicate that the honeymon is over (predictably so and what Harris always had to face).
Conspiracy mongering about Russian influence in the face of data is sterile and inutile.
@DK: your typical empty cliquish ad hominem reaction, rather continues to show your superifciality.
Innumerate misunderstandings harm yourselves not Mr Silver. That you think this is poor Mr Silver rather than “you poor self-deceiving sods engaging in rather too standard tribal clique reaction” not particulalry different than the infamous “unskewed polls” mate some years ago. The ancient Shoot the Messenger reaction.
@Lucysfootball: His point was that if you predict 10 races as 60-40, that means that you would expect 6 of them to match and 4 to not. If you have 10 out of 10 you are either extremely lucky or your model is not reflecting the certainty of the outcomes accurately. By the time you get to 50 out of 50 the odds of just being luck are vanishingly small.
@Lounsbury:
Ok, I would love you to unpack how we should talk about what Nate Silver does. That isn’t me being a jerk FWIW. I would love to get a perspective on how you think we should talk about his work, because I literally (correctly used here) don’t know what you are judging is against.
I will say my intent isn’t to suggest he is an oracle. It’s to highlight that he is giving us a glimpse into one model of how things might play out based on existing data.
Again this is a serious request to help me/us understand your perspective, not any form of “gotcha”.
@Lounsbury:
Ok, I revisited what I wrote and I am trying to refine my question to you. I wrote (and you took execption to:
My understanding is that his model is that in 69% of scenarios, Harris could (not will) win the popular vote. Which leaves at least 31% of scenarios where Trump could win the popular vote.
I would love you to unpack what that analysis is getting wrong. Like I have said elsewhere, I am a qualitative researcher. So I would really appreciate understanding how someone with a more quantitative take would interrupt that.
Alternatively if you have suggestions for resources I should read to better understand probabilistic analysis I would really appreciate the pointers
Silver is suffering from the Cassandra Backlash. When everyone else said Clinton was a sure thing in 2016, he said Trump had a good chance of winning. Dems/Lefties have never forgiven him for being right about that.
@Lounsbury: Ad hominem? Lol, the hypocrisy of it all. Bruh. You are a predictable broken clock who responds to any point you disagree by immediately labeling your targetes partisans, bobo leftists, tribalists, blah blah blah. If you are unaware this is textbook ad hominem, you are just not smart.
Everyone already knows how most of your comments will read. A noun, a verb + something something partisan/clique/the team/tribal. You never have anything new or original to say. Because you don’t have the depth to form any argument more serious than repetitive, self-indulgent performance of nonpartisanship for its own sake — a unpersuasive logical fallacy reliant on an unproven assumption that fence-straddling is automatically preferable.
Your comments reveal a shallow, intellectually lazy one-trick pony. That’s why you can’t address the real reasons people question Nate Silver and mainstream media narratives — their underwhelming track record — to instead retreat robotically back to your boring grab bag of facile whining about partisanship. You are just not capable of thinking crtically enough to engage in good faith.
@Lounsbury: In the case you’re citing the logical way to read “if Silver is…” would be “if Silver turns out to be…”. There’s a grammar convention about what other roles “to be” plays in English, but I haven’t worked with such grammar in more than a decade, so I can’t recall what this convention is off the top of my head. Either way, the statement isn’t declarative; it’s conditional.
ETA: My simply slow waking from an afternoon nap mind suddenly reminded me that even the “if” in the sentence marks a non-declarative situation. Nobody is saying Silver is right about anything. Not even probably or probability-ly.
@Hal 10000:
That’s one of the reasons why I was defending him for so long. The people who are dismissive of his giving Trump a 30% chance of winning are overlooking the fact that he was one of the very few people in the media giving Trump a nontrivial chance of winning at all, and he was attacked by rival analysts like Sam Wang and HuffPost for deliberately cooking the numbers to make the race seem more competitive.
Now, I never was a big fan of the use of probability estimates based on polls, because they’re essentially unfalsifiable (one could argue that Sam Wang’s 99+% for Hillary wasn’t wrong, Trump’s victory was simply that rare <1% chance event). But I did generally think his polling averages/aggregators were among the better ones around–definitely more reliable than the right-slanted RCP.
@DK: pretty spot on there, from what I’ve seen
(I mean your comment, not berry’s)
Chances that Silver is aligned with Thiel and Vance in views about women and it carries over to his results and opinions about elections? Thiel called women voting in a capitalistic democracy, an oxymoron. Silver was interviewed recently at The Hill. He was championing Josh Shapiro (rejected by Harris campaign). Shapiro was the only Democratic presidential candidate who supported the education vouchers cited in Project 2025. Reportedly, 80% of private school parents choose religious schools. Right wing religion has always elevated men to power and subjugated women.
@L.L.:
Silver dismissed critics saying they didn’t know how start-ups begin.Vance was involved in the creation of the start-up, Hallow, a Catholic prayer app. Vance converted to Catholicism in timing similar to his political ambitions (2019).
@Hal 10000: “Silver is suffering from the Cassandra Backlash. When everyone else said Clinton was a sure thing in 2016, he said Trump had a good chance of winning. Dems/Lefties have never forgiven him for being right about that.”
Hand waving: “they’re just resentful”.