this post was submitted on 04 Jul 2023
371 points (95.6% liked)
Asklemmy
43945 readers
874 users here now
A loosely moderated place to ask open-ended questions
Search asklemmy ๐
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- !lemmy411@lemmy.ca: a community for finding communities
~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
When you think about data it actually gets really scary really quick. I have a Master's in Data Analytics.
First, data is "collected."
So, a natural question is "Who are they collecting data from?"
Typically it's a sample of a population - meant to be representative of that population, which is nice and all.
But if you dig deeper you have to ask "Who is taking time out of their day to answer questions?" "How are they asked?" "Why haven't I ever been asked?" "Would I even want to give up my time to respond to a question from a stranger?"
So then who is being asked? And perhaps more importantly, who has time to answer?
Spoiler alert: typically it's people who think their opinions are very important. Do you know people like that? Would you trust the things they claim are facts?
Do the data collectors know what demographic an answer represents? An important part of data collection is anonymity - knowing certain things about the answerer could skew the data.
Are you being represented in the "data"? Would you even know if you were or weren't?
And what happens if respondents lie? Would the data collector have any idea?
And that's just collecting the data, the first step in the process of collecting data, extracting information, and creating knowledge.
Next is "cleaning" the data.
When data is collected it's messy.
There are some data points that are just deleted. For instance, something considered an outlier. And they have an equation for this, and this equation as well as the outliers it identifies should be analyzed constantly. Are they?
How is the data being cleaned? How much will it change the answers?
Between what systems is the data transferred? Are they state-of-the-art or some legacy system that no one currently alive understands?
Do the people analyzing the data know how this works?
So then, after the data is put through many unknown processes, you're left with a set of data to analyze.
How is it being analyzed? Is the analyzer creating the methodology for analysis for every new set of data or are they running it through a system that someone else built eons ago?
How often are these models audited? You'd need a group of people that understand the code as well as the data as well as the model as well as the transitional nature of the data.
Then you have outside forces, and this might be scariest of all.
The best way to describe this is to tell a story: In the 2016 presidential race, Hillary Clinton and Donald Trump were the top candidates for the Democratic and Republican parties. There was a lot of tension, but basically everyone on the left could not fathom people voting for Trump. (In 2023 this seems outrageous, but it was a real blind spot at the time).
All media outlets were predicting a landslide victory for Clinton. But then, as we all know I'm sure, the unbelievable happened: Trump won the electoral college. Why didn't the data predict that?
It turns out one big element was purposeful skewing of the results. There was such a media outrage about Trump that no one wanted to be the source that predicted a Trump victory for fear of being labeled a Trump supporter or Q-Anon fear-monger, so a lot of them just changed the results.
Let me say that again, they changed their own findings on purpose for fear of what would happen to them. And because of this lack of reporting real results, a lot of people that probably would've voted for Clinton, didn't go to the polls.
And then, if you can believe it, the same thing happened in 2020. Even though Biden ultimately won, the predicted stats were way wrong. Again, according to the data Biden should have been comfortably able to defeat Trump, but it was one of the closest presidential races in history. In fact, many believe, if not for Covid, Trump would have won. And this, at least a little, contributed to the capital riots.
Nate Silver was singing a different tune, though. I remember an interview he gave a month out from the election where he noted significant softness in support for Clinton. There were also a lot of undecideds who might swing elections in key states. That is, of course, exactly what happened. When the Comey letter was leaked by Congress, it likely cost Clinton the election. Her poll numbers dropped from +7% to +3%, well within the advantage that the Electoral College gives to Republicans.
On Election Day, the 538 model was about 3:1 in favor of Clinton. That sounds highly in favor of Clinton, and it is. But it still leaves plenty of room for a Trump win. And lo and behold, she lost.
One other thing polls didn't really capture was voter enthusiasm or maybe not enough people was paying attention to it. Just because you answered Hilary when asked who would you vote for, it didn't mean you went out on Election day to vote. A combination of lack of enthusiasm for Hilary, coupled with news constantly reporting that it will be a landslide kept many Democrat voters home.
I believe that's why there's such a huge push for "get out the vote" campaigns in 2020 by the Democrats. Generally, the more people voting means better chances for a Democrat win, given general (non-electoral) election results.
That's interesting, I did not think the letter had that big of an impact.
For me it was Bernie. I remember a lot of us on Reddit were all about Bernie.
Iirc, Bernie had a lot of steam and it seemed like again Clinton was going to be pushed aside for a grass-roots candidate (just like with Barak years earlier).
And Bernie said he was not going to give up the race, because even if he didn't win the votes he could still be voted in at the national convention.
And as the DNC neared, things were looking great. Clinton was giving paid speeches to wall street and Bernie was tearing her whole campaign apart because he was saying, give money back to people and she was saying keep things the way they were.
And then, among mounting pressure, two weeks before the convention he concieded out of nowhere. At least that's what it seemed like to us.
Then emails leaked that showed the Democratic Party had colluded with Clinton to get Bernie out of the race!
We couldn't believe it. We were devestated. So some people went to the DNC and were making a big stir, demanding that Bernie get back on the ballot.
And it all came to a waterfall moment when Sarah Silverman was on stage. And people were chanting Bernie and she lost it and told everybody to shut up and said the Bernie supporters were stupid.
And that was it. The only thing that came out of it was somebody got fired, but there was no regard or representation for us in the Democratic Party anymore.
They didn't care about what we wanted, and they were just as crooked as they had always told us the Republicans were.
For me it was a massive dissolutionment, and drove me to Trump. Since he was saying we need to take our economy back from the 1%.
Do you have a source for the outlets changing their poll results? I did a search myself but couldn't find anything. I find that very interesting and wanted to read more!
It was in an article on Hacker News around that time. It was super interesting, but I can't find it atm, I'll look around tomorrow.
Oh yeah. I might say some wrong stuff since I'm quite ignorant but. Statistics is messy and I tend to avoid including too much stats in my projects, although sometimes I accidentally end up blindly doing so and believing them also drawing inaccurate conclusions. Physical stats are even messier because not everybody has the competence to accurately understand what they mean, or sometimes we just don't understand the world enough. Environmental science data is an example of that. I rely on other people's analyses cause I can't read them. I don't know much about politics.