Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are in Science and Inquiry

Jan 20, 2018 David Rubenstein rated it really liked it

Shelves: business, audiobook, computers

This is an engaging book about how big data can be used to improve our understanding of human behavior, thinking, emotions, and preference. The basic idea is that if you ask people about their behavior or their preferences in surveys, even anonymous surveys, they will often lie. People do not like to admit to low-brow preferences; racists do not want to admit to their prejudices, most people who watch pornography do not want to admit to it, and even voting is often misrepresented; some people wh This is an engaging book about how big data can be used to improve our understanding of human behavior, thinking, emotions, and preference. The basic idea is that if you ask people about their behavior or their preferences in surveys, even anonymous surveys, they will often lie. People do not like to admit to low-brow preferences; racists do not want to admit to their prejudices, most people who watch pornography do not want to admit to it, and even voting is often misrepresented; some people who voted for Trump would not admit to it.

But, by analyzing immense datasets from Google, public archives, social media, and the like, Seth Stephens-Davidowitz has been able to unearth a lot of fascinating answers to puzzling questions. For example, he is able to predict, through Google searches for various symptoms, who is likely to have early stages of pancreatic cancer. He can predict epidemic breakouts of some contagious diseases well before they are announced by the CDC (Center for Disease Control). He shows that the single factor that correlates with voting for Trump is that of racism.

Then there are the fun factoids, about the sorts of things that people search for most often on Google. Most commonly, the search "Is my son ..." is followed by "gifted", while the search "Is my daughter ..." is followed by "overweight". That tells us something about stereotypes for the way people think about their children. Interestingly, the release of a new violent movie in a city is correlated with a decrease in violent crime in that city. Perhaps the reason is that violent people who are watching the movie are not out on the streets, committing crimes.

And here we get to the main problem with this sort of analysis. Undoubtedly, the research and analysis of big datasets is done correctly. However, once a surprising result is found, understanding the motivations behind the online activity are often subjective and open to interpretation. While this book is very careful about its underlying assumptions, it is a slippery road to getting the correct interpretations and explanations.

This is an easy, well-paced book that should appeal to anybody who enjoys books like Freakonomics: A Rogue Economist Explores the Hidden Side of Everything. ...more

80 likes · Like · see review

Apr 17, 2018 Jim rated it it was amazing · review of another edition

Shelves: science, 2non-fiction, 1audio

I am now convinced that Google searches are the most important data set ever collected on the human psyche. writes the author early on & he shows why. (Google trends is available to all here: https://trends.google.com/trends/) He also checked other big data sets including Wikipedia, Facebook, Pornhub, & even Stormfront, the largest racist site. What he found was really interesting & it will help harden the soft, social sciences. It's a new frontier.

He points out problems with traditional report I am now convinced that Google searches are the most important data set ever collected on the human psyche. writes the author early on & he shows why. (Google trends is available to all here: https://trends.google.com/trends/) He also checked other big data sets including Wikipedia, Facebook, Pornhub, & even Stormfront, the largest racist site. What he found was really interesting & it will help harden the soft, social sciences. It's a new frontier.

He points out problems with traditional reporting. In the section about child abuse & abortions, Google searches suggest that child abuse does increase during economic downturns while gov't figures incorrectly show little change. Closing abortion clinics doesn't stop them, it simply leads to more self-induced abortions. Both happen off the books, but there is now convincing supporting data to show us what we need to address & make more informed decisions with resources.

Big data has an advantage over every other type of survey because few realize it is being collected, so we don't lie to make ourselves look better. It's also anonymous & aggregate, so caution needs to be used when forming conclusions. For instance, based on Pornhub searches, the author concludes that about 5% of men are gay because they searched for gay porn. That seemed a reasonable conclusion until he pointed out that 15% of women search for rape porn. Does that mean they want to be raped? The author says of course not & makes a big deal out of the difference between fantasy & reality. That makes me question his first conclusion, although it seems about right.

Gut reactions are often wrong & he provides several examples where it's wrong due to cognitive biases. He also points out "The Curse of Dimensionality". Given large enough sets of data, there will be correlations just through chance. For instance, there are graphs that show how closely autism diagnoses track with organic food sales or Jenny McCarthy's popularity. Separating these out is a whole other problem.

Big Data only gives us trends that we need to examine. We can't use it on the individual level. While 1000 people searched for how to kill their girl friend, only 1 girl was killed in his example. That's horrific & might have been stopped if someone had looked at his search history, but do we give up everyone's privacy for a 1 in 1000 chance that we might prevent a murder? Some might be willing, but I'm not, so we also have new questions to address.

The audio book was well narrated & I didn't miss the graphs too much. They're provided in the extra material, but weren't handy when I was listening & the book took that into account for the most part. Highly recommended in either format. ...more

33 likes · Like · see review

Jan 20, 2018 Charlene rated it really liked it

Shelves: innovation, decision-making, constructs

There are so many things to love about this book. Not the least of which is that it focused largely on how big data would act like a truth serum and replace terrible self report findings when trying to answers myriad questions that arise in all areas of life. I say bravo to that!

However, just because you identify a problem with one measurement method (self-report), it does not necessarily mean you have found the fix. Does big data sound extremely promising? Hell yes. In fact, i think when we le There are so many things to love about this book. Not the least of which is that it focused largely on how big data would act like a truth serum and replace terrible self report findings when trying to answers myriad questions that arise in all areas of life. I say bravo to that!

However, just because you identify a problem with one measurement method (self-report), it does not necessarily mean you have found the fix. Does big data sound extremely promising? Hell yes. In fact, i think when we learn how to escape the many pitfalls of trying to collect big data to represent an accurate picture (those are some humongous pitfalls that have not been adequately addressed in this book), big data might be our best bet at understanding human nature. Some aspects of big data already seem solid and trustworthy. For example, people indicate they will watch a certain type of movie on Netflix-- perhaps a documentary, an intellectual film, or the like-- but they watch a mindless comedy instead and often. No matter how many of one type of movie a person puts in their Netflix queue, their actual watching habits are far more reliable at predicting what types of movies they like to watch than what they have chosen, themselves, to put in their own queue.

There are plenty of other examples scattered throughout the book that are similarly compelling. You will find excellent data mining that shed light on things from the election of Trump to the things people really worry about but won't admit. Data helped Nate Silver figure out how Trump won the 2016 primary. It turned out that Trump won in areas that had googled the word "nigger" most often in the year leading up to the primary. Controlling for other factors, this is an extremely disturbing finding. There were other really interesting examples of what people search for in different areas of the USA and different areas of the world that give us an extremely candid picture of what they might be thinking. I say might because it is entirely possible that people who are concerned about the things their neighbors say or do, might also look those things up. The effect might be smaller than it seems. Nevertheless, all of these searches are a valuable window at least into what people are thinking about. And speaking about what people are thinking about, it seems that the main concern for men is how big their penis is. For women it is if their vagina smells bad. It seems to be a downright preoccupation.

As much as I loved this book, and I have to say there were many parts that I really, really loved, I was sorely disappointed at times. The author is certainly more critical than most in his thinking, and takes pains to convince the reader as such. However, if you are going to write a book about how to think critically, you should probably be in the top 1% of critical thinkers. Chapter Five tells me that probably is not the case. He cited experts' concerns that violence in movies cause violent behaviors and then cited a study that had serious fundamental flaws. The study sought (I use the term sought lightly) to find out if violent movies caused violent actions. Turned out that on weekends that violent movies were shown, violence went down, not up. On weekends with non-violent movies, violence went up. Mind boggling! or so the author claims. Not really. It's easy to imagine many, many reasons why this might be. The authors of the study concluded, using absolute conjecture) is that it keeps violent people occupied. I will buy that……. in the short term. But, that doesn’t prove whether violent movies cause or do not cause violent acts. Violent acts can sometimes take a long time to cultivate. You cannot measure them by a weekend. This study was not a lone lapse in critical thinking, and that bothered me because this book is about critical thinking. Overall though, I have to say the great parts outweighed the bothersome parts. I would recommend.

...more

11 likes · Like · see review