What do you think?
Rate this book
Bayes' Rule: A Tutorial Introduction to Bayesian Analysis
What does a medical test tell us about the chances of having a particular disease?
How can we tell if a spoken phrase is, 'four candles’ or 'fork handles’?
How do we a perceive a three-dimensional world from from the two-dimensional images on our retinas?
The short answer is Bayes’ rule, which transforms meaningless statistics and raw data into useful information. Discovered by an 18th century mathematician and preacher, Bayes' rule is a cornerstone of modern probability theory. In this richly illustrated book, intuitive visual representations of real-world examples are used to show how Bayes' rule is actually a form of common sense reasoning. The tutorial style of writing, combined with a comprehensive glossary, makes this an ideal primer for novices who wish to gain an intuitive understanding of Bayesian analysis. As an aid to understanding, online computer code (in MatLab, Python and R) reproduces key numerical results and diagrams. Stone’s book is renowned for its visually engaging style of presentation, which stems from teaching Bayes’ rule to psychology students for over 10 years as a university lecturer.
How can we tell if a spoken phrase is, 'four candles’ or 'fork handles’?
How do we a perceive a three-dimensional world from from the two-dimensional images on our retinas?
The short answer is Bayes’ rule, which transforms meaningless statistics and raw data into useful information. Discovered by an 18th century mathematician and preacher, Bayes' rule is a cornerstone of modern probability theory. In this richly illustrated book, intuitive visual representations of real-world examples are used to show how Bayes' rule is actually a form of common sense reasoning. The tutorial style of writing, combined with a comprehensive glossary, makes this an ideal primer for novices who wish to gain an intuitive understanding of Bayesian analysis. As an aid to understanding, online computer code (in MatLab, Python and R) reproduces key numerical results and diagrams. Stone’s book is renowned for its visually engaging style of presentation, which stems from teaching Bayes’ rule to psychology students for over 10 years as a university lecturer.
174 pages, Paperback
First published June 4, 2013
17 people are currently reading
258 people want to read
About the author
James V. Stone
23 books34 followersHonorary Associate Professor, University of Sheffield, England.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 16 of 16 reviews
June 22, 2016
Of all the areas of mathematics, probability is arguably the most intriguing to the non-mathematician, and this is particularly the case with Bayesian analysis, which can be delightfully counter-intuitive. However, the more complex aspects can be tricky to get your head around, so I was delighted to have the chance to read this book, subtitled 'a tutorial introduction to Bayesian analysis.'
I need to say straight away that this isn't really a popular science title, and the author is very clear about this - it's a kind of textbook lite - but if you have found out a bit about Bayes this book is an opportunity to dive into it a little deeper without taking on the full rigour of a textbook approach. Why should you care? Bayes gives us a mechanism that enables us to do things like go from a known piece of information like 'what's the probability of a symptom given a disease' to estimate a much more interesting unknown like 'what's the probability of the disease given a symptom' - an extremely powerful mechanism.
James Stone does his best to accommodate us ordinary folk. The book opens well, apart from a bizarrely heavy smattering of references on page 1, with a gentle introduction, and keeps the mood light after the classic disease application by looking for a mechanism of determining whether some said 'four candles' or 'fork handles' in the Two Ronnies style. If you are prepared to make an effort, for most of us probably a considerable effort, you will go on to pick up a lot more about using Bayes than you already knew (if you aren't a mathematician).
It is rather unfortunate for the general reader, though, that the book obeys the rules of the textbook rather than a popular science exposition. This comes across in unnecessary use of terminology - defining things that, frankly we don't need to know - and in rapidly moving to using symbols in equations, where they are rarely necessary at this level and all they do is put readers off. I suspect the moment that Stone introduced the Greek letter theta (θ) he made things ten times harder - unless you do this kind of thing every day, suddenly the text gets far less readable - the eyes bounce off it.
Even though I enjoyed the fork handles, I also thought the choice of examples could have been better. It was okay to use disease and symptom once, as it's an important application, but most of us rarely have to deal with this kind of situation and it would have been better to use more personally relevant applications. It was also unfortunate that when explaining random variables Stone chose a coin which is 90% likely to be heads and 10% likely to be tails - there is too much baggage attached to coins being 50:50. It would have been less confusing to have something that we might encounter (a scratch card, say) that is likely to be one value 90% of the time and the other 10%.
If you make it to the final chapter you are rewarded with a very readable, if too brief, introduction to the distinction between Bayesian and frequentist approaches, and just a touch of the mind bending capabilities of Bayesian thinking. With a bit more of this contextual material throughout the experience would have been gentler and more enjoyable - but even as a closer to the book it provides interesting material.
Don't expect, then that this book will make fun, popular science bedtime reading. It's not that kind of exercise. However, if you are prepared to overcome the onslaught of thetas and don't mind reading some statements several times to get what's being said, it is an excellent way to expand a vague understanding into a more sound knowledge of the basic mechanics of Bayesian analysis.
I need to say straight away that this isn't really a popular science title, and the author is very clear about this - it's a kind of textbook lite - but if you have found out a bit about Bayes this book is an opportunity to dive into it a little deeper without taking on the full rigour of a textbook approach. Why should you care? Bayes gives us a mechanism that enables us to do things like go from a known piece of information like 'what's the probability of a symptom given a disease' to estimate a much more interesting unknown like 'what's the probability of the disease given a symptom' - an extremely powerful mechanism.
James Stone does his best to accommodate us ordinary folk. The book opens well, apart from a bizarrely heavy smattering of references on page 1, with a gentle introduction, and keeps the mood light after the classic disease application by looking for a mechanism of determining whether some said 'four candles' or 'fork handles' in the Two Ronnies style. If you are prepared to make an effort, for most of us probably a considerable effort, you will go on to pick up a lot more about using Bayes than you already knew (if you aren't a mathematician).
It is rather unfortunate for the general reader, though, that the book obeys the rules of the textbook rather than a popular science exposition. This comes across in unnecessary use of terminology - defining things that, frankly we don't need to know - and in rapidly moving to using symbols in equations, where they are rarely necessary at this level and all they do is put readers off. I suspect the moment that Stone introduced the Greek letter theta (θ) he made things ten times harder - unless you do this kind of thing every day, suddenly the text gets far less readable - the eyes bounce off it.
Even though I enjoyed the fork handles, I also thought the choice of examples could have been better. It was okay to use disease and symptom once, as it's an important application, but most of us rarely have to deal with this kind of situation and it would have been better to use more personally relevant applications. It was also unfortunate that when explaining random variables Stone chose a coin which is 90% likely to be heads and 10% likely to be tails - there is too much baggage attached to coins being 50:50. It would have been less confusing to have something that we might encounter (a scratch card, say) that is likely to be one value 90% of the time and the other 10%.
If you make it to the final chapter you are rewarded with a very readable, if too brief, introduction to the distinction between Bayesian and frequentist approaches, and just a touch of the mind bending capabilities of Bayesian thinking. With a bit more of this contextual material throughout the experience would have been gentler and more enjoyable - but even as a closer to the book it provides interesting material.
Don't expect, then that this book will make fun, popular science bedtime reading. It's not that kind of exercise. However, if you are prepared to overcome the onslaught of thetas and don't mind reading some statements several times to get what's being said, it is an excellent way to expand a vague understanding into a more sound knowledge of the basic mechanics of Bayesian analysis.
August 14, 2016
This is a great introduction to the basic concepts of Bayesian probability theory. It's definitely not a textbook, but it's also not a pop-science mass-market book (e.g. Nate Silver's wonderful "The Signal and the Noise"). It is an excellent book for readers who want a thorough conceptual introduction to Bayesian probability and basic applications, but don't want to wade through rigorous proofs, programming code, or explanations of MCMC or Gibbs Sampling. It is an excellent book to read prior to some more detailed or rigorous books such as Gelman et al.'s "Bayesian Data Analysis", Bolstad's "Introduction to Bayesian Statistics", or Jaynes' "Probability Theory."
Repeatedly throughout the book, the author gives several lucid explanations and illustrations of basic Bayesian reasoning, chiefly focusing on the following elements:
1) Using probability as a measure of certainty about an unknown outcome or proposition (rather than considering probability to be only a measure of relative frequency)
2) Expressing one's prior beliefs about an uncertain outcome/proposition by encoding these beliefs as a probability distribution over outcomes/propositions (e.g. prior to examining a patient, a doctor assigning a 70% probability to the possibility that a patient has Chicken Pox and a 30% chance to the possibility that they have a common cold)
3) Describing the compatibility of potential data (e.g observed symptoms of a medical patient) with each of the possible values of a variable one is uncertain about (e.g. which disease the patient may have)
4) Using Bayes' rule to update one's prior beliefs from (2) by interpreting new data (e.g. a medical exam of a patient) in the light of how compatible the new data is with one's prior beliefs.
In other words, while the author discusses various techniques in Bayesian analysis (i.e. estimating means and proportions, predicting future outcomes, etc.) he never fails to explain how the techniques function in terms of the fundamental combination of 'subjective' probability, prior distributions, likelihood functions, and Bayes' rule.
My only complaint with the book has to do with the way it compares some Bayesian reasoning/techniques to Frequentist analogs. The book does not discuss confidence/credibility/uncertainty intervals (which are ubiquitous in statistics), but does compare certain Bayesian estimators (e.g. Maximum A Priori estimators) to Maximum Likelihood estimation and, while the author's discussion of the logic underlying these approaches is excellent, it seems somewhat unfair not to point out that maximum likelihood estimators are often not used because of the Maximum Likelihood logic but are used instead because the ML estimators happen to have desirable properties in the context of estimating confidence intervals.
In terms of math, the book uses some calculus to help more mathematically-advanced readers in certain places, but calculus is by no means necessary for one to get a lot out of this book. Prior knowledge of probability and statistics can help one get more out of this book, but is by no means necessary.
All in all, this is a very clear and highly readable introduction to Bayesian reasoning.
Repeatedly throughout the book, the author gives several lucid explanations and illustrations of basic Bayesian reasoning, chiefly focusing on the following elements:
1) Using probability as a measure of certainty about an unknown outcome or proposition (rather than considering probability to be only a measure of relative frequency)
2) Expressing one's prior beliefs about an uncertain outcome/proposition by encoding these beliefs as a probability distribution over outcomes/propositions (e.g. prior to examining a patient, a doctor assigning a 70% probability to the possibility that a patient has Chicken Pox and a 30% chance to the possibility that they have a common cold)
3) Describing the compatibility of potential data (e.g observed symptoms of a medical patient) with each of the possible values of a variable one is uncertain about (e.g. which disease the patient may have)
4) Using Bayes' rule to update one's prior beliefs from (2) by interpreting new data (e.g. a medical exam of a patient) in the light of how compatible the new data is with one's prior beliefs.
In other words, while the author discusses various techniques in Bayesian analysis (i.e. estimating means and proportions, predicting future outcomes, etc.) he never fails to explain how the techniques function in terms of the fundamental combination of 'subjective' probability, prior distributions, likelihood functions, and Bayes' rule.
My only complaint with the book has to do with the way it compares some Bayesian reasoning/techniques to Frequentist analogs. The book does not discuss confidence/credibility/uncertainty intervals (which are ubiquitous in statistics), but does compare certain Bayesian estimators (e.g. Maximum A Priori estimators) to Maximum Likelihood estimation and, while the author's discussion of the logic underlying these approaches is excellent, it seems somewhat unfair not to point out that maximum likelihood estimators are often not used because of the Maximum Likelihood logic but are used instead because the ML estimators happen to have desirable properties in the context of estimating confidence intervals.
In terms of math, the book uses some calculus to help more mathematically-advanced readers in certain places, but calculus is by no means necessary for one to get a lot out of this book. Prior knowledge of probability and statistics can help one get more out of this book, but is by no means necessary.
All in all, this is a very clear and highly readable introduction to Bayesian reasoning.
July 26, 2016
Review reproduced from my blog.
This is a decent little book by Stone, introducing the reverend Bayes’ seminal contribution to probability theory. It’s aimed at the complete novice, a little lower than I was hoping for personally, but useful nonetheless thanks to the plentiful real world examples. It’s a bit short; each chapter is around 10 pages with relatively large font, which makes the content easy to consume, but could have been just as well represented through a series of blog posts. Stone also makes it clear he wants to avoid jargon, but some of his explanations become opaque and obscure without it, and he often forgets his own rule and uses language without introducing it.
Would recommend to an undergrad just starting out with Bayesian Analysis, who is perhaps struggling with the conceptual understanding. But get it from a library, as it’s a little brief for the price.
This is a decent little book by Stone, introducing the reverend Bayes’ seminal contribution to probability theory. It’s aimed at the complete novice, a little lower than I was hoping for personally, but useful nonetheless thanks to the plentiful real world examples. It’s a bit short; each chapter is around 10 pages with relatively large font, which makes the content easy to consume, but could have been just as well represented through a series of blog posts. Stone also makes it clear he wants to avoid jargon, but some of his explanations become opaque and obscure without it, and he often forgets his own rule and uses language without introducing it.
Would recommend to an undergrad just starting out with Bayesian Analysis, who is perhaps struggling with the conceptual understanding. But get it from a library, as it’s a little brief for the price.
January 4, 2016
Well presented details of Bayes' rule, especially with regards to how a knowledge of prior probability can be used to calculate a conditional probability. In other words, the Bayesian probability includes expert knowledge as well as experimental data to produce probabilities. The expert knowledge is represented by some (subjective) prior probability distribution. The data is incorporated in a likelihood function. The product of the prior and the likelihood results in a posterior probability distribution that incorporates all the information known to date.
October 31, 2018
Bayes’ Rule: A Tutorial Introduction to Bayesian Analysis is a book that delivers what it promises on the cover. James V Stone really went out of his way to repetitively state and restate the theory and application of Bayes’ Rule. Along with tons of examples, there is also a lesson on re-framing questions to make them more applicable to what the person wants to know.
For instance, just for the sake of argument, let’s say that Smallpox wasn’t eradicated back in the 1970s and is still alive and well. If a patient comes into a doctor’s office and has spots or a rash on their person, what is the chance that they have Smallpox given the symptoms? We can use Bayesian Analysis to figure it out if we know those values. It’s actually pretty neat. Of course, you have to know the numbers.
The book has lots of pictures, plenty of equations, a MatLab program you can copy down and a lot of other things going for it. Despite its length, it does cover the subject pretty well, and it doesn’t meander too far into other ideas or situations. It talks about biased coins, the probability of something being said given a particular waveform, binomial distributions, and other such wonderful topics. In addition, the book explains the terms used really well, has a great glossary at the end and contains a bibliography and suggestions for further reading.
So if you are interested in this matter this book is really well done, although it does get a bit heavy-handed telling you where you can skip some portions.
For instance, just for the sake of argument, let’s say that Smallpox wasn’t eradicated back in the 1970s and is still alive and well. If a patient comes into a doctor’s office and has spots or a rash on their person, what is the chance that they have Smallpox given the symptoms? We can use Bayesian Analysis to figure it out if we know those values. It’s actually pretty neat. Of course, you have to know the numbers.
The book has lots of pictures, plenty of equations, a MatLab program you can copy down and a lot of other things going for it. Despite its length, it does cover the subject pretty well, and it doesn’t meander too far into other ideas or situations. It talks about biased coins, the probability of something being said given a particular waveform, binomial distributions, and other such wonderful topics. In addition, the book explains the terms used really well, has a great glossary at the end and contains a bibliography and suggestions for further reading.
So if you are interested in this matter this book is really well done, although it does get a bit heavy-handed telling you where you can skip some portions.
June 6, 2018
It's a very thin book but it took me a while to finish reading since it was something I brought along with me on my commute. I believe it imparted on me a solid understanding of the foundations of Bayes' Theorem. I really liked the extensive use of examples and how the chapters were broken down with most of the detail being in the first three chapters or so. Later on, the other chapters repeat the central idea. I wrote all over this book and took notes. Stone's use of diagrams, limited equations and appendix is incredibly helpful. I also enjoyed the history lessons, quotes and Sherlock Holmes thematic references. It's actually kind of a fun read when I didn't drag my feet about it!
June 19, 2019
Did not like it. The author pitches it as a bottom-up approach to subject in contrast to that of top-down of "the man". This is meant to be something that students really want. However most examples are simple and fragmented, failing to generalise into fundamental understanding of the subject. The chapter that I found most useful was towards the end of the book -- "A Bird's Eye View of Bayes' Rule" (and the code containe therein).
January 23, 2021
The book does a good job of explaining Bayes' rule but not in a starkly different way than other books on the subject. That said, its use of graphs/plots helps visually understand Bayes' rule; animated plots would have made the explanation even more accessible. As the book moves to topics such as estimation using Bayes' rule, the exposition gets a bit dry and inaccessible.
March 17, 2020
Very good introduction to Bayesian analysis, with rapid progression towards more advanced concepts. It takes extra effort to fully grasp the contents, but it is very rewarding in the end.
Currently reading
October 31, 2022I like it
February 28, 2024
my favorite part was when
February 6, 2023
Second Thoughts
For anyone needing an introduction (or a refresher) on Bayes' Theorem, Bayes Rule is a fine choice. Bayes' Theorem provides a mathematical way to update the probability of an event given new information. The book is comprised of seven brief chapters, discussing not only Bayes' Theorem, history, and application, but basic probability as well. Most of the nine separate Appendices at the end of the book (Glossary; Mathematical Symbols; The Rules of Probability; Probability Density Functions; The Binomial Distribution; The Gaussian distribution; Least-Squares Estimation; Reference Priors; and MatLab Code) are also quite valuable. The References section is good, though limited to books published up to 2012 owing to this volume's own 2013 publication date.
April 19, 2015
"[I]t can be shown that no other procedure can provide a better guess, so that Bayesian inference can be justifiably interpreted as the output of a perfect guessing machine, a perfect inference engine. ... The perfect inference engine is fallible, but it is provably less fallible than any other." (9)
"Far better an approximate answer to the right question ... than an exact answer to the wrong question." (quoting Tukey, 119)
"Far better an approximate answer to the right question ... than an exact answer to the wrong question." (quoting Tukey, 119)
January 4, 2016
A patient and thorough bottom-up explanation of Bayes' rule. There are parts that end up being skippable, since Stone often will explain the same concept in multiple ways, and he takes an awfully long time to introduce some clarifying notation. But, it remains a worthwhile read and a very good aid to developing the intuition for Bayes' rule.
February 2, 2020
Good solid introduction
Displaying 1 - 16 of 16 reviews