Math: Probability Question

[oldjones]

Given the wide variation in the mathematical probabilities, the real question is how that dubious witness 'earned' his rating of "…80% reliable"? The calculations seem to quite improbably suppose the rating is 100% accurate.

So who evaluated this person to that level of accuracy?

 

[Roleplayer]

Yes, a conditional probability problem.

Where A represents the colour of the car in the accident, W represents the colour reported by the witness, G stands for green and B stands for blue:

P(A=B and W=B)=0.8*0.8=0.64
P(A=B and W=G)=0.8*0.2=0.16
P(A=G and W=B)=0.2*0.2=0.04
P(A=G and W=G)=0.2*0.8=0.16

Hence,
P(W=G)=0.16+0.16=0.32
P(W=B)=0.64+0.04=0.68

We can then use the conditional probability definition, for events X and Y, P(X given Y)=P(X and Y)/P(Y)

So for question 1:
P(A=G given W=G)=0.16/0.32=0.5

Question 2:
P(A=B given W=B)=0.64/0.68=0.94 (approximately)


In my experience, the thing about probability is that it's not particularly harder than other mathematical disciplines, but people are a lot more likely to think their intuition is going to be correct than if you asked them a question about, say, derivatives. And a person's intuitive understanding of probability without training tends to be pretty awful.

 

[IM469]

If the witness is 100% reliable are not the odds 100% he is correct?


This is a conditional probability question.

You are ignoring what is called the base rate IE taxi probability

Use Bayes theorem. I think the answer is 94% by my calculations

And with Bayes math if the witness is 100% correct you get infinity for the answer or something like that (I am just a truck driver )

Not just a truck driver - you obviously understand this problem and drive a truck - both of which I have difficulty in performing.

I address the "If he reported Green he would be right 16 times and wrong 16 times for a total of 50 per cent right." with an individual that 80% of the time his answers are factually correct. When some defines a number as 80 (e.g. a basket 80 apples) you cannot draw some probability law - look in the basket and tell me that 80 is now 50 (ie 50 apples).

Let me introduce you to IM469's reliability vs sample set theory. If any individual is correct identifying an object correctly 4 out of 5 times (i.e. : 80%) - it is irrelevant how rare or prevalent the objects are in that sample set - I know that during any series of identification of an objects - 4 out of 5 times he/she is correct - it is the definition of his reliability - period.

Completely different is his probability of picking a green cab from a pool containing multiple choices in various numeric ratios. Now I require his probability of choosing correctly with the overall sample set and choices in it. I'll buy that. I'll buy that given those manipulations he has a 50% chance of picking a green taxi when his measuring correct and incorrect (all 5 choice right or wrong) against the entire sample set. This does not mean he is reduced from 80% to 50% correct in his reliability (in my mindset). His odds on picking an object from a sample set has no relationship in his ability to correctly identify an object.

These new fangled mathematics - soon I'll have store clerks giving me $5 change then arguing through reverse Bayesian logic that I'm actually holding $8 !

 

[frankcastle]

It's not my argument. It's the solution put forward by experts in their probability contingency tables


===========================Probability Contingency Table============================


= =========== ======= Blue Taxi Involved ========== Green Taxi Involved

Number of cases ============ 80 ===================== 20============================


Witness Report

Blue Taxi============== .8 * 80 = 64 =============== .2 * 20 = 4 ============= 68 Blue Cases from witness report

Green Taxi============= .2 * 80 = 16 =============== .8 * 20 = 16 ============32 Green Cases from witness report

From the table above you have 100 outcomes 64 + 4+ 16+16= 100

If he reported Blue he would have been right 64 times and wrong 4 times for a total of 94.1 per cent right
If he reported Green he would be right 16 times and wrong 16 times for a total of 50 per cent right.

Wait but don't those four scenarios include correct colour but the witness was wrong? The original question posted was the probability that the witness was correct.

 

[Roleplayer]

Perhaps an extreme case to illustrate the point:

Suppose all of the cars were blue and the witness said green (which he would do 20% of the time). What's the probability that the witness is right? You can quickly see that the probability is 0; the witness is definitely wrong, because you know the car is blue. Point being, the probability that the witness is correct is definitely affected in a particular case by the distribution of the objects in question.

Notice also, from the numbers I gave above in answer to the original problem, that the witness will say blue 68% of the time and green 32% of the time, and that 0.68*0.94+0.32*0.5=0.8. In other words, the witness is correct 80% of the time in general, but the witness is right in 50% of those times he/she said green, and 94% of the times he/she said blue.

Edit to add: Also notice that the question is not "The car is blue. What's the probability the witness will say blue?" It's "The witness said blue. What's the probability the car actually was blue?" Those are very different things.

 

[IM469]

Perhaps an extreme case to illustrate the point:

Suppose all of the cars were blue and the witness said green (which he would do 20% of the time). What's the probability that the witness is right? You can quickly see that the probability is 0; the witness is definitely wrong, because you know the car is blue. Point being, the probability that the witness is correct is definitely affected in a particular case by the distribution of the objects in question.

The witness is still (by definition) correct 80% of the time - it will never change by definition. Your ability to predict the which selection is correct does not affect his ability to be correct 80% of the time. You can predict in this exclusive sample set that the remaining guesses will be blue if the reliability is accurate but the actual selection doesn't change his reliability. As long as there is a viable sample to choose from independent of the ratio of individual objects - there is an 80% chance he/she will be correct. You can use the numbers in the pool to predict results but his 80% correctness will never vary.

I think

 

[Roleplayer]

Suppose for a moment there were a million cars, only one of them green, the rest blue. If the witness said the car was green, would you think it more likely that the car was green, or that the witness made a mistake?

Suppose further that there was one green car out of a million, and that the witness was actually just guessing, and had a 50% chance to say green or blue. This means the actual colour of the car is irrelevant to the witness's testimony. Before the witness said anything about the colour of the car, you would say there's a 50% chance the witness will get it right. But if the witness said the car was green, would you say the same thing?

The witness is still (by definition) correct 80% of the time - it will never change by definition. Your ability to predict the which selection is correct does not affect his ability to be correct 80% of the time. You can predict in this exclusive sample set that the remaining guesses will be blue if the reliability is accurate but the actual selection doesn't change his reliability. As long as there is a viable sample to choose from independent of the ratio of individual objects - there is an 80% chance he/she will be correct. You can use the numbers in the pool to predict results but his 80% correctness will never vary.

I think

What we're talking about here is called a conditional probability. In simple terms, it means that if you know something about the outcome of some random experiment, and you may be able to "update" your understanding of the probability of certain outcomes under these conditions.

I'm not saying the reliability of the witness changes in a general sense. I'm saying that if you know what the witness said about the colour of the car, then you can use that information (along with the distribution of colours among the cars) to know the probability the witness is correct in this situation.

It might be easier to see an example of conditional probability in another situation. Suppose I roll two dice and add them together, but hide the roll from you. What is the probability that the total is 12, based on what you know? 1/36, because you need to roll 6 on both dice, each of which has a probability of 1/6, and 1/6 * 1/6 = 1/36. But now suppose I reveal the first rolled die to you. If that die is a 6, you can now say the probability that the total is 12 is 1/6, because you now only need the second die to be a 6. On the other hand, if the first die is not a 6, then the probability that the total is 12 is 0. It isn't possible. These are two examples of conditional probability, the probability that the total is 12 given that the first die is 6 or the probability that the total is 12 given that the first die is not 6. You aren't saying the probability of a particular roll has changed in general, but that the probabilities are different when you already know something about what happened.

This stuff is involved in what I do for a living, by the way. Not that that necessarily means anything; I've certainly met people that do something involving this stuff for a living that don't actually understand it.