.jpg)
Both produced by Sazerac's Barton 1792 distillery, Costco's Kirkland Bottled-in-Bond is a value darling of the wider internet and Total Wine's Wolcott Bottled-in-Bond has placed well at international spirits competitions, though most of its metal finishes were while it was still made at Sazerac's Buffalo Trace distillery. As both are private label bottles contracted distilled by Barton, we are left to ask the question: which one is better?
The Contestants - Kirkland and Wolcott
You can find our full bottle write-ups on both whiskies here:
The broad summary is that the Kirkland bottle is significantly cheaper though both the Costco and Total Wine brands here cost less than the equivalent 1792 Bottled-in-Bond, a hard to find iteration that is only now starting to show up reliably on shelves again. Per the Bottled-in-Bond act, both drams will be 100 proof, aged a minimum of four years, and made in batches of barrels from the same distillation season(Jan-Jun and Jul-Dec). Tasting notes are broadly similar with what seemed to me to be some minor differences in mouthfeel. After conducting an initial side-by-side assessment, I concluded that the two bottles are too similar and that we should first determine whether or not a perceptible difference even exists.
Difference Blind Tasting
If you've peeped our Blind Tasting Guide, the following procedures will be familiar to you already. To prove that the two bottles are meaningfully different from one another, we will utilize a number of different blind tasting methods. For fun, multiple methods will be used, though you could simply build up a decent enough sample size using one technique. Our target will be 20-30 comparative tastings, which is a rough convention for statistical significance.
Test 1: Basic Blind Head-to-Head
In this test, a glass of each whiskey is poured and labeled. Taster samples each glass to ground their palate. Labels are not visible to the taster. Glasses are scrambled on a lazy Susan and then one glass is randomly selected for tasting. Taster assigns their best guess as to which whiskey is in the glass. After the guess, the label is revealed and the result marked, correct or incorrect. If the two whiskies are imperceptibly different, the proportion of right and wrong answers should approach 50%. Four tastings were conducted over three days in this manner.
- Day 1: Correct, Incorrect, Correct, Correct
- Day 2: Incorrect, Incorrect, Correct, Correct
- Day 3: Correct, Incorrect, Correct, Incorrect
Total from 12 trials: 7 Correct, 5 Incorrect, 58.3% Correct
Expected value of 6 correct answered with a standard deviation of √(np(1-p)) which is 1.73 giving us a p-value of .56 which is not statistically significant. This would lead us to conclude that there is no real difference in the tasting experience of the two whiskies. In reality we should do more samples, but the blind head-to-head has some failings from a methodology perspective: the weight of the glasses may change as the number of samples from each glass is not fixed, it is difficult to control for sip size which can impact experience, and it is very easy to lose the grounding of the initial tasting.
I found myself most targeting the mouthfeel and finish sensation rather than the flavor profiles, though as you can see in the results, that approach may have not borne fruit. I did continue to feel like there was some small difference, but let's see how the more robust test patterns hold turn out.
Test 2: Kirkland vs Wolcott Triangle test
Triangle tests make up for most of the failings of the basic blind head-to-head, one of the many reasons that they are the industry standard for comparative tastings in food and beverage. In a paired triangle test, three samples of each whiskey are poured. One sample of each is swapped so that there is an odd-one-out in each group. Groups and sample order within the group are randomized. The taster then selects their best guess as to which is the differently sourced/prepared sample. If there is no difference between the two products, we would assume to taster to be correct only around one-third of the time.
- Trial One: 2 As - Correct
- Trial Two: 2 Bs - Incorrect
- Trial Three: 2Bs - Incorrect
- Trial Four: 2As - Correct
- Trial Five: 2As - Correct
- Trial Six: 2Bs - Incorrect
Total from 6 trials: 3 Correct, 3 Incorrect, 50% Correct
I continue to be convinced that there is a difference, but it is subtle. There is variation in the amount of heat and nuttiness between the two, but I'm working hard to keep the memory of each flavor on my tongue while spacing things out enough to not obliterate my palate.
Test 3: Duo Trio Test
Potentially my favorite of the discrimination tests, the duo trio test is a setup by which the taster sips a priming sample and then tastes two randomized samples, guessing which one matches the initial taste.
- Test 1: Wolcott - Correct
- Test 2: Kirkland - Correct
- Test 3: Kirkland - Incorrect
- Test 4 Wolcott - Correct
Total for four trials: 3 correct, 1 incorrect, 75% Correct
Conclusion
Having done 22 separate trials with different methodologies, I will commit a statistical sin by combining the numbers:
Total across all trials: 13 Correct, 9 incorrect, 59.1% Correct
Since combining tests with different ( p ) values is tricky, I'll make a simplifying (though less rigorous) assumption an "average" null probability weighted by the number of trials:
- Test 1 and 3 (16 trials) have p = 0.5
- Test 2 (6 trials) has p = 1/3
Now, treat all 22 trials as one binomial experiment:
- n = 22
- k = 13
- p = 0.455
- P(X >= 13) = sum{k=13}^{22} \binom{22}{k} (0.455)^k (0.545)^{22-k}
Calculating exact probabilities is somewhat annoying, so I'll approximate with normal:
- Mean: 22 * 0.455 = 10.01
- Variance: 22 * 0.455 * 0.545 = 5.456
- Std Dev: sqrt{5.456} ≈ 2.336
- z = {13 - 10.01}/{2.336} ≈ 1.279
- P-value (one-tailed) ≈ 0.1005
This means that due to our small sample size, we are hovering around rejecting the null hypothesis at a 90% confidence level (saying a difference DOES exist). While I'm shelving this exercise for now, I'll take a 90% confirmation of some difference with my tongue! In my opinion and with minor numerical reinforcement, there is a difference between Wolcott Bottle-in-Bond and Kirkland Bottled-in-Bond.
I prefer Costco's Kirkland for its slightly smoother finish and higher fruit to nut ratio, but the two bottles are very similar. I suspect that your best Total Wine arbitrage is the Wolcott Rickhouse Reserve which is a proxy for the elusive Kirkland Single Barrel and 1792 Full Proof.