A perfect demonstration of why AB and ABX tests give false negatives

Subjects that don't have their own home
User avatar
Nick
Site Admin
Posts: 15706
Joined: Sun May 06, 2007 10:20 am
Location: West Yorkshire

#31 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Nick »

pre65 wrote: Mon Jul 24, 2017 6:45 pm The choice of the word "audition" was your suggestion Nick, but at the end of the day you call it what you like. :?
Yes I know, and you said you preferred it, so I then asked you why you preferred it in an attempt to try and understand why you disliked "bake off"
Whenever an honest man discovers that he's mistaken, he will either cease to be mistaken or he will cease to be honest.
User avatar
pre65
Amstrad Tower of Power
Posts: 21373
Joined: Wed Aug 22, 2007 11:13 pm
Location: North Essex/Suffolk border.

#32 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by pre65 »

Nick wrote: Mon Jul 24, 2017 6:52 pm
pre65 wrote: Mon Jul 24, 2017 6:45 pm The choice of the word "audition" was your suggestion Nick, but at the end of the day you call it what you like. :?
Yes I know, and you said you preferred it, so I then asked you why you preferred it in an attempt to try and understand why you disliked "bake off"
I see you avoided the "fest" question. :lol:

I'm flattered that you are trying to understand me, but I'm not sure why I find "bake off" distasteful, I just do.
The only thing necessary for the triumph of evil is for good men to do nothing.

Edmund Burke

G-Popz THE easy listening connoisseur. (Philip)
User avatar
Dr Bunsen Honeydew
Old Hand
Posts: 1358
Joined: Mon Sep 21, 2009 10:50 am
Location: Muppet Labs

#33 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Dr Bunsen Honeydew »

Bake-off has become the established term for the last decade. I think it originated at Hi-Fi Wigwam which has kept it as a tradition amoungst members, and a very good tradition sadly damaged by the elitism and in-crowd attitude that grew there over the years. I see no reason to change it apart from perverse stupidity.

It extended to Bake-off shows as in Scalford, Owston, NEBO, MiBO ASBO and as in hiring a hall or a hotel instead of using a members house.
User avatar
pre65
Amstrad Tower of Power
Posts: 21373
Joined: Wed Aug 22, 2007 11:13 pm
Location: North Essex/Suffolk border.

#34 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by pre65 »

Maybe so Richard, but "fest" is a traditional Audio-Talk (and World Designs before) word.

Eggborough gatherings were Eggfest, or Eggyfest, and home gatherings were mini-fests, my own one was a Phil fest.

To say it's "perverse stupidity" is incorrect, and very rude to boot.
The only thing necessary for the triumph of evil is for good men to do nothing.

Edmund Burke

G-Popz THE easy listening connoisseur. (Philip)
User avatar
pre65
Amstrad Tower of Power
Posts: 21373
Joined: Wed Aug 22, 2007 11:13 pm
Location: North Essex/Suffolk border.

#35 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by pre65 »

Perhaps "bake off" seems a bit American to me ? Best I can come up with.

I don't want this blown up out of proportion, it's not a life or death problem. :)
The only thing necessary for the triumph of evil is for good men to do nothing.

Edmund Burke

G-Popz THE easy listening connoisseur. (Philip)
lindsayt
User
Posts: 40
Joined: Sat Jul 22, 2017 8:34 am

#36 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by lindsayt »

Nick wrote: Mon Jul 24, 2017 6:39 pm
lindsayt wrote: Mon Jul 24, 2017 6:34 pm If on playing the distorted clip a second time, I was unable to tell whether I was being played the distorted version or not then yes that would indicate that large differences can be masked by the brain. But that was not the case with this example. Every time I heard the distorted version I was fully aware that I was listening to an annoyingly grossly distorted recording.
Yes, of course, but that was not the point I was making, not that you could not tell the distorted version from the non distorted version, but than once you heard the undistorted version, the distorted version sounded very different, and you could not go back to the situation where the distorted version was unintelligible.
Yes I could. Just played the clip for a 2nd time, a few hours after last hearing it. This time it sounded like R2D2 gibberish again.
Left it a couple of minutes, played the clip a 3rd time and again it was gibberish.

Seems to be a short term brain speech processing thing. Where I can fill it in only if I hear it a few seconds after hearing the undistorted version.

For me the jury's out. There may well be some brain filling in when comparing 2 similar systems back to back in a short space of time. Or it could be some other mechanism that causes false negatives in these types of listening tests?


I would agree that it is entirely possible to get false negatives with AB and ABX tests.
User avatar
Nick
Site Admin
Posts: 15706
Joined: Sun May 06, 2007 10:20 am
Location: West Yorkshire

#37 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Nick »

Seems to be a short term brain speech processing thing. Where I can fill it in only if I hear it a few seconds after hearing the undistorted version.
As you do in a AB or ABX test.

My point is that this is demonstrating a mechanism where we can expect false negatives to be produced using a AB or ABX test, they may be others, but the existence of just one means that as a test its fatally flawed.

As you say if you give it some time between tests then the brain forgets, but if you give it time, then it ceases (IMHO) to be a AB test, it becomes listening to two things at different times. At which point ironically the strict objectivist would declare that limited aural memory makes the test worthless as well.
Whenever an honest man discovers that he's mistaken, he will either cease to be mistaken or he will cease to be honest.
User avatar
IslandPink
Amstrad Tower of Power
Posts: 10041
Joined: Tue May 29, 2007 7:01 pm
Location: Denbigh, N.Wales

#38 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by IslandPink »

I think the 'Bake off' business is just a current fashion based on the TV cake/Hollywood programme. I haven't seen the Europeans calling the next annual DIY meeting the 'Euro Triode Bake-Off'.
"Once you find out ... the Circumstances ; then you can go out"
User avatar
shane
Social outcast
Posts: 3403
Joined: Sun Sep 16, 2007 12:09 pm
Location: Kept in a cool dry place.

#39 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by shane »

Some people, myself included, do find that the accelerating intrusion of Americanisms into our everyday discourse grates a little, but there's also the implied element of competition in the phrase that makes it just a little more likely that the whole thing turns into a willy-waving exercise. What's wrong with meet, or meeting?
The world looks so different after learning science. For example, trees are made of air, primarily. When they are burned, they go back to air, and in their flaming heat is released the flaming heat of the Sun which was bound in to convert air into tree.
User avatar
Dr Bunsen Honeydew
Old Hand
Posts: 1358
Joined: Mon Sep 21, 2009 10:50 am
Location: Muppet Labs

#40 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Dr Bunsen Honeydew »

Wigwam started it, Wigwam called them Bake-Off, Pink Fish continued the name. All the forums that came out of them like AoS and HFS and The Audio Standard have continued the name. Wigwam started using the name nearly 15 years ago.

A quote from Wigwam.

"I think they were called Bake Offs as they consisted of blokes crammed into a small room with large valve amps on all day, making it 'baking' hot."
User avatar
pre65
Amstrad Tower of Power
Posts: 21373
Joined: Wed Aug 22, 2007 11:13 pm
Location: North Essex/Suffolk border.

#41 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by pre65 »

shane wrote: Mon Jul 24, 2017 10:04 pm Some people, myself included, do find that the accelerating intrusion of Americanisms into our everyday discourse grates a little, but there's also the implied element of competition in the phrase that makes it just a little more likely that the whole thing turns into a willy-waving exercise. What's wrong with meet, or meeting?

Well said Shane.
The only thing necessary for the triumph of evil is for good men to do nothing.

Edmund Burke

G-Popz THE easy listening connoisseur. (Philip)
User avatar
Nick
Site Admin
Posts: 15706
Joined: Sun May 06, 2007 10:20 am
Location: West Yorkshire

#42 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Nick »

What's wrong with meet, or meeting
I would say nothing, but in the case of a meeting with the intention of comparing a particular type of item, clearly with a view to finding some form of ordering of quality, I think a bake off is descriptive enough. ETF pre dates us all, though I think the first get together at Ali's place in Leeds was also about 15 years ago. I have some pictures from Paul’s place from 2003, but again, its not a competition.

Of course a "race meeting" also shows that there is noting stopping the use of the word meeting from also suggesting a competition.
Whenever an honest man discovers that he's mistaken, he will either cease to be mistaken or he will cease to be honest.
User avatar
Ray P
No idea why I do this anymore
Posts: 6294
Joined: Thu Nov 22, 2007 5:18 pm
Location: Somerset

#43 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Ray P »

Nick wrote: Mon Jul 24, 2017 10:54 pmOf course a "race meeting" also shows that there is noting stopping the use of the word meeting from also suggesting a competition.
Except that it is the races that are the competitive element, not the meeting; one might have several 'bake-offs' at a meeting.
Dr Bunsen Honeydew wrote: Mon Jul 24, 2017 7:04 pm I see no reason to change it apart from perverse stupidity.
Here's a fairly compelling reason for not using the term 'bake-off' - there's absolutely no baking involved.

These events bear no resemblance to my understanding of the term 'bake-off', which involves true competition, creating something on the day, in a defined amount of time, with everyone having identical facilities and possibly a prescribed set of 'ingredients'. In addition the outcome of a culinary bake-off comes down to the judging panels' tasting of a standalone outcome, unlike audio comparison where the assessment of a component is pretty meaningless as a standalone outcome (unless you have awards for 'best soldering', best casework, etc) and in use its performance is dependent on other components and influenced by its immediate environment.
Sorry, I couldn't resist!
Cressy Snr
Amstrad Tower of Power
Posts: 10552
Joined: Wed May 30, 2007 12:25 am
Location: South Yorks.

#44 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Cressy Snr »

We used to have the coveted tube trophy once. I won one in 2006, but we stopped doing that years ago.
It's more of a meeting these days.
Sgt. Baker started talkin’ with a Bullhorn in his hand.
User avatar
Nick
Site Admin
Posts: 15706
Joined: Sun May 06, 2007 10:20 am
Location: West Yorkshire

#45 Re: A perfect demonstration of why AB and ABX tests give false negatives

Post by Nick »

Tell you what, you can all focus on what we call it, I will spend my time trying to make better things to listen to when you decide what its called.
Whenever an honest man discovers that he's mistaken, he will either cease to be mistaken or he will cease to be honest.
Post Reply