[gnso-rpm-wg] Directly from INTA's website: What the TTAB has to say about sample size

Thu Aug 31 10:30:15 UTC 2017

Hi folks,

Before I went to bed, I made a note to myself to do a quick search to
see what TM practitioners would do in their TTAB statistical studies.
This morning, one of the first hits I found came from INTA's own
website, see below. In particular, note the statements (from footnote
147):

(a) "characterizing 62 interviews as “anecdotal evidence” that did not
lend themselves to statistical conclusions"

(b) "finding 57 respondents raised a question “as to the overall
validity of the survey results"

The INTA survey sample size of 33 is far below even those two
examples, and was closest to the next example:

(c) "finding a survey of 25 pharmacists and doctors to be an
insufficient sampling"

(start of excerpt, sorry for the formatting, footnotes are #144 to
#147; easier to read the PDF I link to)

The Trademark Reporter (The Law Journal of the International Trademark
Association), September-October, 2014.

https://www.inta.org/TMR/Documents/Volume%20104/vol104_no5_a5.pdf

C. Representative Samples (page number 1172)

Probability and nonprobability methods may be
used to select the sample from the universe of possible respondents.
However, if the sample of respondents is not representative of the
universe from which it was selected, it will be accorded little
weight.144 The number of respondents sampled must be large enough for
the results to be reliable. The overall sample size for a survey will
depend on the number of disputed marks tested and whether the survey
includes any control groups. Surveys in Board proceedings often
interview between 100 and 300 respondents about each mark or stimulus
examined.145 In some instances, the Board has considered survey
samples with fewer than 200 respondents to be small,146 and samples
with fewer than 100 respondents routinely have been disfavored.147

And here are the footnotes:

144. iMedica Corp. v. Medica Health Plans, 2007 WL 1697344, at *5
(T.T.A.B. June 7, 2007) (“We also find that the survey results are
questionable because the survey did not fairly sample the universe of
possible respondents and is biased in MHP’s favor.”) and Am. Home
Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61 (T.T.A.B.
1970) (“[T]he persons to be interviewed were not chosen on the basis
of a sampling technique but solely because they were known to opposer
. . .”), aff’d, 473 F.2d 903 (C.C.P.A. 1973).

145. Facebook, Inc. v.
Think Computer Corp., 2013 WL 4397052, at *14 (T.T.A.B. July 23, 2013)
(“Dr. Ford supervised . . . interviews: 270 in the test cell and 272
in the control cell.”); PepsiCo, Inc. v. Pirincci, 2012 WL 2930650, at
*7 (T.T.A.B. June 25, 2012) (“In total, 404 consumers participated in
the survey . . . with 200 consumers participating in one of two ‘test
cells’ and 204 consumers in one of two ‘control cells . . .’”); Sara
Lee Corp. v. Mahmoud, 2007 WL 4663353, at *5 (T.T.A.B. Dec. 27, 2007)
(“[S]urvey respondents in the test group (199 women age 18 and older
from around the country) were shown a stimulus card . . .”); AVA
Enters. Trading Co., Inc. v. Audio Boss USA, Inc., 77 U.S.P.Q.2d 1783,
1786 (T.T.A.B. 2006) (“A test group of 100 respondents [was] shown a
card. . . . A control group of 100 respondents [was] shown a card . .
.”). Note, the test group may include more respondents than the
control group. See Anheuser-Busch, Inc. v. Mambo Seafood #1, Inc.,
2008 WL 4674603, at *7 (T.T.A.B. Sept. 22, 2008) (“The survey was
taken of 296 individuals, 200 of whom were exposed to applicant’s mark
with the remaining 96 persons comprising a control group that was
exposed to the fictitious mark . . .”).

146. 7-Eleven, Inc. v.
Morrison, 2008 WL 2385970, at *13 (T.T.A.B. June 2, 2008) (finding 162
survey respondents to be “small,” but according opposer’s survey some
weight); Kohler Co. v. Kohler Homes, 2008 WL 4877069, at *9 (T.T.A.B.
Nov. 4, 2008) (“[T]he number of actual respondents to the KOHLER HOMES
and KOHLER ASSOCIATES ARCHITECTS surveys is small, i.e., 164 and 163,
respectively.”).

147. Clear Choice Holdings LLC v. Implant Direct
Int’l, 2013 WL 5402082, at *8 (T.T.A.B. Aug. 26, 2013) (finding 90
respondents for each mark tested to be “a small number”);
Bridgestone/Firestone N. Am. Tire, LLC v. Silverstone Berhad, 2003 WL
1559659, at *4 (T.T.A.B. Mar. 2003) (characterizing 62 interviews as
“anecdotal evidence” that did not lend themselves to statistical
conclusions); iMedica Corp. v. Medica Health Plans, 2007 WL 1697344,
at *4-*5 (T.T.A.B. June 7, 2007) (finding 57 respondents raised a
question “as to the overall validity of the survey results”); Am. Home
Prods. Corp. v. B.F. Ascher & Co., Inc., 166 U.S.P.Q. 61, 62 (T.T.A.B.
1970) (finding a survey of 25 pharmacists and doctors to be an
insufficient sampling), aff’d, 473 F.2d 903 (C.C.P.A. 1973); Guardian
Life Ins. Co. v. England, 2002 WL 31173415, at *3 (T.T.A.B. Sept.
2002) (finding it inappropriate to draw conclusions based on a survey
with only three respondents).

(end of excerpt)

Of course, the above focused on sample size, but let's not forget the
other part, about the non-representative nature of sample. Re-read the
part above that said:

"However, if the sample of respondents is not representative of the
universe from which it was selected, it will be accorded little
weight"

That's exactly the second problem experienced with this INTA survey,
as  previously discussed.

Q.E.D.

Have a nice day.

Sincerely,

George Kirikos
416-588-0269
http://www.leap.com/