In his Creative Checklist column, Dean Rieck warns about making decisions based on insignificant results, but doesn't say how many pieces you have to mail to get a statistically valid test.
Ed McLean addressed this topic decades ago in his classic monograph, “The Basics of Testing.” In it, he included a table to help the reader determine the minimum acceptable test cell size.
The table shows that to mail with an 85 percent confidence level that your rollout results won't decline more than 25 percent of the test figure, you must mail enough to collect 14 responses. If your response rate is 1 percent, that means each test cell must have 1,400 pieces.
Bob Bly, Copywriter/consultant,Dumont, NJ,
