OFR Design
Home Up Member Posters OFR Design Research Pointers Understanding Stats Research Reports Cropping Resources Livestock Resources Thompson Alternatives Cooperator Discussion Audubon Co. Farms Cuba Study Trip PFI and Perú Photos & Sounds The Practical Farmer

 

For a printable, formatted version of this document, click here  (requires Adobe Acrobat™ Reader)

The Paired-Comparison: A Good Design for 
Farmer-Managed Trials

revised August, 2003

Rick Exner, ISU Extension PFI Coordinator and

Richard Thompson, Practical Farmers of Iowa

THERE IS A PRESSING NEED FOR AN ON-FARM RESEARCH METHODOLOGY THAT:

  1. EMPOWERS FARMERS TO QUANTITATIVELY EVALUATE ALTERNATIVE AGRICULTURAL PRACTICES;
  2. PRODUCES RESULTS CONSIDERED CREDIBLE BY BOTH AGRONOMISTS AND OTHER FARMERS;
  3. IS “FARMABLE” USING EXISTING EQUIPMENT AND WITH A MINIMUM OF ADDITIONAL EFFORT

The paired-comparison trial can meet many of the needs of on-farm research. If treatment strips of one or two equipment-widths are run completely across the field, no extra stopping, turning or adjustments are required. Because pairs of these narrow strips lie close to one another, field variability between treatment strips is minimized. The pair of treatments is replicated at least six times in the field to overcome chance field differences. The order of the two practices in each pair is chosen at random, avoiding a source of unintentional bias.

Each data pair yields one difference. These differences can be analyzed using a simple, single-sample technique. Farmers, themselves, if they have a calculator or spreadsheet that does square roots, can generate the L.S.D. (least significant difference) with which to evaluate a two-treatment trial.

RANDOMIZATION AND REPLICATION

In this trial the practices, or “treatments,” are paired six times, making six “replications,” or “reps.” Replication is crucial because it gives you a “second opinion” (and a third, fourth, ...sixth) on your question. With only one pair in the trial, you would have to base your conclusions on only one observation - without knowing how representative that single result really was.

Notice that in this trial the practices or “treatments” are paired in random order. That is, the strips don’t just go “AB AB AB...” across the field. Randomization is important to avoid the biasing effects that you often don’t even anticipate. For instance, if practice A were always downhill or were always downwind from practice B, the results could reflect the position of the strips as well as the practices in them. So to avoid any question of conscious or unconscious bias: randomize!

Work out a new randomization for each trial. (A field layout is only random the first time you use it.) If the trial will have two treatments, start with two pieces of paper and write the name of a treatment on each one. Put them in a milk bucket or a hat and give it a shake. Now pick one piece of paper out without looking to see which it is. That will be the treatment for the first strip of the first pair. Obviously the second strip in the pair gets the other treatment. Repeat this procedure for each pair of strips (each replication). Write down your results as you go. When you finish, take a look at your randomization. If you don’t see any really good reason why that arrangement of treatments would prejudice trial results then keep it. Otherwise repeat the whole process. Write down your field diagram in two places - one safe place in your office and one convenient place like a pocket notebook. The last thing you want is to ruin your work by losing track of where you did what in the field. This is the first step in your record keeping. As you see things in the field, write them down in a pocket notebook, then transfer the information to a more permanent record. Your recorded observations can lead to insights months - even years later.

 

AN EXAMPLE OF A PAIRED-COMPARISON TRIAL AND ITS ANALYSIS

Fertilizer Type

Pair number None Starter Difference
(X)
(X-X) (X-X)2
(Yield in bu. /acre)
1 117.9 118.9 -1.0 -6.62 43.78
2 125.2 112.0 +13.2 7.58 57.51
3 119.9 100.7 +19.2 13.58 184.51
4 114.9 110.8 +4.1 -1.52 2.30
5 116.9 119.9 -3.0 -8.62 74.25
6 119.7 118.5 +1.2 -4.42 19.51
n = 6

X =  5.62

§ = 381.85


Sample Variance: sx2 = §/(n-1) = 381.85/5 = 76.37

Variance of the Mean: sx2 = sx2/n = 76.37/6 = 12.73

Standard Error of the Mean: sx = (12.73)˝ = 3.57 bu./acre
(the “˝” means take the square root of 12.73)

Student’s t.05, 5 = 2.571

Get t from a “t table.” A high school statistics book would have a t table. “.05” (sometimes called “a”) is the chance of an error at the 95% confidence level. “5” is the number “n” minus 1, or (6-1) here. The table may use the term “degrees of freedom” for the number (n-1). The correct number for t is in the table at the intersection of the “.05” column and the “5” row. You might want to demand a higher level of confidence (99%), or you might settle for a less stringent burden of proof (90%, 80%).

Least Significant Difference at 95% confidence level:

    LSD.05 = sx × t.05, 5 = 3.57 × 2.571
   
                                  = 9.17 bushels per acre

So here the observed treatment difference (X=5.62 bushels) was less than the difference (9.17 bushels) that would occur simply by chance 5 times out of 100. That is, the difference observed was not statistically significant at the commonly used 95% confidence level. So in this case there was no significant yield difference between the corn that got starter fertilizer and the corn that received none. However, the result comes from just one field and one year. It’s a good idea to repeat a trial elsewhere to see if the outcome is reproducible.

Properly conducted paired-comparison trials on Iowa farms in 1987 were capable of detecting finer treatment differences than some experiment station research (P. Rzewnicki, et al. Fall 1988. American Journal of Alternative Agriculture, Vol. 3, No. 4). On-farm research does not replace experiment station work, which often uses more complex designs. The point is that for what these simple on-farm trials set out to accomplish, they do a very credible job.

STUDENT’S t-DISTRIBUTION CRITICAL POINTS

 “2-TAILED” TEST OF DIFFERENCE (EITHER GREATER OR LESS)

Number of Pairs 

Degrees of Freedom 

a = .20 

a = .10 

a = .05 

a =  .01

1  N.A.  N.A.  N.A.  N.A.  N.A.

2

3.078 

6.314 

12.706 

63.657

1.886 

2.920 

4.303 

9.925

1.638 

2.353 

3.182 

5.841

1.533 

2.132 

2.776 

4.604

1.476 

2.015 

2.571 

4.032

1.440 

1.943 

2.447 

3.707

1.415 

1.895 

2.365 

3.499

1.397 

1.860 

2.306 

3.355

10 

1.383 

1.833 

2.262 

3.250

11 

10 

1.372 

1.812 

2.228 

3.169

12 

11 

1.363 

1.796 

2.201 

3.106

13 

12 

1.356 

1.782 

2.179 

3.055

14 

13 

1.350 

1.771 

2.160 

3.012

15 

14 

1.345 

1.761 

2.145 

2.977


When a = .20, the confidence level = (1.00 - .20), or 80%. (Chance of wrong conclusion = 20%.)

When a = .10, the confidence level = (1.00 - .10), or 90%. (Chance of wrong conclusion = 10%.)

When a = .05, the confidence level = (1.00 - .05), or 95%. (Chance of wrong conclusion = 5%.)

When a = .01, the confidence level = (1.00 - .01), or 99%. (Chance of wrong conclusion = 1%.)

A “Student’s t” table can be found in the back of most statistics textbooks. If the experimental question is: “Is practice ‘A’ different from practice ‘B’ (less or more)?” then you want a “two-tailed” t table that divides the chance for error () between the lower and the upper tails of the bell curve. If your question is specifically “Is practice ‘A’ greater than practice ‘B,’?” then you would use a one-tailed t-table. Most t tables are two-tailed and don’t even bother to say so. Notice how t diminishes as the number of replications increases? Other things being equal, a smaller t means a smaller L.S.D., and a smaller L.S.D. means the trial is more sensitive to the treatment differences you are looking for.

WORK SHEET PROBLEM

DEEP-BANDED P & K FERTILIZER VERSUS NONE

HARLAN AND SHARON GRAU FARM, 1990

Treatment Type

Pair number

Deep Band

None

Difference
(X)

(X-X)

(X-X)2

(Yield in Bu./acre)
1 98.8 85.8 _______ ____.____

_____.____

2 96.9 85.8 _______ ____.____

_____.____

3 97.4 87.0 _______ ____.____

_____.____

4 101.2 89.9 _______ ____.____

_____.____

5 103.5 93.2 _______ ____.____

_____.____

6 85.8 87.8 _______ ____.____

_____.____

n = _____

X = ______.____
(Mean Difference)

§ = _________.____
(Sum of Squares)

 

n–1 = _____

Sample Variance: sx2 = §/(n-1) = ______.____

Variance of the Mean: sx2 = sx2/n = ______.____

Standard Error of the Mean: s = (sx2)˝ = ______.____ bu./acre
(the “˝” means take the square root of sx2)

Student’s t.05, n-1 = ______.______

t.05, n-1 comes from a “t table.” A high school statistics book would have a t table. “.05” (sometimes called “a”) is the chance, at the 95% confidence level, of incorrectly concluding there is a real yield difference between treatments. The number (n-1) is one less than the number of pairs in the field. The table may use the term “degrees of freedom” for the number (n-1). The correct number for t is in the table at the intersection of the “.05” column and the “n-1” row.

Least Significant Difference at 95% confidence level:

LSD.05 = sx × t.05, n-1 = _____._____ × _____._____

                                    = _____._____ bushels per acre

Was the observed treatment difference (X) more or less than the difference (LSD.05 in bushels) that would occur simply by chance 5 times out of 100? If it was greater, then the yield difference observed was statistically significant at the commonly used 95% confidence level.

BLANK WORK SHEET

YOUR EXAMPLE HERE

Treatment Type

Pair number

Deep Band

None

Difference
(X)

(X-X)

(X-X)2

(Yield in Bu./acre)
1 _______ _______ _______ ____.____

_____.____

2 _______ _______ _______ ____.____

_____.____

3 _______ _______ _______ ____.____

_____.____

4 _______ _______ _______ ____.____

_____.____

5 _______ _______ _______ ____.____

_____.____

6 _______ _______ _______ ____.____

_____.____

7 _______ _______ _______ ____.____

_____.____

8 _______ _______ _______ ____.____

_____.____

n = _____

X = ______.____
(Mean Difference)

§ = _________.____
(Sum of Squares)


n–1 = _____

Sample Variance: sx2 = §/(n-1) = ______.____

Variance of the Mean: sx2 = sx2/n = ______.____

Standard Error of the Mean: s = (sx2)˝ = ______.____ bu./acre
    (the “˝” means take the square root of sx2)

Student’s t.05, n-1 = ______.______

Least Significant Difference at 95% confidence level:

LSD.05 = sx × t.05, n-1 = _____._____ × _____._____

                                    = _____._____ bushels per acre

Was the observed treatment difference (X) more or less than the difference (LSD.05 in bushels) that would occur simply by chance 5 times out of 100? If it was greater, then the yield difference observed was statistically significant at the commonly used 95% confidence level.


BLANK FIELD PLOT LAYOUT