Wind Tunnels:  How good are they?
A brief discussion of experimental error

Words and Images by: Kraig Willett

Conducting a rigorous scientifically based experiment isn't as easy as it might seem, since all experiments are subject to errors.  There are two main sources of error in most experiments:

Random errors
Systematic/Bias errors

It should be the responsibility of the experimentalist to address/comment on these errors in the communication of the results.

Random Error

In the specific case of wind tunnel experiments, random error can be addressed by simply taking lots of data.  As one might imagine, the larger the sample size (more data points), the lower the random error.  In the limiting case, as the sample size goes to infinity, random errors will go to zero.  The elimination of random error is a good thing - though, often times extreme pursuit of this goal adds costs to the test and can reach a point of diminishing returns.

A common theme raised by wind tunnel critics is that the data produced in a tunnel is not "real world".  As support of this assertion, "real world" data is provided that is contrary to the conclusions drawn based on wind tunnel data.  Unfortunately, the problem with most "real world" tests is that they lack control of all relevant variables (primarily ambient wind) and they also lack a sufficient number of trials to reduce random errors.

If a field test is conducted properly, though, the results should correlate with wind tunnel tests.  This has been demonstrated in the academic literature:

Validation of a Mathematical Model for Road Cycling Power
J.Martin, D.Milliken, J.Cobb, K.McFadden, A.Coggan
Journal of Applied Biomechanics, 1998, 14, 276-291

http://www.humankinetics.com/products/journals/showarticle.cfm?articleid=3751&journalid=JAB

In order to generate statistical confidence in the test and the results, random error needs to be addressed.  In a wind tunnel, this means a sufficiently long sample period or sufficiently high sample rate.  In a field test, this means many runs over the test course (on the order of 16+ runs - depending on the magnitude of the effect one is trying to detect), which takes personnel time. 

Systematic Errors

At the root of systematic error during a wind tunnel test is the instrumentation used to measure the relevant variables of axial force (F), air density (rho), air speed (u).  In a nutshell axial force (or drag at 0 degree yaw), is defined as (Cd/Cx is axial force coefficient and A is projected area):

Most tunnel data is presented as an axial force at a normalized tunnel speed of 30 mph or some other constant value (this corrects for the slight variations in tunnel speed during the actual trial - 30.1 mph or 29.9 mph, etc...).

Therefore, the presented data is really a function of the tunnel determined product "CxA".  As shown above, CxA is dependent on F, rho, and u.  Physical instruments must measure these quantities, and each of these instruments will have their own contribution to the CxA term.  An uncertainty analysis is a common exercise used to explore what is primarily responsible for the error of the measurement.  In essence, the total uncertainty is the square-root of the sum of the squares of each of the individual instruments (that equation above that has all the partial derivative terms).

So, if we continue the uncertainty exercise we find that for each of the terms:

A note on one of the assumptions used in the equations above - it is assumed that lab quality, precision instruments are being used that have been calibrated to be accurate to +/- 0.1 % of a full-scale value.  So, if an instrument was designed and calibrated to measure a 100 lb load and exhibits a +/- 0.1% of full-scale value, the uncertainty of that instrument would be +/- 0.1 lbs.

Now, if we plug these terms back into our original equation, and investigate a typical wind tunnel run done at 25 mph, we can see what component (air density, tunnel speed, or the force) of our instrumentation is driving the total uncertainty in the final results:

As the grayed/highlighted boxes show above, the  force measurement term is several orders of magnitude larger than the air density and tunnel speed terms.  Thus, it is reasonable to conclude that that the primary contributor to tunnel error is typically the wind tunnel force measurement system.

The tunnel at A&M claims an uncertainty on their force measurement to be 0.05 lbs.  The Allied Aerospace facility in San Diego claims that their balance is calibrated to be accurate within 0.02 lbs (the Allied balance and the A&M balance are completely different in their design, FWIW).  The University of Washington tunnel makes no public claims about the accuracy of their balance, but they do say that it RESOLVES down to 0.02 lbs (resolving a force doesn't really help and is more a function of how good the A/D card is). 

All this math is great, but one quick test to ballpark a tunnel's "goodness" is to do a repeatability run.  That is, take data - typically a sweep in yaw angle - and then re-take the first data point.  Typical values from two facilities are tabulated below: 
 

Conclusion

Experimental error is typically a function of random errors and systematic errors.  Random errors can be reduced by increasing sample size, and systematic errors can be reduced by improving instrumentation.  For a cycling related wind tunnel experiment, the primary source of error is typically the force measurement system.  Overall, it should be pretty clear that wind tunnels are "good" at measuring forces (+/- 0.02 lbs at best), and just "how good" is facility dependent.