Is anyone aware of any research, or at least conclusions, regarding different ways to measure bullet hole groups?

I know the convention is to shoot X number of Y shot groups, then measure the extreme spread of each group and calculate the average. However, I don't want to shoot more groups/shots than necessary, or waste my time by shooting fewer than necessary. For example, if I can be 90% confident that a 10% difference in group sizes is meaningfull by shooting three groups, I'd rather do that than shoot 10 groups to be 95% confident. Heck, I might settle for 80% confidence.

I'm looking for answers to questions like this

1) How many shots are enough and how many groups are enough to be X% confident that a given difference is meaningfull?

2) Is one 10 shot group better than two five shot groups?

3) If I measure the Average Group Radius (AGR) instead of the Extreme Spread (ES), can I reduce the number of shots required to get a statistically significant answer?

It seems likely that AGR is a better method than ES, but has anybody actually tested it? Just because everybody has been using ES forever doesn't mean it's the best method. One good reason to use ES is that it's far easier to do than AGR, but I have a computer program that makes AGR easy enough.

GROUP SIZE ANALYSIS METHODS

(Maximum Spread)
The Maximum Spread (also called group diameter or extreme spread) is simply the distance between the centers of the two most widely dispersed shots in a group. This is the most common method used to describe group size because it is the easiest measurement to take. Unfortunately it only considers two shots in the result so an otherwise tight group of 10 shots with one "flyer" can produce results similar to a widely dispersed two shot group. For bench rest shooters who are capable of putting five shots in one hole it is a valid measure; but for the rest of us, or for handgun shooting it does not provide a "statistically significant" number for comparing targets.


(String Measurement)
This is an old method still used to determine a shooter's skill at hitting a target. It assumes the point of aim is always the desired point of impact and is simply the sum of the distances from the point of aim to each bullet hole. Originally a string was used to gather the distances, hence the name. It is still a valid measure of total error relative to the aim point. String Measurements however cannot be used to analyze sight settings because it only measures the magnitude of error, not the direction of error. It is also not a useful measure of group size because a tight group located away from the Bullseye will produce a large String Measurement.


(Average Group Radius)
The average of the distances from the statistical group center (not the aim point) to each shot is the Average Group Radius. It accurately reflects how far a typical shot will impact from the aim point with a well sighted arm. This measure is the best indicator of group size or firearm performance because flyers have less impact on the result so it provides a "statistically significant" number for comparing a series of targets or groups. Unfortunately it is difficult to calculate manually. Unlike a String Measurement, the Average Group Radius does not assume the point of aim is the desired impact point, so sighting error is not a factor in the result. Average Group Radius can therefore be used to compare group sizes when the impact point is intentionally high (rifle sighted for maximum point blank range and shooting dead center). It is also the best measure for evaluating the overall accuracy of loads. Magazine articles using Average Group Radius indicate the writer has taken care to provide readers with statistically significant data, but you should be careful not to compare it to the more common Maximum Spread.


ERROR ANALYSIS METHODS

(Vertical and Horizontal Spread)

The Horizontal and Vertical Spread of a group is simply the greatest distance between shots on the vertical or horizontal plane. This contrasts with the Maximum Spread which can be at any angle across the group. This measurement is understood by most shooters, easy to make, and can be used to help detect load and mechanical problems or "pulling" by the shooter. If the Vertical or Horizontal Spread is significantly larger and shots are well dispersed, it is called "stringing". A cross wind will obviously disperse shots horizontally. Vertical stringing may be caused by irregular powder charges and detonation problems (inconsistent ignition due to variations in primer pocket depth or primer thickness). Improper crimping, bullet inconsistencies or other loading problems usually result in larger overall group sizes, and not "stringing" in any particular direction. A loose gun sight, or broken scope may also cause stringing.


(Average Vertical and Horizontal Error)

The Average Horizontal and Vertical Error is the average of errors on the Vertical and Horizontal plane from the group's statistical center. This is a much better measure for detecting errors than Vert./Hor. Spread because the shot data is averaged to reduce the influence of "flyers". If the goal is to track "pulling" by the shooter over time or to isolate problems, then the Average Vertical & Horizontal Error is a better measurement for comparing a series of targets.


(Maximum Shot Radius)

The Maximum Shot Radius (or maximum group radius) is the distance from a group's statistical center to the center of the most distant hole. It really only indicates how far from the group center the worst shot should fall and is not a good indicator of overall performance. On a target where shots are evenly dispersed, this measurement will be about 1/2 the Maximum Spread and larger than the Average Group Radius. It is best used to quantify the worst shot in a series of targets.


(Maximum Shot Radius Compared To Average Group Radius)
The Maximum Shot Radius should be only slightly larger than the Average Group Radius. A big difference between these measures is another indication of shooter error or a bad load.


(Average Elevation and Windage Error)
The Average Elevation and Windage Errors compare a group's statistical center to the point of aim. It accurately indicates where the group center is located (or average shot impacted) with a set of vertical and horizontal dimensions. The two measurements show how well the firearm is sighted to hit the Bullseye and can be used to adjust sights if the results are converted to Minutes of Angle.

Fire three shots, (forget 5 shot groups), take a set of calipers or a ruler and measure the farthest center to center of the three shots. It ain't rocket science, unless you're anal retentive of course. jorge

Some of those questions are fairly simple: There is absolutely no useful advantage to using AGR (a statistical abomination anyway) over center-to-center distance between the two most distant shots, for n=5.

Some questions are beyond my ability to compute directly, but, I know do how to cheat well, using simulation.

The basic assumption is that you have rid your rifle of all error except truly random variation. That means that stock rubs that could cause shifts have been fixed, etc. The test for this is that you are not seeing any stringing or shifting in your groups. If that is met, then the following holds, and is scalable. That is, for reasons best left unexplained, this data is for a rifle that shoots 1.26" groups, on average. I can get the data for a 1" rifle, by dividing all the numbers by 1.26, or for a 2.52" rifle by multiplying all the numbers by 2.

If you shoot 5-shot groups, 95% of individual groups will fall between .5 and 1.5 X the average. For our 1.26" rifle, groups as small as .63" and as large as 1.89" are normal, and should not be taken as evidence of a difference in performance of the rifle, ammunition, or shooter.

If you then shoot 5 such 5-shot groups, and average their size, you have quite a bit more statistical power.

For our 1.26" gun, the 75% confidence interval is 1.02-1.47", and the 90% CI is .84-1.70". This statement was incorrect. See later post.

What this means is that if you repeat this experiment many times, under identical conditions, 75 or 90% of the time, you'd get an answer within the CI.

I thought I was very clever when I first worked out the .5/1.5 thing for single groups. I later found that Audette had already made the same contribution, long before me.

I fire three shot groups and fire three groups of each load during my initial load testing.If there are two or three loads that are very close as far as group size,I then fire an additional two or three groups with each before deciding on my final load.

The statistical methods of group measurement are academic, and I appreciate them, but for a HUNTING rifle I fire a three shot group at 100 yards and simply measure the extreme spread of the two furthest holes. If I like what I fired I'll attempt to put two more bullets into that group and make it a five shot group. If the group size is about the same I'm a happy camper and the rifle is ready for the field.

If, however, the groups are fired for purposes of reporting accuracy to the internet I suggest firing twenty rounds and only measuring the holes found in a 1/2" circle and report that. I swear that's what everyone else is doing!!!

OOooooops! Sorry folks. I made an error in my first run at calculating the numbers I posted. Please use these instead.

If you shoot five five-shot groups, and average the Extreme Spreads of the five groups, then 95% of the averages so obtained will fall between 75% and 123% of the true mean. In other words, with five five-shot groups, you know true group size within about plus or minus 24%.

If you shoot ten five-shot groups, and average them, then 95% of the averages so obtained will fall between 82% and 117%. So, for twice as many targets, you know the true group size within plus or minus 18% or so.

The numbers I posted earlier were a bit pessimistic... computational error.

A word of caution here. Statistics are great, but before you resort to statistics, use common sense.

There are a couple of rules that govern statistics that can really reduce the need to fire multiple groups. The first rule is that the smaller the variation, the more "robust" the estimates are. For example. If you shoot three groups of .4", you confidence in those groups being "representative" can be much higher than if you shoot a .5, a 1.0 and a 1.9". The latter gives you much less confidence.

So, the more variation you see, the more observations you need to get a robust estimate of the real "average" accuracy.

Also, the number of shots per group makes a big difference. There is a trade-off between additional shots vs. additional information obtained. Speer put the ideal number of shots per group at 7 -- but again, this number is dependent on the level of variation present. If there is lots of variation, you need more shots. FWIW, Dutch.

I shoot 5 shot groups. If I am working up a load, I shoot 1 5 shot group with powder variations of about 1 grain increments. Usually I will find that the groups will start somewhere and get smaller as I shoot heavier loads until they start opening up again.
I then load enough ammo near the best load I had initially, (usually 1/2 grain increments) to shoot 3 5 shoot groups at each loading. I use extreme spread (measure the diameter of a circle that just circumscribes the outer edges of the group and subtract 1 caliber) to determine which load is best. I elliminate from the measured group any "called" flyers.
As you can see, this amounts to a lot of shooting to do this for various bullets and powders, which is why I'm there to begin with!

OK, Denton, why is AGR a statistical abomination?

Because I am a statistician, and don't like it.

Well, actually, here's the reason: Standard deviation contains all the information there is to know about the global variation of a set of data. Also, if you're using SD, there is a whole, huge pile of math that has been worked out for you. You know how to make meaningful comparisions between SD's, you know how to add and subtract them, etc. So, if you're measuring dispersion in a target, SD is the "golden standard" way to go.

When you start doing things like AGR, you give up all the stuff that has been worked out for more conventional measures, like SD. You don't know how to add them, or compare them. Actually, what I should say is that you can compare them, but you have to work out your own math to test whether the difference could easily be from random variation.

For small samples, range ("extreme spread") contains practically as much information as SD. For n=5, it's 90%. Might as well use that. It's easier. Just as important, there is no measure of dispersion that gives a lot more information.

jlongo...

There is a great problem with your approach, that has the potential to lead you around in circles.

A five-shot target is a sample, that represents the long term performance of the firearm. Without changing the performance of the shooter, gun, or ammo, just luck of the draw will commonly allow groups as small as .5" and as large as 1.5" from a gun that averages 1" groups. Any single group size within that range does not provide evidence of real change.

So a shooter bangs off 5 rounds of Brand A, and they group .83". He thinks that is cool. Then he bangs off 5 rounds of brand B, and they group 1.25", and he thinks they are not as good as A. Fact is, he has no evidence that the two brands are different.