Recently, after I blogged about the list of approved animals to hunt with a semi-automatic in PA, I submitted a Right to Know Law request to the PA Game Commission.
I sought records which included the survey itself, any and all responses, and documents relating to the selection of those who would receive the survey, information (age, county) of the individual completing the survey.
The cost to produce the records I requested would be almost $400, as there were an estimated 1,473 pages of records that were responsive to my request. I was informed that approximately 75% of the respondents returned the survey via the USPS, which accounted for the cost associated with the production of the records.
The Commission did furnish me with the responses that were returned electronically (523 responses). Please note, this only accounts for 1/4 of the total responses received.
I sent the responses I received to a fellow member on AR15.com who is more well versed in decoding the information provided. He responded that “the sample, at least the observable portion, is rather skewed geographically and by age.”
He found that within the partial dataset there was a high degree of correlation between the ownership of a semi-automatic firearm and support and a high degree of non-ownership and strong opposition. Out of the 523 respondents that I received results for, less than 40% owned a semi-automatic rifle. It was further explained that opposition increased with age as well as counties that had the highest opposition rates also had the highest rate of response.
He also noted concerns relating to the sample population. In order to determine whether a representative sample of PA hunters were surveyed, we would need more information such as a summary detail on the CID pool from which the sample was drawn and populations by county with counts by age.
As for the methodology, I was told that the survey recipients were chosen via a SQL Server Reporting Services (SSRS) report that utilizes a data source based around a T-SQL query that makes use of the built-in NEWID function. Since, that stuff is a bit beyond my knowledge scope as to technology and databases, the response may have well have been in french.
The was it was explained to me was that, the “function assigns each distinct Customer ID (CID) number within the given parameters a randomly generated globally unique ID number. An example of this ID looks like 6F9619FF-8B86-D011-B42D-00C04FC964FF. These Customer ID numbers are sorted by the generated ID. The desired number of CID’s are taken from the top of the sorted list. These randomly select CID’s are then joined with the associated demographic information.”
As my technical resource notes
The sampling method used, (randomly assigning a Global ID and sorting in alphanumeric order) is inherently flawed, as there is no way to ensure that the resulting sample is representative of the hunter population as a whole. The preferred method used throughout market research would be a “stratified random sample.”
By using county and age as strata, the PGC would have ensured, with 99% confidence, that the sample group was completely representative of the 900,000+ PA hunter population.
Choosing not to use this method, in conjunction with the vague statement “within given parameters,” indicates incompetence at best, or intent at worst.
I’d like to remind readers that I am only working with a portion of the data that the Commission received. Without the rest of the information, we are only looking at a small portion of the picture. But it does seem to raise questions as to the validity in which the survey was conducted.
Did you find this blog helpful or informative? Be sure to share it with your friends by clicking the buttons below. Don’t forget to like Firearms Industry Consulting Group and Prince Law Offices, P.C. on Facebook by using the “Like” button to the right.