BioCycle October 2006, Vol. 47, No. 10, p. 21
A comparative analysis method was developed to evaluate the accuracy of the State of Garbage Survey’s 2004 recycled tonnage data and national recycling rate.
Phil Simmons, Scott M. Kaufman and Nickolas J. Themelis

IN APRIL 2006, BioCycle published the findings of the 15th State of Garbage In America, a national survey on the generation and disposition of municipal solid waste (MSW). The 2006 survey initiative, which utilized data from calendar year 2004, continued the collaboration between BioCycle and Columbia University’s Earth Engineering Center (EEC). The 2006 survey used the new tonnage reporting methodology established during the 2004 State of Garbage Survey (see “State of Garbage In America,” January 2004). Each survey effort has proven to be a learning experience. This article addresses recycled tonnage data gaps that became apparent during the survey and the data analysis processes that were developed for filling these gaps.
Recycled tonnage is the focus of this discussion because it represents the most obvious area of inconsistency among states’ data. (For the purposes of this article, the term “recycling” refers to materials recovery for recycling (e.g., paper, metals, commingled containers) and composting (e.g., yard trimmings, other organics.) Many states estimate large portions of their recycled tonnages since they do not get hard data from weight scale receipts, as is typically the case for waste-to-energy (WTE) facilities and landfills.
The uncertainty regarding the actual amount of materials recycled nationwide is heightened by the varying methods employed by many states to estimate recycling tonnages (e.g., many states estimate recycled tons based on historical data or waste composition studies, both current and dated). Some states in the 2006 survey underreported recycled tonnage by not including certain categories, such as composted tonnages. Also, in a few instances, states reported unusually high amounts recycled for certain categories, such as metals or wood. In those cases, it could be likely that non-MSW materials such as auto scrap or construction and demolition debris (C&D) were included in the MSW recycling rate calculations. (“Non-MSW” includes recycled or disposed materials that are outside of the US EPA definition of MSW.)
Because of these uncertainties regarding recycling tonnages, EEC developed a method of comparative analysis of EEC-estimates of recycled tonnages and the State of Garbage (SOG) Survey results. The objective was to get a sense of how large an impact the inconsistencies in the recycled tons reported by some states may have on the national recycled quantity results reported in the SOG survey.
ANALYTICAL METHODS
The comparative analysis components developed to analyze recycled tonnage data and their impact on the national recycling rate are as follows:
Comparison Metric Creation: The comparative analysis uses a typical standard of measurement, or metric, to compare recycled tonnage data from around the nation. This metric, per capita recycled tons, was calculated by dividing the 2004 tons recycled in a given jurisdiction, such as a county, city or state, by the 2004 population of that jurisdiction. This simple metric of annual tons recycled per person allows for comparison of recycled tons across jurisdictions that have different size waste streams, recycled streams, and populations.
Meaningful Reference Range: A reference range (comparison range) of per capita tons recycled was created to which the SOG Survey data could be compared. The reference range was developed from the averages of five sets of per capita recycled data, as listed in Table 1. Averages were calculated by dividing the total recycled tons for a data set by the total population of the data set.
As shown in Table 1, the first data set was based on the SOG data as reported in BioCycle (April 2006). The second data set was developed by screening out from SOG what appeared to be nonrepresentative “outlier” data – per capita recycled rates from places like Washington State with very high amounts recycled per person, and Oklahoma with very low amounts recycled per person – in order to make the reference range more representative of the typical per capita recycled rates in the nation. To provide a conservative lower bound of the range, the SOG Survey was screened only of high outliers (i.e., the extremely low per capita recycled rates were not screened) and an average was calclulated from this third data set. The fourth set includes data from counties, municipalities, and solid waste authorities in seven states. The county data was obtained as part of two projects separate from the SOG Survey being conducted to investigate recycled tonnage data management issues in EPA Regions 3 and 9. In the fifth set, the county data were screened of low and high outliers.
Since the reference points were based on averages of several data sets, they suggest a reasonable range of what levels of recycling, as measured by per capita recycled rates, may be achievable on a national scale. The low end of the reference point range is roughly at the 49th percentile of 389 available per capita recycled rates developed from counties in Maryland, Minnesota, New Jersey, Oregon, South Carolina, Virginia, and Washington (chosen because these states publish fairly detailed, tonnage-based data from all counties), and including the state data from the SOG Survey and data set averages. The high end of the reference point range is close to the 75th percentile (Figure 1). The mid-range reference points represent calculated averages from remaining data sets listed above.
Hypothetical Scenarios to Translate Per Capita Recycled Rates to Tons and Recycling Rates: The per capita recycled rates were used in two ways to create hypothetical recycled tonnage and recycling rate scenarios. First, the total set of 389 per capita recycled rates were applied to the national population of 293 million to produce a set of national amounts recycled along with national recycling rates (Figure 2). Second, the five average per capita recycled rates from the reference range (see Table 1) were multiplied by populations of those states that had significant uncertainties in recycled tonnage reporting for the SOG Survey. These new, calculated rates for particular states were combined with the SOG recycled tonnage from those states where the reporting was considered strong to yield a hypothetical national amount recycled, recycling rate, and total MSW generated (Figure 3). These hypothetical scenarios were plotted as graphs (Figures 2 and 3) to aid the comparative analysis – a method of comparing “what if” scenarios – that shows how the national calculated recycled tons and recycling rates change as the per capita recycled tons vary.
RESULTS AND DISCUSSION
As depicted in Figure 2, the application of variable per capita rates to the total U.S. population yields a large range of tons recycled and national recycling rates. To narrow the focus from the entire set of hypothetical scenarios – many of which are clearly unrealistic – the reference points (i.e., the average per capita rates from the State of Garbage and county data sets) and their resulting scenarios have been highlighted as the “comparison range.” The comparison range represents the “ballpark,” or the range of recycled tonnage and recycling rates that could be considered reasonable. Since the landfilled and WTE tons are considered reliable and were held constant, as per the SOG Survey results, the changes in total generated (i.e., recycled tons + WTE tons + disposed tons) mirror the changes in recycled tons.
The ballpark (Figure 2) created by applying the reference per capita rates to the national population is fairly large, from 80 million tons recycled (22.5 percent recycling rate) to 126 million tons recycled (31.2 percent recycling rate). In comparison, the SOG survey showed 110 million tons recycled (28.5 percent recycling rate; red vertical dotted line in Figure 2). The per capita averages from the screened data sets of Table 1 bracket a narrower spread – 88 to 108 million tons hypothetically recycled.
As a final test to narrow the comparison range, the reference point per capita recycled rates were applied to only those states considered to have significant uncertainties in their recycled tons reported (Figure 3). The base – recycled tons from those states with strong reporting – was kept constant as per the SOG results and represents 54 percent of the national population and roughly 63 million tons recycled. (The criteria used to categorize the “certain” and “uncertain” states were arguably subjective, but the process is insightful because it helps narrow the comparison range by making sure that states supplying unquestionably good data sets were accounted for in the hypothetical scenarios.) As shown in Figure 3, these scenarios yielded a range from 100 to 121 million tons in recycled tonnage and recycling rates from 26.5 percent to 30.4 percent.
BROADER APPLICABILITY
The following discussion expands on how these findings apply in the context of the total U.S. population and for states with limited recycled tonnage data available:
Per Capita Averages Applied to the Total US Population (Figure 2): Applying the reference point average per capita rates to the national population is a rough but useful tool that yields a very large – 46-million ton – range of material recycled on a national scale. The tool is rough because the method ignores the recycled material that was reported with certainty from a number of states. By ignoring the recycled material amounts from states that certainly have high per capita rates, the low end of the comparison range could be considered far too low. Conversely, by ignoring the material of those states with certain, low per capita recycling rates, the high end of the comparison range is likely too high, hence the very large comparison range of calculated national amounts recycled. These issues can be addressed to some degree by creating scenarios keeping the “certain” states’ recycled tons constant and varying those states with uncertainties.
Addressing State-Specific Uncertainties with Per Capita Averages (Figure 3): As shown in Figure 3, the SOG Survey results fall within the comparison range, suggesting that the SOG results were within a reasonable range. The differences between the Survey and the low and high ends of the comparison range are roughly 10 million tons and 11 million tons, respectively. The percentile ranking (Figure 1) shows that the comparison range contains the center of the available per capita data set and is skewed toward the high end. However, all of the comparison range reference points are roughly between the 50th and 75th percentiles, with the SOG Survey results lying at the 67th percentile.
The low end of the range, with 100 million tons recycled and a 26.5 percent recycling rate, represents a conservative estimate because the per capita rates were calculated by screening (excluding) the high outliers from the data set. Low outliers, even those with 0.0 tons recycled per capita, were included in the calculated average. Relative to this low end of the range, the SOG Survey tons recycled results would be about 10 percent too high.
The high end of the range, with 121 million tons recycled for a 30.4 percent recycling rate, could be considered overly optimistic. The per capita recycled rate in this case was calculated from the county data set without screening any outliers. The states from which the county per capita data were calculated have strong recycling programs. It is possible that the calculated average rate may not be representative of other parts of the nation where recycling programs are not as strong. Relative to this high end of the range, the SOG Survey results would be about nine percent too low.
Logical arguments also could be made that the two screened data sets should represent the comparison range. This interim range has its low end calculated at 103 million tons, by screening the outliers from the SOG Survey results. The high end of the interim range was calculated at 113 million tons, by screening the outliers from the county data. It should be noted that the county data were obtained independently of the SOG Survey and that the data set is large – 300 data records vs. the 50-record State of Garbage data set. Recycled tonnage data from the individual reporting jurisdictions were not adjusted to exclude non-MSW. In any case, the Survey results could be considered either seven percent too high (relative to 103 million tons at the lower end) or three percent too low (relative to the 113 million tons at the high end).
CALIFORNIA EFFECT
A conservative factor was introduced into the comparative analysis by the inclusion of California – representing 12 percent of the national population – as a less than certain state. California arguably has a higher than average per capita recycled rate, but because the state did not report tonnages for the SOG Survey, only a 48 percent diversion rate, it is difficult to determine how many tons were actually recycled in California in 2004. By back-calculating from the reported 48 percent rate and from the sum of landfilled and combusted tonnages, an estimated 31 million tons were recycled in California in 2004. However, the SOG based its final estimates on actual tonnage reports from recovery facility surveys (yielding 22 million tons recycled) because the 48 percent recycling rate seemed to include a significant amount of non-MSW (e.g., automobile scrap).
Figure 4 shows the effects of using the comparison range per capita reference points for California, keeping all other states as reported in the SOG Survey. Depending upon the reference point used, the effect of estimating California’s recycled tonnage could affect the U.S. recycled stream by up to 21 million tons, or the national recycling rate by 3.9 percentage points. Had California been classified as a “strong reporting” state and included in the base, the effect would have been to increase the comparison range reference point scenario results by roughly 10 million tons.
The blue bars in Figure 4 show the variability in California’s estimated recycled tons if different per capita recycled rates from the comparative range are applied to California’s population (columns 1-5) and the resultant changes in the California recycling rate (white boxes) and the U.S. recycling rate (red boxes). The per capita rates in columns 6 and 7 represent the California recycled tons using the SOG results and the 48 percent diversion rate reported by the state, respectively. Figure 4 also shows the corresponding changes in the U.S. recycled tons and the national per capita recycled rates.
Conclusions
The comparative analysis provides a method to evaluate the accuracy of the State of Garbage In America Survey findings. The method essentially provides a comparative framework for examining if the results are within a presumably acceptable range of results. The methods used are inherently conservative as they are based on averages, which tend to mitigate the influence of states with high recycled tons per person on the estimation methods. California would seem to provide a good case in point, as it has an impressive recycling infrastructure and a large population; however, due to uncertainties in the reporting of California’s recycled tons, the methods used in this study assign average per capita recycling rates in the state in order to estimate the national recycled tonnage. If California’s per capita recycled rate is actually higher than the national averages used, then this comparative analysis is probably too conservative and underestimates the national recycled tons and recycling rate. The opposite would be true if California’s per capita recycled rate is actually lower than the national averages used.
Uncertainties aside, this study leads to the following conclusions:
The comparison range of hypothetical scenarios suggests that the amount of MSW recycled in the United States in 2004 ranged from 100 million to 121 million tons, corresponding to recycling rates of 26.5 percent to 30.4 percent. A more narrow focus suggests that the comparison range is 103 million to 113 million tons corresponding to recycling rates of 27.2 to 28.9 percent.
The SOG Survey resulted in 110 million tons recycled for a 28.5 percent recycling rate. Relative to the extremes of the comparison range scenarios, the State of Garbage recycled tonnage results could be off by about 3 percent (113 million tons – 110 million tons) /113 million tons = 0.027) to roughly 10 percent (110 million tons – 100 million tons/100 million tons = 0.100).
Applying the same methods and hypothetical scenarios based on the per capita recycled rates for California, the comparison range is very similar to the national hypothetical scenarios (a range of 98 million to 119 million tons and corresponding recycling rates of 26.2 to 30.1 percent).
The State of Garbage survey used a recycled tonnage for California that translates into a per capita recycled rate that is higher than nearly 90 percent of all the 389 per capita rates available to this study. This suggests that the SOG estimate of 110 million tons recycled nationally is too high.
The reference points based on the screened State of Garbage data sets – which assign average per capita recycled rates to California – yield scenarios of 100 million and 103 million tons recycled for the nation with 26.5 percent and 27.2 percent national recycling rates. They are considered to be more representative of the national recycling picture.
Further work is needed to determine the impact of uncertainties in the data reported for disposal. This study focused only on recycled tonnage, and recycling rates can be dramatically affected by amounts that end up in the denominator of the rate calculation [(total recycling) ÷ (total recycling + WTE + landfilling) = recycling rate]. Furthermore, the State of Garbage in America Survey has traditionally tracked only MSW and excluded large portions of the total solid waste stream such as C&D, auto scrap, and industrial wastes that are not part of the U.S. EPA’s definition of MSW and yet often end up in MSW landfills or counted as “recycled MSW.” The varied exclusion or inclusion of these waste categories from state to state, as well as the accounting for imported and exported wastes, also present significant challenges to national waste stream characterizations. Many of these issues will be investigated in ongoing and future EEC projects, which, along with this study, aim to broaden the knowledge and understanding of how the nation handles and tracks its recovered and landfilled solid wastes.
Phil Simmons is completing his thesis for a Master of Science degree in Earth and Environmental Engineering at Columbia University in conjunction with research at the Earth Engineering Center (EEC). He is also the primary author of the BioCycle/EEC April 2006 State of Garbage in America and a senior project scientist with HydroQual, Inc., in Mahwah, NJ. The support by HydroQual of Mr. Simmons’ studies at Columbia is gratefully acknowledged. Scott Kaufman, primary author of the 2004 State of Garbage in America, is a research scientist in the EEC of Columbia University, where he is pursuing his Ph.D. in Earth and Environmental Engineering and serves as the director of the Recycling Research and Technology Center. Nickolas J. Themelis is Director of the EEC and Stanley-Thompson Professor Emeritus, Earth and Environmental Engineering, at Columbia University.