Progress Report: Independent Cost-Benefit Analysis of Proposed Regulations

On September 1st, 2014, four members of the EA Society of DC met to do a test run of our project to comment on proposed regulations. One of us had heard of a proposed EPA regulation through industry contacts and expressed interest in commenting on it, so we decided to take that one as our test case. We met at a University of Maryland computer lab for a day, and spent a few more hours doing follow-up over email. We submitted our comment, which can be read here, before the deadline.

Cost-Benefit Analysis of Cost-Benefit Analysis

Our comment recommended accelerating the regulatory deadline, on the basis of our analysis which suggested that this would save the world about $1.5 billion in economic costs due to global warming. We spent something like 40 person-hours on this, which – if our recommendation is solely responsible for the recommended change – amounts to about $37.5 million per person-hour invested, fairly impressive even if you discount by a large factor for the probability that we affected the outcome.

What We Did

The proposed regulation was to ban a set of refrigerants with a high impact on global warming, now that feasible alternatives with a lower global warming impact had been invented. There were about seven comments posted, of which we were able to read three. Two were from a manufacturer of frozen margarita-makers, asking if their company would be affected by the regulation, and one was a strongly worded emotional comment from an engineer asking the EPA to ban a different refrigerant as well. We were optimistic that we could contribute significantly to the discourse by performing a competent cost-benefit analysis.

The first couple of hours were spent getting a picture of what exactly were the most important effects of the regulation, which turned out to involve a lot of digging, because the rule covered refrigerants that affect a lot of industries. It turned out that some of the biggest global warming savings were from banning a few refrigerants used in automobiles, so we decided to focus on just this aspect of the regulation.

Once we had a basic framework of analysis, we set about gathering sources to estimate the annual impact of the regulation on global warming, and building our model for what the effect was, noting our sources for data in a Google Doc separately from the Excel doc with the model. Our initial estimate was that the effect of the regulation was strongly positive, and that the deadline should be accelerated.

After we wrote up a first draft of the comment, we started to cite our sources, and couldn’t find the source for one of our numbers. When we tried to re-estimate this part, the numbers we found were different, and the estimated impact of the regulation was negative. By this point it was getting late and we were all tired, so we called it a night.

Later, communicating over email, we realized that when counting the social cost of carbon, we hadn’t correctly adjusted for time. Once we corrected this, the numbers came out strongly positive again. We finished the write-up, satisfied ourselves that the reasoning was correct and backed up by facts, and submitted the comment.

Lessons Learned

  1. Spend a while figuring out the most important few effects of the regulation. This is something we did, which saved us a lot of time.
  2. Make sure someone with access to the venue commits to get there in advance. Two of us were waiting outside the locked computer lab until someone with a key card got there.
  3. List sources, reasons as we build our model, writing the whole thing down in one place, rather than building the model first, then going back and citing sources/evidence later. This would have helped us notice problems earlier.
  4. Expect to have to go back and change our minds later, don’t spend a lot of time making it perfect, give people a chance to sleep on it.


As of now there appear to have been 194 comments submitted. Many appear to be from people in industries affected, but a quick skim of a few of them suggest that most are not related to the auto industry, which is the effect we focused on.

We don’t have a good way to estimate how influential our comment was yet, but the results of this attempt were encouraging. Further iterations of this project seem high-value.

This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to Progress Report: Independent Cost-Benefit Analysis of Proposed Regulations

  1. Kudos! When will you find out if your comment had an impact? (Even if it’s by looking at final regulations and guessing.)

    • Matthew Gentzel says:

      This rule, and another one we commented on are both in the final rule stage. We could hear back from the EPA at anytime, but most likely before the end of 2015.

      The latest possible response I can imagine could be in 2016, however this would not give industry much time to react, and we think this is very unlikely.

  2. Pingback: Solve your problems by fantasizing | Compass Rose

Leave a Reply

Your email address will not be published. Required fields are marked *