Where do we go from here?

Having had almost the exact same thing happen in my marriage, just give him time, and try not to have blow ups. Notice that he said "if you want to leave go", not "I'll be packing my stuff and leaving when I get back". So he's still there, just needs some cool down time. It was hard, my wife had a few moments where she really lost it early on and I reacted quite like he did. but eventually we both moved past it, and I think we are happier now than we've been in the whole almost 16 years of our marriage