I definately say be patient. Smoking is a chemical and mental addiction, and just as hard to quit as any other drug.
My Husband and I quit when we decided to start trying to get pregnant with our first child. I swear I thought we were going to kill eachother!!!

It is tough and he needs your support. From an entire pack down to 5 is significant. And let him know you are proud of him for that.
I definately agree that smoking should never be done around children, and when he sees your baby I am sure that will be just the motivation he needs.
Stress is not an excuse, it is a valid trigger and it will take him time to learn to deal with his stress in a different way.
And in his defense you met, fell in love with, and decided to have a child with a smoker. While the reason is valid, you are still asking him to change and it will take time.
Good luck and congrats on the baby!