You are correct in stating that no attorney is going to advise you to violate a Court Order. However, as the above attorney stated, we are in new territory. He may be exercising appropriate judgment. I recommend speaking with him as to the risk to which he is exposing the children, as well. If he will not cooperate, you may need to file an emergency motion for modification of placement.

The answer is nobody knows for sure because this crisis is so new. As always in custody matters, the issue is what is in the child's best interest. You have presented several reasons why your decision is in the children's best interest as well as the best interests of protecting them and society from infection with Covid.
Given the circumstances, I cannot tell you for sure that you would not be held in contempt if you keep the children, but I suspect the courts would favor caution unless it looks like you are using the current troubles as an excuse to defy the order. My suggestion is to tell or email the father the fact that you do not believe it is appropriate to go ahead with visitation at this time and give him your best factual support for your decision. Your idea to offer him make up time is good. I would also suggest offering him telephone or video chats with the children. Given how fast things are changing, it does not make sense to try to make a long term agreement. If he is dissatisfied it is up to him to seek to enforce the order. At least in my state the courts are only taking up emergency matters and losing out on visitation is generally not an emergency.