Based on the facts described in your question, my advice is:
1. Under Florida law a POA or DPOA (Durable Power of Attorney) does not authorize the "attorney in fact" (your brother) to restrict visiting rights; a POA is to authorize a person to act in behalf of or represent another person. A POA does not appoint a guardian.
2. Whether a guardian for your mother is advisable depends on her legal capacity: Does she have limited mental capacity which seriously affects her ability to make basic life decisions (such as where to live, payment of bills, etc)? In Florida a probate court can determine whether a person such as your mother is incapacitated and requires appointment of a guardian.
3. My question: Why does your brother restrict your mother in this way? He may have a difficult challenge in explaining such restrictions to a court unless he can show such restrictions are for the benefit of your mother.
