In your divorce agreement you should have visition specified. If you want to alter that you need to bring a case in Family Court.
The standard for altering a visitation arrangement is that there must be a change in circumstance and the change must be in the best interest of the child.