It depends upon the Order or agreement which gave joint custody 5 years ago. Do not violate that order (if it is a court order) but otherwise, yes you can keep your child. You should talk to a local attorney about the specific Order or agreement in your case.