The answer to this question will depend on the circumstances surrounding the removal of the children and if your grandfather obtained legal guardianship as a result of this removal by child protective services. If this is the case, you may need to take matter to court. Otherwise, you may want to discuss his will or make arrangements with other family members until you can resolve it legally.
