I think the time is different for all situations. Depends how long the parent has been gone, where they are and who is in their life presently.
I do believe that if the parent/s decides to leave the child and never comes around then I would say the age as nothing to do with it, its how long the parent is gone.
If the father is gone lets say 3 years and the mother is not around, but the children are living with other family members then at that point, the right for termination is justified.
This should be done with love and fairness as not to hurt the children. If you do it for your own selfish reasons you are only hurting the kids and they are the ones to suffer in the long run.