It wouldn't occur to me that this would be an issue. To me, when you love children you're happy for everything and everyone in their life that adds to their life. When someone else loves the people we love it's a nice thing - not a reason for struggle or "issues". When it comes to nieces or nephews, thinking this way is a kind of "elementary-level version" of giving children the "wings", referred to in the saying, "Give your children just two things - one is roots, the other wings." That saying, of course, is generally aimed at a different situation; but the idea that children's world should include more than just us is, in a way, a version of "wings".
So, to me, we should enjoy the fact that children have lots of special, close, people in their lives. It should always be about what's nicest/best for the children - not us. If, by any chance, "scheduling problems" arise I think adults can usually work things out.