It depends on where they're playing outside, how old they are, whether there are other children around (how old any of them are), what's around (like water), how likely it is that something like a bear or coyote may show up (in some areas that's not at all very likely). It also depends on what, if any traffic, there is and whether there are, say, parked cars little kids may go near.
It just depends on too many things, and there are just too many different individual situations (even with only one child at any given time). If someone has, say, two preschoolers (a four-year-old and maybe a two-year-old) keep in mind that some four-year-olds may get the "bright idea" to do something like tell the younger one to eat dirt or grass (or something like that); so sometimes one four-year-old or three-old by himself in a very safe yard is safer than if there were more than one child.
I think most mothers know their own child/children and their own yard or living circumstances. They just have to use good judgment and do what seems most sensible. Keep in mind that as long as preschoolers have the chance to play on their own (in the house, out of the house, whatever) much of the time, they don't see being with, going along with, their mother as the negative thing that older kids (or immature adults who don't know children) may.
Being a mother isn't high school. You can't worry about someone else saying/thinking you're not cool or calling you names. There's a big difference between "protecting" and "over-protecting", and some people (particularly those who don't know much about child development and children in general) are just really "paranoid" about adequately protecting children without "over-protecting" them; and so often, as a result, it's the child who pays the price.
Keep in mind that while there is such a thing as "over-protecting" children (to the point that, say, mental-health professionals would recognize it as "unhealthy"; so often, "over-protecting" is nothing more than what is seen by people who can't, won't, or don't know how to sensibly and adequately protect their own (or sometimes other people's) children.
I think, though, if you really feel it's necessary to bring them in "even for a minute" (and I'm not second-guessing), you need to be careful about not making them feel as if it's not safe to be outside in their own yard (maybe by saying nothing about safety and instead passing off the trip inside as, say, a "little break").