"Safe" is relative to what standard? Assuming all genetic modification accomplishes is speeding up genetic inheritance of a trait which could have come about naturally, albeit more slowly, then, relative to existing genetic evolution, speeding up the process is "safe". Can we trust researchers motivated by grant funding, career prospects, the promise of fame and fortune, etc.? Again relative. The less these are compromised for the researcher, the more likely they are to be transparent about their activities and results.
Should we worry about frontier biotech technology like genetic engineering, stem cell therapy, cloning, etc. Yeah maybe, you never know whom might get the upper hand on technology which might morph the direction of human evolution for the better or worse permanently. Can banning such technology ever by successful long term? No. Individuals vary widely, but humanity as a species will continue to be unyieldingly curious. Given time, every conquerable puzzle offering conceivable leverage towards the benefit of humanity will eventually be conquered. better to be receptive towards change and anticipate responsible methods for managing and regulating it, then to trust a ban will squelch it into the quagmire.
History shows time and again, no amount of suppression will permanently discourage pursuit of solutions to viable scientific puzzles.