Glass is a more "natural" substance and is not toxic even if it lays around forever. It will not be mistaken for food by various sea critters, as plastic often is. Since glass is essentially fused sand, it will sink to the bottom of lakes and seas, just like rocks and sand.
Furthermore, plastic often leaches its chemicals into food and drink. Plastic water bottles are particularly dangerous.