One of the primary ways we have damaged the Ozone layer is from emitting Chlorine compounds that move into the upper atmosphere. Some fireworks do use the Chlorine compound Hexachlorobenzene (HCB), which is a pretty nasty substance, but that doesn't necessarily mean it destroys ozone, moves into the upper atmosphere, or is in enough quantities to be noticeable. It is probably possible, but I don't know that anyone has studied it closely enough to have any idea one way or the other.
Aside from the Ozone Layer, the issue of pollution from fireworks is worth looking into. Exploding toxic substances into fine particle aerosol clouds a few hundred feet above our heads is the absolute worse thing you would want to do with toxic substances. Along with nasty stuff like HCB, fireworks can include all sorts of heavy metals as well as things like arsenic and barium.