Passive aggressiveness itself is not an action, so it cannot be a form of violence on it's own. That being said, a person who is being passive aggressive or whom has a passive aggressive personality, usually is not directly violent. That's why it's called PASSIVE-Aggressive. The passive parts breaks away from it simply being an aggression problem.
Now, that doesn't mean that the passive aggressive person wouldn't do anything aggressive enough to be considered violent. It has happened before, though I think at that point it moves away from the description of being passive-aggressive.