Milk can contain things that are not good for us (some milk more than other milk, depending on the processing), but the fact that it does contain protein, calcium, and Vitamin D means it offers us some very important nutrients. Vitamin D is an extremely important thing, and all kinds of medical conditions have been linked to insufficient intake of this particular vitamin. Protein and calcium are also important, of course.
Fat can be a factor with milk. Children need more fat than adults do. Adults need to watch the amount they take in.
Another factor may be the calories. People hoping to keep their daily calorie intake down may consider milk "not good" for this reason, alone. The person who packs on an unhealthy amount of weight as a result of drinking too much milk might say that milk was not "good for him".
Any hormones, chemicals, or bacteria present, or potentially present, in milk are, may be either bad for us or potentially bad for us. Considering the amount of these in any milk, and generally weighing the benefits of nutrition against the potential risks of "questionable" substances, can help us determine whether milk's nutritional benefits make it "good for us" - or just "better for us than not getting the same nutrients through other foods".