Many older neighborhoods have a mix of apartment buildings, single family homes and luxury homes in close proximity. As demand went up, large luxury homes were turned into apartments.
The delineation you see in neighborhoods today is due to strict zoning rules, which we probably should loosen, if only to allow for more people to live in the apartments above older businesses to create more affordable housing in downtown.
What I disagree with is rent control or set-asides for "low income" housing, since developers either offset it by only building condos for the wealthy or housing costs for everyone else go far up while fewer units are created. Both of these trends hurt the housing supply that makes it harder for those who are not wealthy to find housing.