Residents of hundreds of inexpensive rental units in a longtime black and Latino neighborhood will have to look for new housing, and many fear they can't afford it.