While certainly not part of Museum Mile (which is on 5th Avenue, 10 blocks north of here) the inclusion of the Whitney Museum of American Art makes it easy enough to consider these blocks part of the greater "Museum District". Lined with ornate buildings ranging from churches to retail stores, the block is masked in the beige wash of limestone.
Madison Avenue, especially on the Upper East Side, wouldn't be itself without a slew of high-end stores (and a shuttered Sharper Image) to entice walkers. All of this makes the unique Whitney Museum building all the more interesting. Finding the strange, avant-garde museum amongst the classic architecture is a bit like finding a spoiled hipster kid in Spanish Harlem: It's an interesting juxtaposition to observe, but you're not so sure it's the best place for him to be.