As soon as you use pure black or white, you have nowhere else to go. Consider painters who when using pure white, will reserve its use for parts of the picture they want to highlight, like reflected highlights in a subjects eyes for example. If the same level of white had been used elsewhere in the picture then the subjects eyes will not be the part of the image that pops, since it has to compete with those other areas where the same value is present.
This principal applies for black also, if you use pure black elsewhere in the picture you can't create another area that will be darker, so if you want a specific area of dominant darkness in the image then your darkest black should be reserved for that area only. This works for any level of dark or light, it's all relative to the work and the pallet being used.
The above dramatic light and dark composition from Citizen Kane contains no absolute black or white, which means brights can get brighter if need be, and dark's can go darker. Notice that the image does not lack impact, we get the strong composition created by the contrast of dark and light with the added benefit of being able to flesh out detail in the dark and light areas. This ability of seeing into the image is lost when the dark and light areas are pushed to extremes because both white and black at 100% are impenetrable.
Here there are some almost pure blacks around the wheels in this image of a glossy black car, but nothing even close to pure white. As you can see the reflected highlights stand out just fine.
Rob Adams, he re-works a painting of a field at dawn to make it nocturnal. Since the original painting contained no values close to the ones that are used in the reworking, the effect is dramatic. It's a good example of how a reserved color palette gives artists room to move should inspiration push them to go further.
Here we see no blacks or whites at all in this design, yet in the universe of the image itself, the pale orange substitutes for white, and the dark purple stands in for black, mostly our world resembles this kind of color relativity.
In print graphics the use of pure white is common, because these areas are actually transparent and left for the paper stock, as such the modulation of the stock tends away from pure white, indeed it's probably impossible to have a pure white paper stock even if it's advertised as such. Likewise when solid black is used, it is absorbed by the paper and loses a percentage of it's value.
The digital medium is less forgiving, true that a pure white computer screen is not as blinding as the sun, but it is illuminated and therefore white mean white.
At the beginning of this post I said this rule was meant to be broken, there are times when pure white and black makes perfect sense, I'd only suggest it not be your default position, like everything in design, it should be a conscious decision.