How Narrow a Network is Too Narrow?