There are other factors that can cause you to lose more than this, and in some cases the hairs don't grow back. The most common cause is due to male pattern baldness, which is hereditary, and caused by specific sex hormones.