U.S. margin debt hits another record as leverage keeps climbing, raising the risk of sharper market stress if stocks pull back.