The "Pose" actor said producer Ryan Murphy invited him the one time he attended the fashionable fete.