Angelina Jolie Hasn't 'Sworn Off' Men — But It's 'Hard for Her to Trust' After Brad Pitt: Source