You should try and give the pristiq as long as you can. Sometimes it takes a bit of time to really notice the potential positive effects on your functioning that meds can have. The longer you are on it the more perspective you will have on its effects on your psyche and functioning levels. Things may not get any better but these thorough observations can really help you to zero in on a med that works best for you in the future by fully understanding each meds effects on you and relating this to their mechanisms of action. If you haven't yet I would start keeping a journal that documents the effects you feel from each med in painful detail.
Many people think that their psychiatrists are perfectly adept at finding the best med for them and thats why they are paid so well. The fact is they are pretty much next to worthless compared to the (very well informed) treated individual's ability to effectively zone in on a med that will be effective for them (provided the treated individual is not too significantly.... messed up). You just have to do a very good deal of research and reading to sufficiently understand the very limited knowledge of modern psych meds.
I would give pristiq another week or two and then go back to xanax if you don't feel any positive benefits. There are a crap load of meds you have yet to try. It shows your intelligence that you already have recognized that there are almost always going to be some amount of trade offs instead of just damning all meds to hell just because we must take some negatives along with their positives.