I'm double majoring in biological anthropology and criminal justice, and I love both majors. I enjoy most of the classes, but a handful of required classes are a drag (req. math, some labs, supporting courses). Sometimes its the subject or the professor is lousy. But I enjoy them very much overall.
Engineering is a tough major. I know engineering majors at my school, whenever they tell people what they're majoring in they get a sympathetic "ouch" look or comment on how much math that must involve. And yeah I would imagine difficulty only increases in the subject. A lot of students I know either hate the major but keep going because its a solid career field with good money, or are deeply passionate about their engineering field and can talk for hours on the topic (I have a crush on a guy who does that lol).
But if that's not what you are planning to do, then its probably a bad match for you. If I can ask, what kind of career do you want?