Spot on Kicker!
It sounds like UFE's and the resulting frustrations are what's causing you to implode.
My guess is that you warm up well and start strong but then after a few games you make a few mistakes. It's at this point that you remember your history against this type of player and you start to get tight which results in even more errors. It's a downward spiral from there as you totally forget the sweet warm up and great start to the match. At some point your head completely explodes. Ahhh, I know the feeling well.
Kick has proposed some excellent solutions. In addition to forcing the 'pusher' to play shots out of his comfort zone, I think a bit of pre-match preparation might boost your chances as well.
Mental preparation: You know what the challenges are (minimize UFE's and reduce the frustration level) and what your solutions are (which you've practiced of course!) for overcoming the challenges. Take 10 min. before you play and during change overs to review those plans.
STEEL YOURSELF FOR THE BATTLE. You're tough, mentally and physically. You have what it takes to win. Sure, you'll make some mistakes but you're going to learn from them and come back even more resolute. You may not win, but winning is irrelevant. The only thing that matters is that you give your best effort. Every....single...point.
I'd also suggest playing against more than one opponent on a regular basis.
Here's a good formula that worked for me.
Develop a pool of opponents from 3 skill levels.
The first group you can beat fairly easily. This allows you to try new tactics and strokes without feeling pressured.
The 2nd group is players that you're evenly matched with. I thought of this as the 'barometer group' (am I improving over the course of a year).
The 3rd group is players that are a level ahead of you. This was the group that I learned the most from though I NEVER won against these players.
It takes time to develop such a pool of players, but it is well worth the effort.
It can be as simple as starting a Player Contact List amongst your friends and adding to it as you all find new/old players in your area.
Good Luck with the pusher. You will find him at nearly every level of the game.