A number of years ago, while goofing around with the classic Super Mario Bros., my brothers and I decided to up the challenge factor of the game by forcing the player to constantly hold both the run button and the D-Pad right, so that Mario would be forced to continually run. This left the player with only the options of jumping or ducking. We dubbed this new variant of game, Super Mario Bros. Challenge Mode.
This way of playing the game made it much more difficult but at the same time it was a lot of fun. We kept passing the controller around to see if we could make it further in the game than the person before. I later introduced Super Mario Bros. Challenge Mode to some friends at the office, and they too found it quite fun to play.
It got me wondering, what was it about this way of playing the game, that even though it was much harder than traditional Super Mario Bros., it was also tons of fun to play. As I played and I found that whenever I would die, I could instantly see what I had done wrong and how to correct it on the next try. I was blaming myself for my failure, not the game. As I continued to play the game and get further with each attempt, I could feel myself improving and this was the root of what was fun about the game.
This to me stood in contrast to what is commonly believed in some game design circles, that a difficult game is inherently frustrating to players. I came to the conclusion that it was not the difficulty alone that causes frustration, but the lack of understanding of what causes failure. If the player understands why they have failed and how to not fail in the future, difficulty can in fact be fun.
I think this accounts for the popularity of arcade games like Pac Man or Space Invaders. The games were designed to be difficult to force players to spend their quarters, but the reason the players kept coming back was because it was easy to tell what they had done wrong. This type of difficulty design has seen a bit of resurgence in retro style games like Geometry Wars and Super Meat Boy, however, I think this lesson can be applied to all games.
In the end, game designers need to better communicate the challenge in their games. If the challenge is communicated properly players will not be frustrated by it and will in fact enjoy it.