the only time real grappling lasts a long time is if you run into a scenario where you are forced to make the other person submit, and you don't really want to hurt them permanently.
I used to wrestle and play football, so I've had my share of grappling bouts. Basically you want to hit someone hard enough so that they have no time to react. If given the chance, and they are ready for it, there are reactions they can use to counter you. So honestly its a question of surprise/initiative vs preparation. If I had 2 seconds to get ready for someone to attack me, and they were coming at me with the intention of incapacitating me, odds are I would have at least one or two counters for them. I've had matches last for 15 solid minutes in the past, but again, this was when I wasn't trying to actively harm the other person, but put them into a position where they would submit.
Honestly, I think grapple could be a fun dramatic conflict that effectively goes on during a combat round between two characters while the rest of the fight is going on.
The amount of damage scenery and a well placed slam into it can actually do to a person. That scene in Princess Bride with Andre vs Dread Pirate Roberts was very inaccurate to how it would go down in real life. Just playing, I was able to split a friends shoulder open on a drywall corner with little to no damage to the wall itself (and I wasn't even trying to hurt him).