I kind of mutated the grapple rules slightly. There's more that depends on some other ideas I had but this is the base.
Brawl 1 Full Action: Unarmed Attack Action
The character tries to brawl an adjacent opponent. They make an opposed Athletics check. If the initiating character wins, he inflicts his unarmed damage and the two characters are engaged otherwise he is sprawled and the brawl ends. All characters within 30' of a brawl also enter the brawl with the disengaged status. In effect, a brawl is like several grapples going on at once with some extra options.
Each character in a brawl is in one of four states: Disengaged, engaged, held, or pinned. Each round, every character in a brawl makes an opposed athletics check against any adjacent non-ally characters. Engaged allies fighting the same character (or team) may make a single check, with a +2 bonus for each ally after the first engaged. The winner may shift his own or any adjacent character's status one step in either direction or he may apply a brawl benefit. If multiple characters make a single check, any status change or brawl benefit affects them all.
Disengaged characters need only make checks if they are targeted by another character in the brawl. They may take any action they could outside of a brawl. If a disengaged character moves more than 30 foot from any brawling character, they lose the disengaged status and exit the brawl.
Example: Dastardly Dan, Evil Ed and Heroic Hank are in a brawl. Oblivious Otto is ajacent to them. Dan, Ed, and Hank all have the engaged status. Dan would like to use Otto as a shield so he targets Otto. Ed is Dan's ally and opts to assist him. Hank does not wish to harm Otto so he targets Dan. Dan, Hank and Otto all make checks, with Dan receiving +2 for Ed's help. If Dan wins the check, Otto becomes engaged and Dan can try to use him against Hank. If Otto wins, he remains disengaged while if Hank is the winner, Otto not only stays disengaged, Dan and Ed will both face whatever benefit Hank chooses.
The brawl benefits available depend on the status of the target.
Engaged, Held, or Pinned StatusInjure: You may inflict your unarmed damage or the damage of a held 1 handed weapon.
Move: You and any adjacent engaged characters move 10 foot in a direction of your choice.
Held StatusAny valid grapple benefit
Pinned StatusAny valid grapple benefit
Bounced out: The character becomes engaged, is thrown 10 foot, and lands sprawled. Any characters in the landing space are sprawled unless they make a DC 15 save. All characters in the landing space take your unarmed damage.
Optional: Brawling areas
The GM may break up the place into linked areas. Each area may have special qualities. In this case, an ongoing brawl takes up an entire area. The move benefit moves between areas in this case.
Example: A brawl takes place in a western saloon. The GM breaks it up as follows
Stage
|
Pigpen----Street----Floor----Stairway----Bedrooms
|
Bar
The GM decides that most areas are normal. On the stage there are several dancers who could be grabbed as shields while the bar contains lots of glass. Anyone can choose to inflict edged damage here. Anyone pinned in the pigpen takes a -4 to Impress checks for the rest of the scene while anyone bounced out of the bedroom takes 1d6 falling damage and ends up in the street.