Luke Summerfield converted from the spot in the 52nd minute after Rory Fallon had opened the scoring to end a six-match wait for a Championship win.
But it was the award of the penalty which was the chief talking point after the match.
Referee Roger East played the advantage as Paul Gallagher was floored crossing from the right to Fallon.
The striker's first-time shot was blocked by recalled Saints central defender Olly Lancashire, but East ruled Lancashire used his arm, resulting in protests from the visiting defender.
Lancashire, keeper Kelvin Davis and right-back Rudi Skacel were all cautioned before Summerfield sent the keeper the wrong way from the spot.
Argyle boss Paul Sturrock said: "I must admit I would have felt aggrieved if that one had gone against us.
"But you take what you get and obviously there is a relief around the place that we have now got that win under our belts.
Saints boss Jan Poortvliet felt his side had been hard done-by.
"It is unusual for my players to react like that," he said.
"If they think it is a penalty they would not have said anything. They felt hit was Olly's head the ball hit, not his hands."
The Saints' Dutch boss was pleased with the way his side kept going despite trailing 2-0.
Poortvliet said: "We still tried to play football and we hit the post. We did not create that many chances today but I thought in the second half especially we tried playing football."
Saints striker David McGoldrick hit the post with a curling 30-yard shot just after Argyle had scored their second goal but still home keeper Romain Larrieu stayed unbeaten.
Larrieu made a tremendous 25th-minute save to deny the striker with the score at 0-0 after recalled Ryan Smith's cross from the right had taken a wicked deflection to sit perfectly for McGoldrick's powerful header.
That save came seconds after Davis made a brilliant double stop.
Chris
He then used his reflexes again to somehow claw away Fallon's header as he followed up on the rebound.
Those two saves suddenly injected much-needed excitement into what had been a dour south-coast struggle.
And the match turned when Fallon powered home a 38th-minute header from six yards following debutant Craig Noone's cross from the left.
Sturrock was relieved to see his side finally claim three points.
"We needed a win," he said.
"I still think that football is a strange old game though because I thought we played better when we lost to Birmingham and in the second half against QPR than we did today - and yet from those two home games we only got a point."
Sporting Life