Two goals in two minutes midway through the second half took Oldham to the top of the Second Division table and extended their unbeaten away run to nine games after they were rocked by the home side's dramatic start.
David Eyres' 74th minute penalty and Wayne Andrews' volley immediately afterwards salvaged a point for the Latics after Ian Stonebridge and David Friio had put Plymouth 2-0 up within seven minutes.
Plymouth showed three changes from the team defeated 2-1 at Tranmere seven days previously, handing a debut to goalkeeper Danny Milosevic on loan from Leeds United, in place of injured Roman Larrieu and recalling striker Stonebridge and midfielder Ossie Lopes.
Tranmere manager Iain Dowie stayed faithful to the starting line-up which knocked Premiership side West Ham out of the Worthington Cup on Wednesday despite having Darren Sheridan available again after injury.
It took Argyle just four minutes to break down Oldham's much-vaunted defence. Lee Hodges' cross was helped on by Michael Evans and Friio, and Stonebridge stole in on the blind-side to sweep the ball home.
They doubled their lead within three minutes, with Stonebridge this time the creator, seeing Friio's run on the left and timing his pass perfectly for the Frenchman to fire home.
Oldham's only response to the home side's pressure was a far post header from Eyres following Clint Hill's in-swinging corner but Milosevic managed to keep his on-target header out.
Oldham introduced Sheridan from the bench at half time in place of John Eyre as Oldham tried to find a way into the game.
However, the home side maintained their pressure until midway through the half when the game turned dramatically.
The Oldham lifeline came when Argyle substitute Brian McGlinchey ended Chris Armstrong's surging run into the penalty area by tripping him.
Eyres took the subsequent spot-kick shooting low to the right of Milosevic who was struggling with a knee injury.
Two minutes later the Latics were level. Fitz Hall's cross from the right was met by Andrews, who volleyed the ball home off Milosevic's left-hand post.
Oldham strove to find the winner against a home side now clearly struggling with their substitute keeper Luke McCormick on for Milosevic.
Chris Killen was twice denied by blocks from Paul Wotton and Carlo Corazzin shot over as Argyle's defence held out for a point