A headed goal six minutes from time by Micky Evans secured a hard earned passage into the second round for the visitors.
Chester had taken the lead after 28 minutes when defender Scott Guyett rose in the box to power a header home.
Six minutes later the visitors were level through a well worked effort that saw Marino Keith finish superbly from an acute angle.
From the re-start the Pilgrims were on the offensive and they gave Chester a scare as Martin Phillips twice hit the woodwork.
A chance for scorer Keith to double his tally and settle the tie went begging after 71 minutes, but it was left to Evans to clinch victory for Argyle