The match looked to be heading to extra-time and possibly penalties but a 94th minute direct free-kick put the hosts through to the second round.
Boro enjoyed periods of possession during the game and created a number of chances but couldn't break down their stubborn opponents, who'd knocked Barnet out in the previous round.
Boro goalkeeper Tyler Reading made a couple of comfortable saves early on whilst Jimmy Hartley forced the Rovers 'keeper into a finger-tip save with a fierce effort from the edge of the box.
Fejiri Okenabirhie saw one effort saved and another fly over the bar as Stevenage continued to push, with Dan Udoh the next to see his low effort gathered by the goalkeeper.
In the second half Rohdell Gordon forced another top save from the home 'keeper with a direct free-kick, with Hartley nodding the resulting corner wide. The latter also saw another curling effort go over.
Despite looking relatively comfortable all game, Boro were dealt a sucker punch deep into injury time when Barton scored. Ross Millard was penalised just outside the area and up stepped Mahmut Caglar to curl a beauty into the top corner and spark wild scenes of celebration amongst the home faithful.
Barton's reward is an away game at Portsmouth in Round Two whilst Darren Sarll's youngsters will look to bounce back from this disappointment in their next league fixture.
Boro: Reading, Casey, Carey-Morrell, Joseph, Millard, Yamfam (Cathline 70), Towner, Gordon, Udoh (Gorman 45), Hartley, Okenabirhie
Subs not used: Robinson (GK), Jones