An entertaining game saw chances at either end but the hosts caught the Glassboys out with a sucker punch towards the end when a replay seemed likely.
The Linnets took the lead on 29 minutes through George Thomson. Set up by Michael Frew, he dinked the ball beyond Dean Coleman to open the scoring.
Undeterred, Stourbridge hit back on 38 minutes when, on the counter-attack, Drew Canavan's pass put Jordan Fitzpatrick clear and his low shot into the corner brought the sides level.
The Glassboys had chances through George Bowerman and Fitzpatrick again before the interval but as the game became increasingly open, it was the home side that looked most like grabbing a decisive goal.
And with just one minute on the clock, that goal came through Greg Smith, who tapped in Thomson's cross after Stourbridge had been caught short of numbers at the back.
To add salt to that wound, the Linnets snuffed out any chances of a late fight-back from the Glassboys two minutes into stoppage time. Coleman brought down Smith in the area and although he was able to save Thomson's penalty the same player was quickest to react on the follow-up to make the final score 3-1.