Jordan Fitzpatrick opened the scoring for Stourbridge on 20 minutes after being set up by Ben Billingham, but Whitby were deservedly level on 52 minutes thanks to Liam Shepherd.
However, the Glassboys secured all three points thanks to an 82nd minute deflected effort from Sean Geddes.