Exeter Chiefs are still looking for their first victory of the season after George Furbank's late penalty saw Northampton maintain their 100% record.
The Chiefs raced into a 14-0 lead with Don Armand and Jack Maunder crossing within seven minutes.
Saints hit back through Alex Mitchell and Matt Proctor, but a Rus Tuima try edged Exeter ahead at half-time.
Joe Simmonds stretched the lead but two Furbank penalties in the closing stages saw Saints snatch a memorable win.
Exeter made a fast start after their opening day defeat at Leicester and it took Armand only four minutes to cross the whitewash through a driving maul, with Joe Simmonds adding the conversion.
They had their second two minutes later when Maunder grounded Henry Slade's kick in behind and Simmonds' boot put them 14 points ahead.
The shell-shocked Saints soaked-up the early onslaught and hit back through Mitchell's try and they had levelled it up after 20 minutes when Proctor raced in from 20 metres for a score converted by James Grayson.
The Saints fly-half then kicked a penalty to give his side the lead for the first time but Rus Tuima's close-range try two minutes before the break meant the Chiefs took a slender 21-17 lead into the changing rooms.
It took 16 minutes for the first points of the second period to materialise, with Simmonds adding three more from the tee to make it 24-17 after a Saints infringement.
Grayson cut the deficit with a penalty of his own just after the hour mark before Furbank took over kicking duties and made it 24-23 with nine minutes remaining.
He was then given the chance to seal the win with two minutes left on the clock and made no mistake with a well-struck penalty from almost 30 metres.
Northampton Saints director of rugby Chris Boyd:
"I have to feel for Rob (Baxter) as it's a tough way to start the season with so many players absent. It still wasn't a complete performance from us, but I was very pleased with our line-out defence as they are very strong in that area.
"We know we can play but the biggest problem for us is that we don't play well consistently enough and that's where we need to improve.
"It was a big decision to take our goal-kicker, James Grayson, off but in the 15 minutes prior to doing so, George (Furbank) did a lot of the kicking and fortunately luck went our way."
Exeter director of rugby Rob Baxter:
"It's not so much missing the Lions but the level beneath them. Jacques Vermeulen and Dave Ewers are both missing from our back-five together with Lions Jonny Hill and Sam Simmonds so this is limiting our options.
"They as a group are not delivering and if I was an opposition coach, I would be saying that we are vulnerable so next week's trip to Sale will be interesting.
"It's not the end of the world losing by two points to a team who won last week but we need to get our heads down and work hard to eradicate our mistakes."
Exeter: Hodge; Cordero, Slade, Devoto, O'Flaherty; J Simmonds, Maunder; Hepburn, Yeandle (capt), Williams, Witty, Lonsdale, Kirsten, Armand, Tuima.
Replacements: Innard, Moon, Iosefa-Scott, McCauley, Tshiunza, Hidalgo-Clyne, H Skinner, Whitten.
Northampton: Furbank; Sleightholme, Proctor, Dingwall, Collins; Grayson, Mitchell; Waller, Matavesi, Painter, Ribbans, Ratuniyarawa, Wood, Ludlam (capt), Harrison.
Replacements: Fish, Auterac, Hill, Moon, Augustus, Lomani, Litchfield, Freeman.
Referee: Christophe Ridley (RFU).