Newcastle survived a controversial second-half sending-off as they came from behind with 14 men to beat Wasps at Kingston Park.
Having led 8-0 at the break after Carl Fearns' try and a penalty from Will Haydon-Wood, Tom Cruse's converted try for Wasps was followed by Penny's red card for an alleged finger in an eye.
Wasps then went over again through Tom Willis after Brett Connon's penalty.
But Callum Chick scored in the dying seconds - and Connon converted.
Falcons had briefly been back to level numbers at 14-a-side when Wasps had Vaea Fifita yellow-carded for a high challenge. But the visitors were back to 15 when Chick went over to earn Newcastle's second Premiership win of the campaign.
Jacob Umaga slotted both his conversion attempts, but the Coventry side had to return south from Tyneside with only a losing bonus point.
Just five minutes had gone when Newcastle cashed in on an attacking platform set up by an astute box kick from scrum-half Louis Schreuder to take advantage of the new 50-22 rule, which allowed number eight Fearns to touch down.
But Falcons were reduced to 14 men for the first time by referee Anthony Woodthorpe after Schreuder was sent to the sin bin for not being 10 metres away following an infringement.
And the game's next try did not come until early in the second half when hooker Tom Cruse crashed over from a rolling maul on his 101st appearance.
After flanker Chick ran 70 metres only to be stopped just short after intercepting Umaga's flat pass, Newcastle were then dealt a further blow when they were reduced back to 14 men again, this time for the rest of the match after full-back Penny was shown red for making illegal contact with Jimmy Gopperth's head as he tried to escape from the base of a ruck.
Referee Anthony Woodthorpe was in charge of this same fixture in March when Newcastle wing Mateo Carreas, although unpunished at the time, was later given a nine-game eye ban for intentional eye contact on Wasps winger Josh Bassett.
This time, having been alerted to the incident by his television match official, he opted to send Penny off without consulting the video screen himself.
When Wasps lock Fifita was then yellow carded for his high tackle on centre Pete Lucock to even up the numbers and Connon kicked the penalty, it was 11-7 with 15 minutes left.
Flanker Willis' try, improved by Umaga, put Wasps ahead for the first time, but Chick's late touchdown turned it round again for the hosts.
Falcons director of rugby Dean Richards told BBC Radio Newcastle:
"Tom Penny is distraught. He didn't do anything deliberate. His leg was trapped in by Jimmy Gopperth. You have to look at who the protagonist is - and it's clearly Jimmy Gopperth. Why should our boy get penalised for being trapped and held in a ruck when he's trying to get out?
"He pushes his head down once, twice and then as he's pushing himself up he catches his face, and I think his eye, and Jimmy makes a meal out of it. If there is any contact with the eye, then it's not deliberate.
"It was the same official who did the Wasps game last season and missed the Josh Bassett incident, so he wasn't going to miss anything this week, so we were expecting a little bit of something. I'm surprised that the refereeing department put him on this game, to be honest.
"But everyone will have gone home entertained. It was in doubt right up until the final whistle, yet we showed courage to see it through and to score that try in the way we did was great."
Wasps head coach Lee Blackett:
"There were a few bounces of the ball that didn't go our way. There were things we could have been miles better at.
"But still, we know that we put ourselves in a position to win it at 14-11 up and you're inside their 22 with a couple of minutes to go, we'd expect to convert that.
"The wind was so strong. You could tell by the way they were kicking. I felt eight points was probably a fair reflection of the wind.
"We came out second half fairly confident and we put them under loads of pressure. We got into the opposition 22 plenty of times to come away with points but we've got to look at not converting those opportunities."
Newcastle: Penny; Radwan, Stevenson, Lucock, Earle; Haydon-Wood, Schreuder; Brocklebank, McGuigan, Tampin, Fuser, Van der Walt, Chick, Welch (capt), Fearns.
Replacements: Blamire, Cooper, Davison, Montgomery, Graham, Nordli-Kelemeti, Connon, Wacokecoke.
Sin-bin: Schreuder (28 mins).
Sent off: Penny (49 mins).
Wasps: Crossdale; Watson, Spink, Gopperth, Bassett; Umaga, Robson; Hislop, Cruse, Toomaga-Allen, Fifita, Stooke, Shields (capt), Oghre, Willis.
Replacements: Frost, West, Alo, Cardall, Morris, Porter, Simonds, Kibirige.
Sin-bin: Fifita (63 mins).
Referee: Anthony Woodthorpe (RFU).