Lifted by hundreds from David Warner and captain Steve Smith, Australia posted an imposing 6-371 at Kingsmead – at the time the highest ODI score in Durban.
However a brilliant unbeaten 118 by South Africa’s David Miller guided the Proteas to the victory target with four balls remaining and four wickets in hand.
The victory means South Africa take an unassailable 3-0 lead in the five-match series.
Warner’s 117 and Smith’s 108 appeared to have delivered a target Australia’s inexperienced bowling lineup could defend.
But Smith and his men had no answer for Miller, who blasted his third ODI century off just 69 balls.
It’s Miller’s first score above 50 since his hundred against Zimbabwe at last year’s World Cup.
The big-hitting No.6 was well supported by youngster Adile Phehlukwayo, who scored a career-high 42no in an unbroken 107-run seventh-wicket partnership.
Australia’s bowling attack couldn’t contain the pair in the final 10 overs.
John Hastings, who took two early wickets, was belted for 79 runs off his 10 overs while Dan Worrall – recalled to the team in place of Joe Mennie – conceded 0-78 off his 10.
Legspinner Adam Zampa was also belted for 1-55 off 7.2 overs and three Travis Head (1-31) overs went at more than 10 an over.
Earlier, Warner’s highest score by an Australian opener in an ODI against South Africa provided just the start the tourists needed in the must-win clash.
Warner’s knock eclipsed Mark Waugh’s 115 at Port Elizabeth in 1997.
He and opening partner Aaron Finch (53) made 110 for the first wicket at a steady rate to set the platform for the innings.
Warner and Smith then combined in a 124-run second wicket stand before Imran Tahir (2-54) lured Warner into one big shot too many to end his 107-ball knock.
Smith steadily built his innings after a slow start, reaching his sixth ODI century with his third boundary of the 46th over.
Smith was eventually removed by veteran Dale Steyn in what was a rare highlight for the veteran quick, his 2-96 off 10 overs the worst ODI bowling figures by a South African in history.
Travis Head (35) and Matthew Wade (17no) then combined in a quickfire 46-run partnership to lift Australia past 350.
At the time, Australia’s innings set a new mark as the highest at Durban in an ODI and was just the third 300-plus score at the venue in an ODI.
But South Africa set about their task with the air of a team full of confidence.
Veteran opener Hashim Amla, playing in his first match of the series, belted a rapid-fire 45 off just 30 deliveries.
Quinton de Kock’s 70 came off just 49 balls but when he fell and South Africa were 3-164, Australia should have been able to restrict the hosts and secure the victory.
A costly decision to review a caught behind appeal on Miller when he was on 40 came back to haunt Smith when Phehlukwayo appeared to edge a Chris Tremain (1-65) delivery to Matthew Wade but Australia could do nothing when the umpire didn’t give the 20-year-old out.
From there it was one-way traffic as the Proteas roared home to condemn Australia to a third consecutive ODI defeat for the first time since 2012.
The series continues at Port Elizabeth on Sunday.
South African fast bowler Wayne Parnell won’t travel to the southern coastal city after he was ruled out for the remainder of the series with a rib injury.
A replacement won’t be drafted in to the Proteas’ squad for the final two matches while Parnell now faces a race to be fit for South Africa’s Test series in Australia starting in Perth on November 3.
Help our journalists uncover the facts
In times like these InDaily provides valuable, local independent journalism in South Australia. As a news organisation it offers an alternative to The Advertiser, a different voice and a closer look at what is happening in our city and state for free. Any contribution to help fund our work is appreciated. Please click below to donate to InDaily.