Palmer was eventually found in a severely disoriented state at Copacabana beach at 1.40pm on Wednesday, nine hours after leaving his Olympic swimming teammates.
Despite being robbed, he has not reported it to Rio police.
When asked if illicit substances were involved, chef de mission Kitty Chiller said: “I certainly don’t know if there were any substances involved. He admitted he had been drinking, that’s all we know.”
The swimmers had been at a nightclub until 4.30am, but Palmer stayed drinking at a beach kiosk with a friend from Melbourne when his teammates returned to the athletes’ village.
The lawyer has said he left the 25-year-old at 10.30am and assumed he was on his way back to the village but Palmer was found more than three hours later by another Australian businessman and an Englishman.
It’s unclear what happened to the breaststroker in the intervening hours.
He says he was approached by a man selling hats while sitting on the beach and threatened with a gun.
The hat seller and several others then took Palmer to an ATM where he said he was forced to withdraw $1000.
While swimming team leader Wayne Lomas knew at 10.30am Palmer was missing after another swimmer rang the lawyer, he did not tell Chiller.
Chiller only found out at around 2pm when the Australian consulate in Rio told her they had been contacted by the businessman.
Chiller said she had no reason to doubt Palmer’s version of events and suggested the recent drama surrounding American swimmer Ryan Lochte may have convinced him not to go to police.
Lochte had initially claimed he was also robbed at gunpoint but footage has emerged of him and a group of teammates in a dispute with service station attendants after allegedly damaging the station.
“There’s been a lot happening in the last couple of days with going to police,” Chiller said.
“We gave Josh every opportunity and the pros and cons of going to police. We can’t force him, it’s his decision.”
Palmer and fellow swimmer Emma McKeon, who also failed to return to the village, have been disciplined.
McKeon broke team protocol by staying in Copacabana with friends, believed to be Swedish swimmers, without telling team management.
The pair have been put on a curfew of 8pm to 8am for the remainder of the Olympics and banned from the closing ceremony.
The entire Australian swimming team must also be back in the athletes’ village by 2am.
Chiller said athlete safety was paramount and officials were not just being the fun police.
“I don’t want to call a parent and tell them their child has been robbed, mugged or anything worse,” she said.
Chiller has also chastened Lomas.
“I have raised the need for the swimming team leader to ensure he is aware of his athletes’ whereabouts,” Chiller said.
Palmer finished fifth in his heat of the 100m breaststroke on day one.
McKeon won four medals at this Games.
Both Palmer and McKeon, who have apologised for their actions, have been banned from attending the Games’ closing ceremony.
But Olympic relay gold medallist Melanie Wright has launched a petition to have McKeon march in the parade on Monday morning, Australian time.
“To clarify, I’ve followed AOC safety protocol in not travelling to village alone. My error was not texting my team manager. #noclosingforme,” McKeon tweeted.
Wright responded by tweeting her support and setting up a link to a petition.
“Completely over the top punishment for @emma_mckeon. This girl would avoid getting involved in a pillow fight. Too harsh for a small mistake,” Wright posted.
Local News Matters
Media diversity is under threat in Australia – nowhere more so than in South Australia. The state needs more than one voice to guide it forward and you can help with a donation of any size to InDaily. Your contribution goes directly to helping our journalists uncover the facts. Please click below to help InDaily continue to uncover the facts.