But the prime minister remains positive about changes Attorney-General George Brandis will introduce to parliament this week.

The government is proposing to amend a contentious section of the Racial Discrimination Act to replace the words “offend”, “insult” and “humiliate” with “harass and intimidate”, setting the bar higher for complaints.

The Australian Human Rights Commission will be given power to head off frivolous claims at an early stage.

With Labor and the Greens opposed, the government is on the hunt for nine of the 11 crossbench votes.

Jacqui Lambie won’t be one of them.

“I just think it’s a waste of time speaking about this, there’s so (many) bigger things on the table that we should be talking about,” she told ABC radio on Wednesday.

“If it’s dead in the water let’s move on because we have some economic issues in this country.”

When asked whether the bill was doomed, Turnbull insisted his government would continue negotiating with all parties.

“I recall you predicting many elements of our legislative program are doomed to fail and you’ve been disappointed in the past, so we will just focus on getting our agenda through the Senate,” he told reporters in Canberra.

The Nick Xenophon Team, which has three senators, has signalled it won’t back a re-wording of Section 18C.

“But we do support sensible changes to the process involved in the handling of such complaints so the process does not become the punishment,” Xenophon said.

Brandis said he would continue talking to Xenophon.

“I spoke to him last night. I’m seeing him again today,” he told Ray Hadley on 2GB radio.

“But Ray, I can promise you and your listeners, we will give this our best shot.”

The government did not want Australia to become a country in which there was a culture of grievance.

Brandis invoked the Anzac legend, saying the freedom of speech they fought for was now one of our core social values.

At least four Coalition MPs expressed misgivings about the changes when they were debated in a partyroom meeting on Tuesday, but they are not expected to cross the floor on the issue.

Deputy Prime Minister Barnaby Joyce said there were clearly defects in race-hate laws which exposed it to accusations of overreach.

To harass or intimidate someone based on their race was a more robust threshold than to insult or offend.

“If I am threatening to you by reason of your religion or by reason of the colour of your skin then I have every reason to be hauled across (the coals),” he told Jon Faine on Melbourne ABC radio.

“But if I insult you and say, ‘Jon you’re a big fat slob’ you can say that’s an insult but it’s no reason to take someone to court.”

Labor frontbencher Ed Husic said election material had been targeted against him based on his Muslim faith.

“I’m big enough and ugly enough to look after myself in politics but I have a genuine concern about how people – ordinary folk on the street – get treated with a system that gives license to speak first and think later,” he said.

“I expect the rough and tumble of politics, but I don’t think general members of the Australian public should be subjected to it.”

– AAP