Daryl Janmaat, Gerard Deulofeu and Roberto Pereyra all scored late on in response to Eden Hazard dragging Chelsea back into the game with a superb equaliser in the 82nd minute.
Tiemoue Bakayoko was sent off for a second booking on the half-hour before a contentious penalty, slotted away by Troy Deeney, gave the Hornets the lead at halftime.
Chelsea cruised to the title in Conte’s first campaign in England last season but they are 19 points behind leaders Manchester City and clinging to a top four Champions League qualification spot.
Asked about his future, Conte told Sky Sports: “It is not my problem, I repeat. I try to do my job and I try to put everything in my job, if this is enough it is OK, otherwise they can take their position.”
Mid-table Watford had taken the lead in the 43rd minute with Deeney making no mistake from the penalty spot after Chelsea goalkeeper Thibaut Courtois was adjudged to have brought down on-loan Barcelona winger Gerard Deulofeu.
“Obviously the penalty is a clear dive,” Courtois told Sky Sports.
“If a striker just puts his feet there and that’s a penalty then I think we’re not far away from everything being a penalty, he clearly leaves his foot and he dives.”
By that stage, Chelsea were down to 10 men after France midfielder Bakayoko was booked twice for challenges in the space of a few minutes.
Watford reacted quickly to Hazard’s equaliser as defender Janmaat weaved his way past defenders and shot low past Courtois.
Deulofeu sealed the win in the 88th with his first goal for the club before Pereyra got the fourth in added time.
The loss has added to the pressure on Conte, after a poor start to 2018 which has seen Chelsea knocked out of the League Cup by Arsenal before a 3-0 home loss to Bournemouth last week in the league.
“I tried to warn everyone about the difficulty of our path in the league and Champions League,” Conte said.
“We performed very bad but maybe I made the mistakes with the starting XI today. My choice was very poor I think.”
It was a different story for new Watford manager Javi Gracia who modestly described the victory in his first home match in charge as a “good beginning” for his side, who sit in 11th place but are still only six points clear of the relegation zone.
“I think it is important not only for three points but for the motivation to believe in our possibilities,” Gracia told reporters.
Help our journalists uncover the facts
In times like these InDaily provides valuable, local independent journalism in South Australia. As a news organisation it offers an alternative to The Advertiser, a different voice and a closer look at what is happening in our city and state for free. Any contribution to help fund our work is appreciated. Please click below to donate to InDaily.