Jurgen Klopp's side has squandered a comfortable position in Group C after losing two of their last three matches.
Marko Marin delivered from the right-hand side, and Pavkov - who had never before scored in the Champions League - rose unmarked to nod into the bottom corner on 22 minutes.
Politics will keep Liverpool forward Xherdan Shaqiri from making the trip to face Red Star Belgrade in the Champions League, almost five months after the Switzerland forward made a nationalist hand gesture while celebrating his winning goal against Serbia at the World Cup in June.
Jurgen Klopp has said Liverpool "have to do better" after they were beaten 2-0 by Red Star Belgrade in Serbia on Tuesday night.
Liverpool's Champions League campaign as been full of peaks and troughs so far and this latest result has seriously jeopardised their chances of reaching the knockout stages. Still unbeaten after 11 games, the only points they have dropped came in draws against other sides in the top five in Manchester City, Chelsea and Arsenal.
"We didn't start well enough and that showed with them getting the two goals", Milner added.
'He has qualities like Joe [Gomez] has, like Dejan [Lovren] has, like Joel [Matip] has, so that is really good for us.
Red Star had conceded 10 goals in being thrashed by PSG and Liverpool in their last two European encounters.
"I think it's one of those periods we need to get through".
A few we had to make and a few we wanted to but it is easy to say it didn't work out in the end.
Liverpool will be without the services of captain Jordan Henderson and summer signing Xherdan Shaqiri, when they face off against Crvena Zvezda.
He admitted the defeat to Red Star was "a rather massive knock" but that the outcome of the squad's meeting has not changed their targets.
"The Champions League final past year, new signings, players settle-it sounds like automatically you would win the Champions League, but the other teams don't sleep!"
Liverpool had a chance to grab the lead early on when a deflected ball across the box fell to Daniel Sturridge, but the Englishman steered over from close range.
Henderson's return after almost a month out with a hamstring injury is particularly timely following two below-par performances from Jurgen Klopp's midfield.
Liverpool have won just three of their last nine matches, against Red Star, Cardiff and Huddersfield.