After a scoreless first half, Navas fought off a barrage of challenges to open the second with the boisterous Costa Rican fans cheering him on at Samara Arena.
After David Guzman was handed a yellow card, Kolarov curled the ensuing free kick over the wall.
"It's very important that we have these three points in the bag, but it's not over, Switzerland will be the hardest match".
Serbian captain Aleksander Kolarov scored a free-kick in the 56th minute to give Serbia the victory. Costa Rica had a few chances at the goal but failed to convert them.
The 23-year-old midfielder was the standout player in his side's 1-0 victory and he proved his worth on the field after a stunning debut performance in Russian Federation.
Brazil are overwhelming favourites to top Group E, which also includes Switzerland, so a dogfight is likely to ensue to follow them into the knockouts - and defeat would leave the losers in a precarious position.
Brazil v Switzerland, 17 June 2018
Marcelo was the only survivor from that Germany defeat that started - and it was down his left side where Brazil looked risky . Perennially among the favorites to win the FIFA World Cup , Brazil will open the 2018 tournament against Switzerland at 1 p.m.
Kolarov's 56th-minute effort rewarded Serbia for their overall quality, although Costa Rica were to rue a number of missed opportunities in the first half.
This will be his first tournament as a manager, having been an ever-present as a player for Costa Rica in their first ever World Cup in 1990. Serbia return to the World Cup stage after missing the 2014 Brazil world cup and Euro 2016. "I'm really happy, but it is a huge responsibility", said Costa Rica head coach Oscar Ramirez. It's a World Cup, here there are the best national teams around the world and we are one of them.
While Serie A was undoubtedly one of the most exciting leagues in Europe last season, with its tense race for the title and top four, it unfortunately does not have the audience of some of the continents' bigger leagues.
The Ticos were the surprise of the last World Cup, reaching the quarterfinals in Brazil before being ousted by the Netherlands on penalties.
In 2015, Serbia were proclaimed Under-20 world champions and Milinkovic-Savic was awarded the Bronze Ball.
See how to watch and follow live updates from the game here.