Error loading navigation: The Managed Navigation term set is improperly attached to the site

I created a local term set in one of the site collection and I set the Managed Metadata Term Set as a Site Navigation. 
But after setting the Managed Navigation, I was getting the following error:
 
 
 
 
Refresh the page 2-3 times, the issue will be resolved.