Yagoda ran the NKVD between 1934 and 1936, was dismissed in 1937 and executed in 1938 for treason and conspiracy, becoming one of millions of victims of the Soviet system that he himself helped establish under Joseph Stalin.
"The court rejected the request for the rehabilitation of Genrikh Yagoda," Russia's state-funded RAPSI news agency quoted a court official as saying.
No one at the Supreme Court could be reached for comment and it was not clear who had filed the case or whether the court was looking into it as a matter of course under a law on the rehabilitation of victims of Stalinist repression.
The decision indicates there are limits to what critics of President Vladimir Putin say is his drive to whitewash the Soviet past, including the legacy of Stalin, and buoy his own political reputation by channeling the pride felt by many Russians about the Soviet Union's achievements.
Stalin's rule remains the subject of bitter debate in Russia, where many believe he did some good for the country.
A Levada poll this week showed that 30 percent of Russians felt "respect" for Stalin and as many were indifferent, compared with nine percent who felt hostility towards him.
While 46 percent associated Stalin's death in 1953 with an end to an era of terror, 45 percent of respondents said the Soviet Union's "great achievements" justified the loss of millions of lives during his rule.
Levada also said 57 percent were against considering Stalin a criminal.