Political commentators, left and right, are urging the Biden administration to move away from the nationalistic realism of Donald Trump and to place “human rights” front and center in American foreign policy. It is certainly painful to watch an American president performing a sword dance with Saudi princes only a year before those same princes ordered a journalist literally carved up by hitmen. That said, we should also recognize that advocates of human rights sometimes have a very simplistic notion of how those human rights actually flourish.

The invasion of Iraq in 2003, for example, was justified by many because of the terrible human rights record of Saddam Hussein. In order to build support for the invasion of Iraq, George W. Bush asserted in his State of the Union address: ”The dictator who is assembling the world’s most dangerous weapons has already used them on whole villages, leaving thousands of his own citizens dead, blind or disfigured.”

While Saddam was unquestionably brutal, there’s also little doubt that the U.S. invasion of Iraq was a human rights disaster for the Iraqi people. Brown University estimates that 200,000 civilians were killed in the Iraq war. Over the years, Saddam himself killed tens of thousands of innocents in his wars upon his Kurdish and Shia foes, but one cannot say that the U.S. invasion of Iraq was a neat solution to the problem of human rights abuses in Iraq. In short, those who held out regime change as an opportunity to put human rights “at the center” of Western foreign policy were gravely mistaken.

So, then, where do human rights come from? Let’s first ask why the United States has long been considered a beacon of liberty and human rights. Because Thomas Jefferson wrote the Declaration of Independence? Obviously it is more complicated than that.

The rights and liberties of the American people are an inheritance that we were gifted by the great American and British constitutional traditions going back before even the Magna Carta. Our liberties and rights do not exist because we declared them in a document in 1776, but because our ancestors did the work necessary to make them a reality. And a great part of that work was restraining their own passions by avoiding physical attacks upon their political competitors, operating with good will toward those who disagreed, and respecting the parameters of the Constitution. America has certainly had its share of vitriolic politics, but we have generally avoided murdering our political opponents. The success of the American polity is not due to paper documents, but a constitutional culture that sets rough boundaries of restrained behavior upon our leaders. If that culture and tradition break down, no paper document will save us.

It was Martin Luther King who insisted in his “Letter from a Birmingham Jail” that the American culture of freedom and liberty was a tradition and African Americans had earned the right to be a part of it. “We have waited for more than 340 years for our constitutional and God given rights,” said King. “We will win our freedom because the sacred heritage of our nation and the eternal will of God are embodied in our echoing demands.” King was successful because his restrained and dignified example pointed out that it was men like Bull Connor who were violating our sacred constitutional heritage, not the marchers in Birmingham.

Civil liberty is a tradition, a “sacred heritage” that is earned over many generations by those who approach politics with restraint and good will rather than an arbitrary will to power. A certain moderate way of acting in politics gives birth to human rights. The danger in declaring that human rights are a free gift of nature is that it ignores the reality that rights emerge only when citizens and leaders restrain their own behavior. When citizens are well-ordered, there is no need for government to impose order, and liberty blossoms; when leaders are restrained, they are no threat to their citizens.

It is, of course, particularly important that leaders feel constrained by a tradition of moderation and restraint as they possess the actual power to violate human rights. The citizens of Hong Kong and the Uighurs regularly have their human rights violated because the leaders of the Chinese Communist Party have largely jettisoned the rich Confucian tradition of restraint. They feel no cultural, constitutional, or religious checks upon their will to power.

This is why, while declaring that all human beings have “human rights” may be true in the abstract, it tells you nothing about how these rights take hold in actual societies. And there are some perverse supporters of “human rights” who view the American military as the guarantor of abstract rights even for societies that have no tradition of liberty. This is when abstract rights become a genuine menace.

Ethical statesmen cannot ignore calls to protect the innocent. But prudence is the greatest virtue in politics and grandiose gestures to “protect human rights” can sometimes result in wholesale violations of human rights, as our actions in Iraq, Libya, and Syria have all demonstrated. Prudence, restraint, and temperance are the qualities of statecraft that build a legacy of human rights; ahistorical and abstract demands for liberty give birth to violent revolutions and invasions. And the best policy for promoting human rights abroad is to set the strongest possible example of sound statesmanship here at home. Rather than crusading for human rights around the world, we may wish first to consider the qualities now driving our own politics.

William S. Smith is senior research fellow and managing director of the Center for the Study of Statesmanship at the Catholic University of America. His recent book is Democracy and Imperialism from the University of Michigan Press.