The irony of Sen. Mark Udall’s stunning defeat to Republican Cory Gardner in this year’s midterm election is that he lost, in part, because of his association with a significantly unpopular president.

In reality, he was never a likely real stand-in for the president or the Democratic caricatures painted by the GOP. A western state mountaineer and self-described libertarian who supported concealed carry permits alongside a woman’s reproductive rights, this Colorado Democrat wasn’t your typical limousine liberal.

But come January he will be gone, leaving behind a canyon-sized void that civil liberties advocates say will be difficult to surmount.

“To be blunt, he’s leaving very big shoes to fill,” said Neema Singh Guliani, legislative counsel for the American Civil Liberties Union, which has worked with Udall for over a decade. “When I think of his work over the last several years, he was really only joined by (Oregon Senator Ron) Wyden and a few others since the early days, asking the hard questions,” she added.

“He was one of the few who were willing to cross examine and take issue with any statement that was posited by the national security establishment – things they would prefer not to be questioned about,” said Tom Drake, a whistleblower and privacy activist who has plenty of experience battering the doors of Congress to force transparency on the government’s classified surveillance programs.

“It doesn’t take much opposition to get opposition – all you need is a check,” he said, noting that Democrats Udall and Wyden, who both serve on the Senate Select Committee on Intelligence, provided that “check” for years. “We are going to lose one of the key voices in the committee that is supposed to have oversight.”

Republicans saw Udall, 64, as an easy target for a Senate majority takeover in a midterm election that became more about President Barack Obama and his policies than anything else—and it worked. Taking up the challenge, Udall by pundits’ accounts, flubbed the campaign by focusing too heavily on whether Gardner opposed reproductive rights for women and not enough on other issues.

A cruel-fated end, his supporters say, to a career spent being the type of elected official most Americans say they want in poll after poll—someone who works for them, and not necessarily for party dominance or parochial allegiances. He may have had a “D” after his name, but particularly on the issue of constitutional rights and liberties, Udall was a champion of every American.

“Mark Udall has been a real leader on these issues, and he’s not someone who can be easily replaced,” his old compatriot Wyden told TAC in a statement.

Udall has not only been on the forefront of privacy issues, but his independent streak has extended to voting against the war in Iraq and against the 2012 Defense Authorization Act, which critics say would have allowed for the indefinite detention of Americans. Advocates now want him to read the classified CIA torture report, the release of which has been delayed by the White House and CIA, into the congressional record before he leaves office at the end of the month.

Airing the 6,000 page final report for the first time would be his boldest move yet. His office told TAC this week that the senator is considering “all options,” and is currently in talks with the White House to get the report released before it gets to that point. News late Thursday suggested a summary of the report might be released early next week.

Udall declined to be interviewed for this story, but e-mailed some of his thoughts about his work in the Senate. “At the heart of freedom is the freedom to be left alone. That’s why I’ve been a steadfast advocate of Americans’ privacy rights and worked to rein in the National Security Agency’s overreach,” he said. “Our intelligence community should focus on combating terrorists, not collecting the private phone records of millions of law-abiding Americans when there is no compelling national security interest to do so.”

Udall served 10 years in the House before winning a six-year term in the Senate in 2008. He drew his first line in the sand by voting against the Patriot Act in October 2001. It overwhelmingly passed, but he was one of 62 Democrats and 66 members total in the House to vote against it. Anyone who was working in Washington in those emotionally charged weeks after 9/11 knows that was a bold stand to take.

“I don’t care about the politics, it’s bad policy,” he reportedly said to Democratic consultant Steve Welchert at the time. “Mark turned out to be right,” Welchert recalled later to The Denver Post. That was an understatement.

More recently, as a senator, he voted again against extending several provisions of the Patriot Act, chiding his fellow Democrats for a lack of debate. For years he and Wyden had been banging their heads against the wall to get federal law enforcement to disclose just how they were interpreting the laws. As members of the Intelligence Committee they know things most Americans don’t, and they expressed frustration that secret interpretations of the law had allowed the government to run roughshod over the constitution while calling it a matter of “national security” in order to keep it all secret.

“We believe most Americans would be stunned to learn the details of how these secret court opinions have interpreted section 215 of the Patriot Act,” Udall and Wyden wrote to Attorney General Eric Holder in March 2012 after the Justice Department sought the dismissal of two lawsuits by the ACLU and The New York Times. The suits endeavored to disclose how 215 (the provision that allows the feds who go after “any tangible thing” they say is related to a terrorism investigation without a warrant) was being used by law enforcement.

“As we see it there is a significant gap between what most Americans think the law allows and what the government secretly claims the law allows,” they wrote. “This is a problem because it is impossible to have an informed public debate about what the law should say when the public does not know what its government thinks the law says.”

The two senators—who are among the very few privy to the classified interpretations—went on to make some damning assertions, not only about over-classification, but they also suggested the government was misusing the “national security” card in order to obfuscate the truth. “We have grown increasingly skeptical about the actual value of the ‘intelligence collection operation’ …this has come as a surprise to both of us, as we were initially inclined the take the executive branch’s assertions about the importance of this ‘operation’ at face value.”

In the summer of 2013, Udall called for the Patriot Act to be “opened up” and reformed. This time, he didn’t just have his own evidence about government abuses gleaned from classified briefings, but the whole of Edward Snowden’s leaks from that June. He and Wyden found common cause with Sens. Rand Paul (R-Ky.) and Mike Lee (R-Utah) to push for reforms at the National Security Agency, which we know now has been using FISA to dragnet millions of Americans’ personal data and phone records without warrant.

“It’s time to end the dragnet – and to affirm that we can keep our nation secure without trampling on and abandoning Americans’ constitutional rights,” Udall, Wyden, and Paul wrote in a June 2013 op-ed.

Jesselyn Radack is a federal whistleblower and attorney whose clients include Edward Snowden. Like Drake, she has emerged as one of Washington’s toughest privacy advocates. Radack told TAC that after the Snowden leaks rocked Capitol Hill, Udall went straight to work, “conducting aggressive oversight of the NSA’s surveillance programs,” she said. “At a time when the majority of the congressional intelligence committees are unable or unwilling to properly oversee the surveillance state, Senator Udall stands out as a member dedicated to informing the public and protecting civil liberties,” she added.

“He wasn’t just talking about the NSA only when it was cool to talk about the NSA,” added the ACLU’s Guliani. “He was talking about these issues for a long time and continues to talk about them.”

But Snowden gave Udall and Wyden a chance to grill people like Director of National Intelligence James Clapper and CIA Director John Brennan, with headline-grabbing and illuminating results. After a cross-examining from Wyden, Clapper told the Intelligence Committee that the NSA was not gathering millions of Americans’ phone records without warrant—contradicting to the evidence right in front of the committee. Clapper was forced to apologize.

Meanwhile, the senators have doggedly quizzed other administration officials about the NSA’s role in collecting massive amounts of data under Section 702 of the 2008 FISA Amendments Act (which Udall also opposed). This tool has allowed NSA to dragnet and keep massive amounts of data without warrant even if it’s useless to any investigation. Two previously secret programs—PRISM and Upstream—were revealed in the Snowden leaks.

The Washington Post reported in June that “ordinary Internet users” far outnumbered actual terrorism-related suspects among those swept up in these programs, and more than half of the tens of thousands of surveillance files reporters were able to analyze came from American citizens—not foreigners—ranging from their emails to photographs to academic transcripts and medical records.

Marcy Wheeler, a privacy policy expert and scribe behind the popular Empty Wheel blog, points out that Udall is also on the Senate Armed Services Committee, and from that perch was able to question Vice Admiral Mike Rogers, who was nominated (and confirmed) to head U.S. Cyber Command/NSA this year. In much the same fashion as Wyden did with Clapper, Udall asked Rogers to say whether the NSA was conducting warrantless surveillance on Americans through Section 702. Rogers declined to answer.

“SASC was the only venue to ask Admiral Mike Rogers any questions before he was confirmed. Udall tried to use Rogers’ confirmation to get him on the record on back door searches,” Wheeler recalled to TAC.

“I also think he’s good at finding places where the government isn’t meeting the spirit of the law,” she added, pointing to the growing concerns about NSA data collection under Executive Order 12333, which is also secret and warrantless. She said Udall used the February confirmation hearing for John Carlin to Assistant Attorney General for National Security to press his points, and to show just how transparent these agencies want to be with the people. Turns out—not much.

However, said Guliani, “I feel that we are much further along with the debate than we were before thanks in part to Senators Udall and Wyden.”

In July, Udall called for CIA head John Brennan to resign after the agency was forced to admit it spied on Senate Intelligence Committee staff computers while the CIA torture report was being pulled together. Brennan was forced to apologize.

Supporters hope Udall will read the torture report into the record if the White House continues to delay. Certainly, said Drake, it would be his last great stand as the independent senator.

“There’s been way too much cover-up and collusion and too much year after year of keeping critical matters of state from public view,” he said. “He is in a unique position to do this.”

Kelley Beaucar Vlahos is a Washington, D.C.-based freelance reporter and TAC contributing editor. Follow her on Twitter.