The assembled audience was there for the swearing in of Justice Neil Gorsuch, but many eyes were trained on Kennedy, who like no other justice in recent history controls the outcome of the highest profile cases before the court.
Kennedy's role that day was to deliver the oath to his former clerk, but some of the judicial conservatives assembled in the Rose Garden were wondering if Kennedy might soon provide them with the opportunity to return to their vetting binders. Would he retire and give President Donald Trump a second vacancy and the chance to cement a conservative majority on the court for decades to come?
In the end, Kennedy made no announcement. After almost 30 years on the highest court in the land, he determined he wasn't quite finished.
Now, as retirement speculation ramps up again, court watchers wonder what went into his calculation for staying on the bench last term -- and if anything's changed since then.
Significant cases facing the court could have served as the siren call, or maybe Kennedy felt the weight -- and pull -- that one of his colleagues called the "awesome responsibility" of holding the critical vote for the biggest cases. It's conceivable he was spooked by the volatile first few months of the Trump administration, or perhaps retirement is easier to contemplate than to execute.
Whatever motivated Kennedy to stay, conservatives waiting for a chance to push the court to the right were disappointed. And liberals, even though they have been on the losing end of Kennedy votes on issues such as campaign finance and the Second Amendment, now cling to him. He has, after all, given them victories in LGBT rights, abortion access, affirmative action and his replacement under the Trump administration would surely be much younger and more conservative.
The speculation about Kennedy's future comes as the justices are considering an unusual number of extraordinarily consequential cases where his vote and input will be critical.
For example, partisan gerrymandering is front and center. The Supreme Court has never been able to articulate a test concerning the overreliance on politics in map drawing.
Back in 2004, the court rejected a challenge to a political gerrymander in Pennsylvania. While four justices believed that courts should never hear such cases, Kennedy held the door open in a separate opinion.
Kennedy acknowledged that ordering a correction of all election district lines for partisan reasons might amount to an "unprecedented intervention in the American political process," but he would not foreclose the possibility of judicial relief down road. He said that in the future a "limited and precise rationale" might emerge.
That day may or may not have come this term as the court is hearing not one, but two cases concerning the issue.
At oral arguments during one of the cases concerning Wisconsin last October, Kennedy did seem open at one point to the possibility that a standard grounded in the First Amendment could be found.
In late March, the court heard a second challenge regarding a map in Maryland. It was clear during oral arguments that the justices were still struggling with the issue.
A workable framework could be a landmark opinion that could revolutionize how states draw their lines.
"If Justice Kennedy agrees with plaintiffs in either of these cases, then he'll be known for decades as the justice who succeeded in rooting out the worst abuses in partisan gerrymandering," said Josh Douglas, a professor University of Kentucky College of Law. "Making his mark on this issue could be one of the reasons he hasn't retired yet."
Gay marriage and religious liberty
Kennedy will forever be remembered for penning Obergefell v. Hodges, the 2015 opinion that cleared the way for same-sex marriage nationwide.
In Obergefell, Kennedy wrote that opponents to same-sex marriage reached their conclusions based on "decent and honorable religious" beliefs.
In Masterpiece, however, Colorado cake baker Jack Phillips, who believes marriage should be between a man and a woman, refused to make a cake for customers seeking to celebrate their same-sex marriage.
Now Kennedy might have to reconcile two strands of his jurisprudence: the dignity of same-sex couples and the respect for religious liberty.
At oral arguments, Chief Justice John Roberts referred back to Kennedy's words in Obergefell and noted that the majority had "gone out of its way to talk about the decent and honorable people who may have opposing views."
Kennedy did seem, at one point, to be receptive to the religious liberty concerns of the baker. Kennedy pointed out that a commissioner on the Colorado Civil Rights Commissioned had expressed hostility toward Phlipps's beliefs.
"It seems to me that the state in its position here has been neither tolerant nor respectful of Mr. Phillips's religious beliefs," Kennedy said.
Ivanka Trump visits the court
Journalist Charlie Rose asked Justice Ruth Bader Ginsburg in 2017 about Kennedy's role as the deciding vote in so many hot button cases.
It's an "awesome responsibility" she said.
At another event held in California in 2017, one questioner took a different tack when asking the question.
"Who do you want to eat more kale in Washington?" the questioner asked.
Ginsburg and other liberals are well aware of the President's promise to reshape the courts.
One question is whether Kennedy would feel comfortable retiring during the Trump administration.
Heads spun just after the inauguration in February, when Ivanka Trump appeared at the court and sat in the section reserved for VIPs to hear oral arguments. According to the court's spokeswoman, the President's daughter was there as the invited guest of Kennedy, who she met at the inaugural lunch at the Capitol.
It wasn't lost on court watchers that Trump chose a former Kennedy clerk as his first pick. Other former Kennedy clerks such as Judge Brett M. Kavanaugh, of the US Court of Appeals for the District of Columbia Circuit, and Judge Raymond Kethledge of the 6th Circuit are high on a list of potential Supreme Court nominees.
Like every other Justice, Kennedy cares about the institution of the court and he will take that into consideration whenever he chooses to step down.
As a rule, justices don't like to retire in an election year, always trying to keep the court at an arm's length from the political process.
When Justice Antonin Scalia unexpectedly died in 2016 the court was forced to work with only eight members for more than a year as the political branches went to war over Scalia's replacement. The eight-member court is not ideal because if the justices split 4-4, they are simply left to affirm the lower court opinion creating no new precedent.
And all eyes will be on November, with the potential for Democrats to take over the Senate.
"Indeed, the very real prospect that Democrats will win control of the Senate in the November 2018 elections might clinch his decision to do so. If he waits until next year, and if Democrats take control of the Senate, his seat would probably remain empty until 2021," Whelan speculated.
Between now and June, Kennedy has almost a full docket of cases to consider. It's a real possibility that he hasn't even made up his mind.
He's hired clerks for next term, but warned at least one, two sources say, that retirement is always a possibility. There is precedent for the clerks who are hired by one justice to be absorbed by another justice if a retirement occurs. Indeed, Gorsuch was hired by Jusstice Byron White but served out his clerkship with Kennedy after White retired in 1993.
Court watchers, even some in the White House at the time, thought that Justice John Paul