Unlike Rice, Kerry is not seen as an Obama insider, but he has been a presence at key moments in Obama's rise.

He served with then-Sen. Obama on the Foreign Relations Committee and, as a presidential nominee in 2004, gave the young senator a platform on the national stage when he asked him to speak at the Democratic National Convention. In this election, Kerry also helped prep Obama for his debates with Mitt Romney and offered a strong argument of the administration's foreign policy during his address at the convention in September.

Obama is considered a president who likes to drive foreign policy himself, and the White House plays a major role in both its conception and execution.

There's a question of whether Obama will empower Kerry to fashion areas of foreign policy on his behalf. Miller, the Middle East expert at the Wilson Center, cites legendary former secretaries such as Henry Kissinger, James Baker and George Schultz as diplomats who were empowered to create the policies that their presidents then implemented.

"That is going to be the difference, I think, between John Kerry being a good secretary of state and ... truly be a consequential, if not great, secretary of state," Miller said.