On Tuesday, Kelly said that despite the Tucson, Arizona, shooting that wounded his wife and killed six others, he and Giffords still support the Second Amendment, which guarantees Americans the right to possess firearms.

"But we really need to do something about the safety of our kids and our communities. It's gotten really out of hand," he said.

After Wednesday's hearing, Kelly made a point of approaching LaPierre to shake his hand as media cameras recorded the moment.