"I may leave fast" if progress does not seem possible, Trump said at a campaign rally for Republican congressional candidate Rick Saccone in western Pennsylvania. Trump said he believes North Korea wants to make peace and that, "I think it's time."
A time and place to meet has not yet been set, although the meeting is supposed to happen by the end of May.
"Who knows what's going to happen?" said Trump, who added that if the meeting takes place.
"I may leave fast or we may sit down and make the greatest deal for the world."
Trump made the shocking decision on Thursday to meet with Kim after the North Korean leader's invitation was relayed by a South Korean delegation who visited the White House.
The move abruptly reversed decades of US policy aimed at preventing North Korea from developing nuclear weapons and ballistic missiles.
Earlier in Washington, Trump sought to rally international support for a potential summit, saying North Korea had agreed not to conduct another missile test until after proposed meetings had taken place.
"North Korea has not conducted a Missile Test since November 28, 2017 and has promised not to do so through our meetings. I believe they will honor that commitment!" Trump wrote on Twitter.
Trump's comments aligned with what a South Korean official stated on Thursday about the possible talks. Trump's tweet made no mention of nuclear tests, however.
It was not immediately clear which meetings Trump was referring to or their timing. South Korean officials said this week that Trump had agreed to an invitation from Kim to meet by May.
The White House has been under fire for agreeing to talks and responded to the criticism on Friday by warning that no summit would occur unless Pyongyang took "concrete actions" over its nuclear program.
Newspaper headline: Trump predicts Kim meet