The unexploded ordnance was discovered in King George V Dock, during planned works near the runway of London's most central airport.
The bomb was discovered at around 5:00am (5:00 GMT) on Sunday and a 214-meter exclusion zone was imposed "to ensure that the ordnance can be safely dealt with whilst limiting any risk to the public," police said.
Homes within the exclusion zone were evacuated overnight and the local authority was providing residents with temporary accommodation and support.
Police remained at the scene on Monday.
"The operation to remove the ordnance is ongoing in partnership with our colleagues in the Royal Navy," London's Metropolitan Police said in a statement.
City Airport operates short-haul flights and is located in east London, close to the Canary Wharf business district.
It is the capital's fifth-biggest airport.
"The airport remains closed this morning," said its chief executive Robert Sinclair.
"All flights in and out of London City on Monday are cancelled.
"I urge any passengers due to fly today not to come to the airport and to contact their airline for further information."
London City Airport opened in 1987 in the disused docklands. It handled 4.5 million passengers in 2017 and is mainly used by business travelers.