As a huge Greek flag flew over central Syntagma Square, Greeks from all over the country and abroad chanted "Hands off Macedonia," "Macedonia is Greek" and "We won't leave until we are vindicated."
Athens objects to Macedonia's name, arguing it suggests that Skopje has claims to the territory and heritage of Greece's historic northern region of the same name.
However, leftist Greek Prime Minister Alexis Tsipras has been considering a resolution to the 27-year-old dispute, angering many opposition members and his own nationalist coalition partners.
Among those in the crowd was former premier Antonis Samaras - who was foreign minister when the name dispute began in 1991 - along with mayors, senior clerics and army officers.
Organizers said they expected around a million participants on Syntagma Square.
"Macedonia is Greek and only Greek. They are trying to steal history. We all have to fight and let the world know," said Allia Sarellis, a member of the Greek diaspora who flew in from the United States.
Around 2,500 buses from northern Greece and two ships from Crete brought thousands of demonstrators, reports said. Police declined to give numbers.
Some protesters wore traditional garb, including the uniform of Greek guerrillas who fought Bulgarian bands and Ottoman forces in Macedonia in the early 20th century.
"Thousands of Cretans have shed their blood for Macedonia," a bearded Cretan protester told Skai TV.
The protest - the second on the Macedonia issue in a fortnight - has been organized and funded by Greek diaspora groups, with the support of retired officer associations, cultural unions and church groups.