Overlooking Trefechan are the remains of Morlais Castle.

Gilbert de Clare, Lord of Glamorgan built Morlais Castle around 1288 AD. By this time Gilbert had already built Caerphilly Castle.

Humphrey de Bohun, Lord of Brecknock disputed de Clare's claim on the land Morlais Castle was built. This dispute culminated in 1291 with the Battle of Maesvaynor.
The battle is remembered in an old Welsh poem:

'The battle was won by the men of Bohun,
And blood like an ocean in Vaynor was seen.'

During the Welsh Rebellion of the thirteenth-century Morlais Castle was briefly held by Welsh rebels and used as a stronghold in the battle for Welsh independence.
After Edward I won his war against the Welsh he destroyed parts of Morlais Castle to prevent it from ever being used as a stronghold again.

Vaynor remained an agricultural parish until the Industrial Revolution began in the late 18th century. At this time the area changed dramatically. Houses were hastily built for workers, woodland was removed for fuel and huge limestone quarries were created.