The time-worn remains of this castle, on a picturesque, grassy site overlooking a loop on the River Teifi, were once inhabited by, amongst others, Sir Rhys ap Thomas, friend of Henry VII. It would be hard to imagine a more serene setting than this, and one is not surprised to find that it was once intended as a county seat rather than a military post. Yet it had its share of excitement in the Civil War, when held for the king and blown up with gunpowder at the close of the struggle. There are remnants of some of the walls and the ruins of the gate. It is significant as the only native Welsh castle to be built in stone in this part of Wales.
The castle was probably founded by Maredudd ap Rhys around 1240, and if this is so, it is one of the few castles in Dyfed built by the Welsh in stone. His son, Rhys ap Maredudd, held the castle in 1287, and the castle changed hands three times during his successful revolt against the English crown from 1287 to 1289. After Rhys had finally been defeated and killed, the castle became crown property and remained so until 1349. During this time, three refurbishments are recorded, during which this time the gatehouse was constructed and a new town was founded outside the castle walls. In 1403 the castle was taken by Owain Glyndwr, but was described as being in ruins by 1428.
Sir Rhys ap Thomas acquired and repaired the castle in about 1500. It changed hands a number of times before the Civil War during which it was held by Parliament until its capture by Sir Charles Gerard in 1644. Major-General Rowland Laugharne besieged it for Parliament in 1645 but was routed by Gerard in a fierce engagement below the castle walls. After the general surrender of the Royalists, the castle was blown up to make it indefensible and, according to a source of 1700, the castle was plundered and ever since neglected.