The earliest written records mentioning the castle date back to 1050-1052.
The castle was originally a "castrum" or military base held by the 1st lord of Vignory, Gui.
An inventory of 1773 lists a farmyard surrounded by ramparts, linked by 7 enormous towers, which sheltered the farm, stables, byre, dovecote and gardens, and a courtyard surrounded by 3 towers containing the keep, a well tower, a chapel, a manor house, a cistern and various out-houses.