I finished the two main missions that are related to the castle settlement and I seen the cutscene for the "7 people left" and I went and destroyed that lab. The castle has been grayed out this whole time and I can't seem to progress to get it to be out of the "destroyed" state.
Im seeing this is a bug but only for consoles, Im a PC player.
It's simply an inactive settlement. It features briefly in the Broken Dawn novel, which sheds some light on what happened; but beyond that it is supposed to remain inactive.