You'll know the castle when you see it, as it's guarded by imperial soldiers.
Castle Dour is found in Solitude, which is found in the Northwest of Skyrim. Here's how to find the necessary people and how to join the legion if you miss that early-game pointer. One of the first things that can happen in Skyrim is that a friendly character can suggest you join the Imperial Legion, but it's also quite easy to miss this recommendation and skip out on joining them entirely. Some in Skyrim might have secession on their minds, but the Imperial Legion are quite against that. The Imperial Legion is a part of the armed forces of the world of Skyrim that's led by General Tullius. With Skyrim out on Switch and PlayStation VR now, it's a good time to revisit the different factions available to you in the game.
I mean, their symbol is on the game's cover.
