Add the milk, cocoa powder, brown sugar, and cinnamon stick to a small saucepan on medium heat. Whisk until the ingredients are combined. Heat it until it starts to gently bubble, stirring it occasionally.

Whisk in the vanilla extract and Grand Marnier. Leave it on the heat for another minute or two to ensure it's heated through.

Top with marshmallows and sprinkle with cinnamon sugar if desired.