The modelling of dense plasmas using finite-temperature density functional theory has proven very successful in determining transport properties and the equation of state of systems where quantum many-body effects and correlations play a key role in their structure. Here we show how excited-state projector augmented-wave potentials can be used to extend these calculations to explicitly model core-hole states, allowing for the calculation of the electronic structure of a range of integer charge configurations embedded in a dense plasma environment. Our excited-state potentials show good agreement with all-electron calculations at finite-temperatures, motivating their use as an efficient approach in modelling from first principles both the structure of strongly-coupled non-equilibrium plasmas and their interaction with intense x-rays.