We study ordered configuration spaces of compact manifolds with boundary. We show that for a large class of such manifolds, the real homotopy type of the configuration spaces only depends on the real homotopy type of the pair consisting of the manifold and its boundary. We moreover describe explicit real models of these configuration spaces using three different approaches. We do this by adapting previous constructions for configuration spaces of closed manifolds which relied on Kontsevich's proof of the formality of the little disks operads. We also prove that our models are compatible with the richer structure of configuration spaces, respectively a module over the Swiss-Cheese operad, a module over the associative algebra of configurations in a collar around the boundary of the manifold, and a module over the little disks operad.