@allank already answered, but something in your wording reminded me of how I thought when I was starting to learn about PCBs, so … just in case, I want to answer more explicitly. Hopefully the actual experts will correct me on anything I’ve still misunderstood.
Imagine you have a hard wooden table, with some legos on it. Now press a slab of playdough down on top of the legos. The playdough will get a little compacted, but will mostly flow around the legos. The table shouldn’t move (let alone deform), and the legos shouldn’t either, but if you press hard enough, the dough will act almost like water. (I think this is what you meant when you asked if the dielectric layer is conformal – and the answer is “mostly”.)
Afterwards, if you were like me, the legos were poking through. oops. But if you had used enough playdough, then the top of the dough would be fairly flat.
If you pressed hard enough and slowly and carefully enough, then there won’t be many large voids still filled with air next to the legos.
How thick the slab is will depend on how much dough you used, but also on how many legos were also taking up space. When you tip the slab over and try to get the legos out, the parts of the slab right over a lego will be thinner (counting only the dough) than the parts that touched the table.
The core is hard enough to function like that table.
The copper artwork is the legos. Ideally, they’re all the same height with vertical sides, but in reality, some legos are shaped kind of strangely, and the copper doesn’t etch exactly like a rectangular prism.
The prepreg is almost like the dough. The pre-preg’s own copper layer keeps the top of the pre-preg flatter than the top of the playdough. The press itself is flat enough that this mostly holds even when that layer is etched – pre-preg flows up to fill that void, though maybe not perfectly. The final thickness of the prepreg will depend on the copper percentage as well as the initial thickness.
Alas, pre-preg is less consistent than playdough. The resin can flow more easily than the fibers, and they have different thermal and dielectric constants. Different enough to matter sometimes, particularly when a design is sensitive enough that a small air pocket would be a problem. (So the voids aren’t nearly as large as the missing copper, but … the stuff filling them isn’t quite like the rest of the dielectric, which in turn isn’t quite like it was before the press.)
Trying to keep a consistent fiber/resin ratio across the whole board while also keeping the thing flat is one reason to worry about copper balance/copper thieving – if the resin has to flow across the whole board to fill in a void, it has more chances to … not do that.
If your prepreg is thirty times as thick as your copper, then the little bit that filled in a void won’t matter much. If your pre-preg is barely thicker than either surrounding copper layer alone, then … you want the copper balanced pretty well, so no resin has to flow very far, and any resulting change in the effective dielectric or thermal constants is at least uniform across the board, and hopefully something the fab house can predict based on the copper percentage.
And yes, you can remove a layer and use the next layer down as a (more distant) reference layer. The extra distance between them will matter far more than the remains of the gap between the prepreg layers.