Just gonna put this out there in case anyone does wanna do this technique with sch 40.
Your dremel will be your best friend. Any sanding bit doing 20,000 rpm will sand off a good amount of pvc so here's my technique.
1. Cut out your design.
2. Take your hilt and put the top of it underneath the pvc to trace the hilts outer diameter on the pvc's inner.
3. Sand the ID to match your trace erase that sharpie line completely but keep in mind to not go further into the shroud. The height of the bit should be the farthest you go. Wash off the dust and test the fit. Where the hilt stops is where you trace another line for you to sand off.
My mistake was going in deeper, it made it hard for me to tell which parts I needed to sand.
4. Repeat.
5. Hopefully profit.