Rega P3 - Arm hole too small?

I have had my P3 for a while now and just change my cartridge. I now need a 4mm spacer and the regualr nut is no good so I had to get the extended nut. Problem is that the nut doesn't fit in the hole. I dremeled out the hole bigger but the hole is now slightly irregular. I only had to dremel the bottom half and not go near the top where the plinth is. Is this a problem at all? I'm thinking as long as the nut is tight I should be okay. Am I wrong?

It should not be an issue if you use a conventional protractor to set the overhang. However, if you use the Rega three hole mounting scheme, the stylus will not be in the proper plane, though this mounting does not conform to the common Baerwald formulas anyway.
Just out of curiousity why would it not work anymore with a Rega protractor? I should mention I did not touch the upper surface of the the hole. The top of the whole (plinth)is the same diamater as before just about 5mm and under all the way to the bottom is where I dremeled. So 5mm under the plinth the whole becomes irregular.

Thanks again, I am a little inept when it comes to working with my hands so I just want to know if I caused any harm.
The overhang, which is fixed with the three hole mounting, is determined by the pivot to spindle distance. You have changed this distance if you have moved the position of the arm. With two hole mounting the play in the headshell will allow you to compensate by moving the cartridge in the headshell.
The arm distance pivot to spindle should be 222mm for a Rega P3.That is from the exact center of the tonearm's pivot to the exact center of the turntable spindle.Now I think you have not altered your tonearm's position from what you have said so you should be o.k.The thing that is actually more important here is how much you tighten the base nut as this has a big effect on the table's sound.Too tight and you will deaden the music too loose and the music will be diffuse.Try only tightening the nut to finger tight and then just a 16th of a turn more.Using your ears as a guide be patient as it can be labourious but try to listen for the 'sweet' spot,the sound will open out when you get it right.Yes it is worth doing.
Thanks Stephan that is exactly what I did.

So you dont't thing the bottom half of the arm hole being irregular is a problem in any way?
As long as you can tighten the base-nut and the arm is the correct distance for the most part it should be o.k.Now what could also be done is to make support that would cover the hole area so you are tightening the base-nut flush and firmly to the bottom of the plinth.I cut a square piece of hardwood with a 23mm hole in it, that I can use for this purpose.The material is up to you but I would avoid MDF(too dead).Do not make it too big,mine is about 65x40mm and about 10-15mm deep.The regular base-nut can be tightened on to this.Decoupling the arm fixture is said to be an advantage on the Rega if it breaks up spurious resonances.So you can experiment here with size of the support and it's material(Acrylic?).There could be an advantage in having the arm the way it is with less contact.Hope this helps.