The problem as I see it is that when you are self employed, the business and the person are one entity. Which means that if you are domiciled in France it's not possible for your business to be domiciled in the UK. If it was a ltd co or if you worked for a UK based employer it would be different, the business would be domiciled in one country and you would be domiciled in another therefore you would be a cross border worker. But you live abroad, therefore you are not entitled to NHS cover.
I think that in these circumstances, setting up as a micro entreprise would be the correct thing to do, even though you then go to the UK to do the work. You would pay cotisations on your earnings in France and thus be affiliated to the French social security, which again seems correct since you live in France. Tax would need discussing with the fisc and HMRC to check what they want you to do but I would have thought that technically you should declare your earnings to HMRC as a non resident, then declare it as foreign earnings already taxed in France - so you wouldn't opt for the flat rate tax option as a micro, otherwise you would pay tax twice. It sounds a bit odd but I don't see how else it can work, unless they tell you to declare it and pay tax in France since that's where you are registered.