My understanding is unless you've declared your car as Exported or SORN in the UK then you are still liable for road tax and they'll chase you for it at your old address. So export the car with the UK (see below from DVLA website) or carry on paying UK road tax until you've re-registered it in France and then Export it.
Taking your vehicle abroad for more than 12 months (permanent export)
When a vehicle registered in the United Kingdom (UK) is taken out of the country for 12 months or more, it’s regarded as being permanently exported from the UK.
You can tell DVLA by filling in the section ‘Notification of Permanent Export’ (V5C/4) of the vehicle registration certificate (V5C), and send it to DVLA, Swansea SA99 1BD. Keep the rest of the registration certificate, as you may need this to re-register the vehicle abroad. Your vehicle will become subject to the legal requirements of the new country when exported.
If you don’t have a registration certificate you’ll need to get a certificate of permanent export (V561). Download and complete the V756 ‘Application for certificate or permanent export’ and send to DVLA, Swansea, SA99 1AG.