Interesting this as we have just moved into the Correze from the Jura, where it gets REALLY cold; Oil-fired radiators and large wood burner meant HUGE heating bills that we as pensioners were battling to meet. Big rooms didn't help and cathedral ceiling in the lounge was beautiful and warmed the mezzanine a treat!
Now we have a far older house with no double-glazing, but smaller rooms and a loft conversion. We were lucky that the upstairs has reversible heating/cooling which is great. Heating is supposed to be far cheaper with this system than anything, although the cooling has been a blessing as well. However the downstairs is equipped with the old iron radiators which previously have been oil-fired.
Our heating guy has done an analysis on the place, and as we have a small open fire in the small lounge, has suggested a 'pompe a Chaleur' (I think that's how you spell it). This is also apparently an economical system to run BUT expensive to install, at about €10,000 (ouch!). However it then converts the oil-fired heating system to a water one, and obviates the need to buy fuel. Again apparently it is cheap to run on electricity, but we have yet to see any bills so won't know for sure until next year.
However one thing we DO know is that spending €2000 a year in fuel oil - with every sign of it getting even more expensive, the investment seems practical ultimately - and is a permanent asset to the house if at any stage we want to sell again. Still decimates the old savings though.