http://www.direct.gov.uk/en/Dl1/Directories/UsefulContactsByCategory/Over50sContacts/DG_178684 This is the website for the International section of the Department of Work and Pensions.
Before moving I would suggest you contact them to discuss your personal situation as, although most benefits are transferable, everyone's situation is different. Benefits can be paid into a bank account in France but are subject to the rate of exchange.
http://www.direct.gov.uk/en/BritonsLivingAbroad/Moneyabroad/DG_4000013 This is the website regarding the International section for the State Pension. This too can be paid overseas but please contact them and provide details.