As an AE, the cotisations you pay (either monthly or quarterly) are in effect like National Insurance in the UK.
They entitle you to a carte vitale for your health cover from the State which is usually around 70% of any consultation or treatment. In addition, you can choose to pay for top up health insurance with a Mutuelle. There is a confusing array of them and comparing their different offerings is pretty baffling so its best to draw up a list of what health cover you think you might need - do you wear glasses? will anyone in your entourage be needing orthodontic treatment, are you of an age to need much ongoing healthcare etc - then pick three Mutuelles and grill them as to what they offer and how much they reimburse per year for the things you might need. (Remember that emergency treatment (accidents and the like) and long term care for conditions like cancer and heart problems are fully reimbursed by the State. ie they are free)
So, back to tax. In addition to your cotisations, you will need to fill out a 'declaration des revenues' in May (ish) which is effectively a tax return. You fill this out with anyone else in your 'foyer' - partner, child, other dependents. You are all included on one return. In August, you receive an 'Avis des Impots' which tells you if you have to pay any actual tax. But if you are not earning a huge amount, the chances are you won't have any to pay. This document is dead useful for things like a bourse for school fees (if your child goes to a private school) and for the ARS which is a lump sum you are completely entitled to at the beginning of the school year to cover the massive cost of books, cartable, pens etc that French schools demand. You can find out more about this at your local CAF.
Back to tax. You need to register at your local Tresorerie/hotel des Impots/whatever it might be called in your area for Taxe d'Habitation (bins, local street cleaning etc) and Taxe Fonciere. These are billed annually and you are liable to pay them on any property you owned or lived in on the 1st Jan in any year.
Aside from that, there is this thing called CFE which is a Taxe Fonciere on your home if you use it as your office as an AE. Hopefully, you won't be liable for this in your first year (it gets billed in December) and it is unclear whether people have to pay it at all if they are not earning terribly much as an AE. It seems to depend on your area.
I hope this helps! Don't worry overly about missing something. We have found in the three years we have been living here that the folk in the offices are pretty cool and helpful, even if you miss a deadline or omit to register for something.