France’s largest union for air traffic controllers (Syndicat national des contrôleurs du trafic aérien , or SNCTA) has called on workers to walk out in a nationwide strike on Friday, September 15th.
Not enough bank holidays in September I guess, so had to invent one.
For anyone who’s interested and who’s ever had to squirm in post-incident reviews… the preliminary report into last month’s incident is available here
Great link Gareth. No time to read in depth now, but it does confirm shitty code at fault. But there’s more fun in that when the shitty code fell over the backup shitty code did execly the same thing
Some fall back.
I’ve now had a chance to read the whole thing, and sad reading it makes.
The subsystem which interprets the flight data records found invalid data in one record, instead of being designed to flag that record for human intervention and moving on to the next record it crashed. The crash triggered a fallback to the backup system in another location. A process that should happen for hardware or power failure but is pointless for logic failure if the backup system is using the same code, which it was. The backup system started up and correctly started processing where the production system had stopped, and (surprise, surprise) it crashed for exactly the same reason as the primary system. Then a whole load of first level, second level and ultimately developers started running around like headless chickens.
Two of the three recommendations of the report:
- The addition of specific message filters into the data flow between IFPS and FPRSA-R to filter out any flight plans that fit the conditions that caused the incident. (or I would add, any other conditions)
- A permanent software change by the manufacturer within the FPRSA-R sub-system which will prevent the critical exception from recurring for any flight plan that triggers the conditions (or I would add, any other conditions) that led to the incident.
That the update to the software mentioned above could be designed, written, tested and delivered to NATS by September 4th indicates what a small piece of code it is, probably only a few lines long.
Top three areas identified for further investigation by the report:
- The initial requirements specification of the software in the FPRSA-R sub-system.
- The detailed design, coding, testing and validation of the FPRSA-R software by the
manufacturer. - The NATS testing of the FPRSA-R sub-system when delivered in 2018.
Bottom line, a badly designed, written and tested system crashed due to one item of bad data in one data record. All in all, in my professional view, a shoddy piece of work by the supplier and the client.
I wonder who wrote the code, who’s “the Manufacturer” ![]()
Sometimes, consistency is NOT everything. ![]()
![]()
I wonder a little if someone knew a weakness, organised a little tweek to the code. Would NATS admit they had a ransom message (did they get one?) and can we be sure this is the only weakness?
I would be confident that this was an own goal, not a security breech. I don’t think the code that caused this crash is complex or that sophisticated, all it seems to do is extract data and package it for presentation to ATCs. Therefore, given it crashed for such a trivial reason I would have no confidence that there aren’t other data combinations that could cause a similar event. I’d have one of my best and most trusted people dissect it before I’d put my name to any “guarantee”.
Edit: Reading the recommendations again renforces my view. The focus on fixing the “condition” that caused the problem is Whac-A-Mole City. The recommendation should have been to fix that condition and extensively test for other possible conditions.
Michael O’Leary shares his views on the outage ![]()
https://simpleflying.com/ryanairs-michael-oleary-says-heads-should-roll-at-nats/
Who actually gives a damn what O’Leary says. He started off on a good thing and did well, then turned into a numpty. Disgusting bloke
Well love or hate him/Ryanair, it has given many of us the chance to fly cheaply we might never have had. He has done a lot better than others who tried and failed with cheap airline prices and has kept a lot of other businesses afloat too! I could not have got back to the UK when my mother had a near fatal stroke so fast and cheap.
Freddie Laker started the whole economy flights thing.
Yes, I flew on one of his DC10 jets to Venice on a Club 18-30 holiday but he did not last very long.
In part because he was hounded out by the big guys.
Wasn’t it BA which caused the downfall?
Yes, I believe BA is now a little more subtle in squeezing out competition. eg IIRC miraciously announcing and opening flights to San Jose CA soon after Norse (that later went bankrupt although actually for different reasons) did. And miraculously, after the demise of the relevant upstart airline, routes they might have competed on seem to close again, not so long after.