Home » Air » F-35 Watch » Experts to Study F-35 Software Delays

Experts to Study F-35 Software Delays

by Brendan McGarry on December 26, 2013

F-35_training

The U.S. Defense Department’s top weapons buyer is assembling a team of independent experts to study the F-35 fighter jet’s software development delays.

Frank Kendall, the Pentagon’s undersecretary of defense for acquisition, technology and logistics, was ordered to put together a group to study the issue and submit a report to Congress by March 3 as part of 2014 National Defense Authorization Act, which sets policy goals and spending targets for the fiscal year that began Oct. 1.

President Barack Obama signed the legislation into law today while vacationing with his family in Hawaii.

Air Force Lt. Gen. Christopher Bogdan, who oversees the F-35 Joint Strike Fighter, the Pentagon’s most expensive weapons acquisition program, and auditors from the Government Accountability Office, the investigative arm of Congress, have identified the aircraft’s software development “as an area of risk because of its complexity,” according to an earlier version of a report accompanying the legislation.

The F-35, which is made by Lockheed Martin Corp., requires more than 8 million lines of code, compared with about 2 million for the F-16 and less than 1 million for other fourth-generation fighter aircraft, according to Steve O’Bryan, vice president of F-35 program integration and business development at the Bethesda, Md.-based defense contractor.

O’Bryan in June said Lockheed reassigned some 200 engineers to work on the F-35’s software, many from outside the aeronautics division, with specialties in space, ship-board and sensor technology, O’Bryan said. The company also invested about $100 million to build a second laboratory where the employees work in shifts around the clock to write, test, and verify the code, he said.

By using a more limited version of the software, the Marine Corps plans to begin operational flights of the fighter jet in 2015, followed by the Air Force in 2016 and the Navy in 2019.

The full software package, known as 3F, is designed to support a suite of internal and external weapons, including the GPS-guided Joint Direct Attack Munition, laser-guided Paveway II bomb, Advanced Medium-Range Air-to-Air Missile, and infrared Sidewinder missile.

It wasn’t immediately clear how many or which experts would conduct the software study.

The Pentagon plans to spend $391 billion to develop and build 2,457 Lightning II aircraft, according to budget documents. That’s $4.5 billion, or 1.1 percent, less than a projection from last year due in part to revised labor rates. The stealthy, single-engine jet designed to replace such aircraft as the F-16, A-10, F/A-18, and AV-8B.

The legislation authorizes the department to spend $5.5 billion to buy 29 of the aircraft, including 19 for the Air Force, six for the Marine Corps and four for the Navy, according to the legislative report. The figure doesn’t include funding for research and development or spare parts.

Share |

{ 68 comments… read them below or add one }

Mitch S. December 26, 2013 at 5:55 pm

8 million lines from various vendors and teams.
A few things come to mind:
Windows
China
Stuxnet

I have sympathy for the developers tasked with writing software to handle life critical functions on cutting edge hardware that has to seamlessly and reliably perform a multitude of tasks while being secure from outside mischief.
And all this for a program that comprises thousands of planes, in three main variants, to be used by multiple operators (including other nations).
My advice: Test, test, test again. And create a "red Team" to find vulnerabilities before the stuff is fielded.

Reply

guest December 27, 2013 at 1:08 pm

I don't know how Stuxnet, Windows, and China are connected in any way to the failed F-35 project. I know for sure that your solution, "red team", test*N etc is a long red pipe dream.

Reply

Mitch S. December 27, 2013 at 3:11 pm

The JSF (F35) program likely failed in it's goal of providing a low cost 5th gen fighter by building 3 variants off of one basic plane.
But there still are aircraft that are coming out and they might be pretty good if properly sorted.

The article deals with the massive, complex software required to run the F35 and make it's features function.
Windows is an example of a complex piece of software that often works well (I went from XP to 7) but has vulnerabilities lurking within that are often discovered "the hard way".
China is a possible adversary that has had success getting their hands on our sensitive information and has the technical ability to take advantage of weaknesses.
Stuxnet is an example of how a clever software attack can bypass what were thought to be robust security measures by taking advantage of vulnerabilities in commonly used relatively low level controllers. Also Stuxnet illustrates how software sabotage can choose it's time to activate and cleverly disguise itself to appear like a hardware fault.

Unfortunately you have a point about me dreaming of a robust "Red Team".All too often testing is the first thing cut when budgets get tight, or political pressure is on (see "Obamacare" website).

Reply

tee December 26, 2013 at 6:41 pm

If you think the Obamacare website was bad, just wait till they try to make the "3F" Software work in the " Junk Strike Fighter". And from what I've read on Aviation Week it is more like 20 million lines of code not 8 million. 8 million is just the block 2 software. To put it in Perspective think Microsoft Vista and how well it worked with older hardware ( NOT Very WELL). Remember a lot of the current weapons we are using were built on 8/16-32/64 bit hardware (3rd & 4th Gen Fighters ) and the F-35 use new more powerful data buses etc. etc. Thats just the connectors, with well over 20+ years of coding experience, and making new software work with old hardware, the FUN & DELAYS have just begun!!!

Reply

William_C1 December 26, 2013 at 7:01 pm

You could look at Microsoft Vista, or you could look at Windows 7, which worked very well.

How would sticking to extremely outdated hardware and programming languages help us in the long run? This is a transition that is inevitable.

Reply

tee December 26, 2013 at 7:34 pm

The difference is Win 7 works with most old hardware and Vista didn't.

Reply

Dfens December 28, 2013 at 3:43 pm

The government contracts for all of their software the same way. The more you screw up, the more money they pay. Then everyone wonders why government doesn't work. What the f did you think was going to happen?

Reply

LHS3 January 6, 2014 at 4:51 pm

Good grief why are you mentioning ANY Microsoft SW with Safety Critical Weapons SW? Microsoft 7 and all of their previous versions are not in the same quality ballpark. Listen although they quote lines of code it’s not the true measure of what’s out there, but a way to compare to past contracts. Most of F35 SW is top of the line, and if costs more it’s because its doing more.

Reply

STemplar December 26, 2013 at 8:50 pm

What precisely is an investigation of the blatantly obvious going to cost us I wonder? I bet we spend $100 million to be told LM promised too much, assigned too little resources, and the DoD made too many changes in the middle. Which is what we pretty much know now I think.

Reply

Dfens December 27, 2013 at 12:36 am

Ah yes, the vast government conspiracy to make things cost too much so the contractor will make more money. That one just never gets old, does it?

Reply

Nadnerbus December 26, 2013 at 11:04 pm

I will display my complete ignorance of programing here and ask a question.

Why do they have to reinvent the wheel each time a new aircraft is fielded and write entirely new code for everything, including weapons that are already fielded on current aircraft? Is there really no way to create an "industry standard," so to speak, for the various sub systems that the manufacturer can build and spec to without having to reinvent the system each time?

Reply

Dfens December 27, 2013 at 12:37 am

It's almost like the contractors make more money the longer they can drag out development…

Reply

Chuckiechan Chan December 27, 2013 at 11:26 am

When my browser crashes I say, "Dammit". When it crashes on an airplane you say "Eject!"

Reply

oblatt2 December 27, 2013 at 4:35 pm

When your browser crashes do you make another $20 million, lockheed does every time a major system flaw is found.

Reply

Dfens December 29, 2013 at 2:38 pm

Damn straight.

NathanS December 27, 2013 at 5:04 am
FreeAmerica December 28, 2013 at 11:29 am

NathanS hits it on the head! The difficulties lie in the fact that this is real time software. Even video game software is delayed consistently and that's not real time software. In regards to speed and closure rates this software cant glitch or give delayed information in any way. For the most part there aren't a lot of you on this blog that have any idea of what you are talking about. You just come here to complain and offer no real solutions. National defense of this country has always been and always should be priority number 1. Get used to it people, its expensive.

Reply

Dfens December 31, 2013 at 10:05 am

Right, and it's a lot more expensive if you're stupid.

Reply

exnuke December 27, 2013 at 2:23 pm

Way do they reinvent the wheel? Actually they don't. But adapting to new equipment is still a challenge. There are several things to consider, the first being the microprocessor architecture, The old software may or may not be able to run on it. And if it does run, with the advances made in the architecture there may be a faster way to perform those tasks. For example 32 bit software can and does run on 64 bit microprocessors, however it is much slower than the same software designed to take advantage of the 64 bit architecture and software written for 8 bit machines may or may not work at all. One thing to remember about military electronics is that they are not continually up-dated with the latest and greatest. We have several aircraft out there still operating on 8-bit machines. About the only time the processors are up-graded is when they come out with a new aircraft/missile/ship.

Reply

asdf December 30, 2013 at 9:59 am

planes are not made that often

Reply

Dfens December 27, 2013 at 12:39 am

Yeah, this is what we need is a huge, taxpayer funded study to figure out why a company that makes more profit by dragging out development can't get their software done any faster. That's not incredibly stupid at all.

Reply

Bernard December 27, 2013 at 2:06 pm
Dfens December 28, 2013 at 3:39 pm

The USAF should design their own aircraft. F these damn contractor leaches. They thing they are so damn indispensable. We could get rid of all of them tomorrow and only be better off. The government could take over Boeing's old Wichita, Kansas plant right now and start designing and assembling aircraft there just as well as Boeing or Lockheed can. Any idiot could.

Reply

John Deere December 29, 2013 at 6:23 am

I can understand your position, but it would lose the US military's advantage in hardware procurement if it abandoned the commercial sector.

The US excels in technological equipment because it procures from private companies whose commercial endeavours requires constant innovation and refinement to remain competitive. The US military's ability to tap this dynamic production environment, for the best technical offerings, has provided it with a huge advantage over other nations with centralized State controlled production facilities.

The military procurement system provides a positive feedback loop for the private companies involved, too; they get subsides to invest in otherwise uncommercial applications, thus pushing the boundaries of technology, which can (and has) led to new revolutionary products and new markets.

Reply

Dfens December 29, 2013 at 2:41 pm

What "commercial sector"? We pay these f'ing contractors more to fail. We pay them more to come in late with a weapon that costs many times more than it was supposed to. If it fails, we pay these "commercial sector" defense corporations more to fix it. Is that the kind of "capitalism" you say we can't do without?

TonyC. December 27, 2013 at 6:41 am

The complexity of the F-35 software is driven by the integrated sensor suite. The old way of doing business was each subsystem had it's own software with a central computer to direct the activity. Then there was a need to have dedicated signal processors for each sensor and they didn't really interact. Now the idea is that the sensors will be able to interact in real time and the processors will have access to each other's data. It's not easy to meld the sensor and flight control data while tracking a target in real time.

Reply

CharleyA December 27, 2013 at 7:55 am

All the subsystems still have their own software and firmware. The difference is the subsystems supply much more data to the core processor array than before.

Reply

hibeam December 27, 2013 at 8:45 am

Part of the problem is government quotas. When I was in the business we always had a certain number of people who were the “correct color” for the job at hand. I’ll never forget when a black female software engineer asked me what a stack was. I’m a hardware guy and I still knew what a stack was.

Reply

STemplar December 27, 2013 at 10:17 am

Really, honestly? I'm no benchmark for diversity but I've met plenty of white boys that were as dumb as the squeaky toy my dog's chewing on right now. Again, I'm no affirmative action fan but you can close your eyes in any government contract business and point randomly and find useless knobs as far as the eye can see.

Reply

hibeam December 27, 2013 at 11:25 am

I agree. So why compound the problem with skin color quotas. We had people sitting around doing nothing except being the correct color all day long.

Reply

Bernard December 27, 2013 at 2:12 pm
Ben December 27, 2013 at 4:05 pm

He's not saying non-whites are inherently inferior in any way, he's just pointing out the problem with hiring minorities for the sake of being politically correct. Better to not give two shits about race and hire by skill.

Reply

FreeAmerica December 28, 2013 at 11:32 am

Race has nothing to do with anything. Nothing! The color of your skin says nothing about your character or talent. It should have never been brought up.

Reply

Greg December 27, 2013 at 6:21 pm
Riceball December 28, 2013 at 2:42 pm

He wasn't talking about any specific ethnic/racial group being more or less capable than another, he was complaining about positions being filled by unqualified people simply because there was a quota of people of a certain skin color or gender that needed to be filled. I just think he worded it poorly so that it came out sounding racist instead of about the stupidity of having silly quota based on irrelevant factors like ethnic background and/or gender instead of actual qualifications.

Reply

45K20E4 December 27, 2013 at 9:24 am

Our government needs to go back to the good old competitive bidding process. Prices should be fixed, and any overruns not caused by gov't scope creep are absorbed by the contractor.

Reply

Ben December 27, 2013 at 4:08 pm

But that doesn't benefit our government officials, now does it? If they don't play nice with contractors, they don't get all those nice, fat kickbacks.

Reply

Tad December 27, 2013 at 11:16 am

It would be so much easier if everyone just used flight simulator software in an MMO environment to fight wars. The software just works.

Reply

Lance December 27, 2013 at 2:01 pm

Shows real stupidity here. We at one time did have a 5th gen fighter on in production no software problems. called the F-22. Yet the idiots in DC killed production and go with this Junk Strike fighter instead and it all inferior in performance too.

Reply

William_C1 December 27, 2013 at 4:09 pm

Actually in the days of F-22A LRIP there were all sorts of software and computer hardware issues. The DoD was rather shocked by how far behind the civilian world they'd fallen in that regard. They were no longer driving developments as they had been in the past.

Reply

Wonky Wonka December 27, 2013 at 6:25 pm

Ah, how quickly we forget. F-22 had tons of software problems. Remember the international date line debacle? http://www.defenseindustrydaily.com/f22-squadron-

Reply

STemplar December 27, 2013 at 7:17 pm

A 5th generation air dominance fighter. It's limited strike capability is just that, limited. I'm not a fan of the F35 but we need a good 5th generation strike fighter since that is about 99% of what tacair actually does.

Reply

FreeAmerica December 28, 2013 at 11:35 am

It wasn't made to be superior in performance. It was made to penetrate hostile airspace. The F-22 was made for AIR superiority. They were made to complement each other, not to see which one can turn faster.

Reply

Big-Dean December 27, 2013 at 2:14 pm

Every time I point out the truth about this failing program my comments get deleted :-[

Reply

Hunter76 December 27, 2013 at 6:59 pm

It's Obamacare.

Reply

goguddy December 27, 2013 at 10:10 pm

I suspect censoring. NSA is empowered to black line kernels of truth from comments here. When in doubt, eliminate the entire comment. It has happened to me, more than once.

Reply

beelzebubba December 28, 2013 at 7:15 am

Considering that the F32/35 series of aircraft were the first operational types designed and built to fly relying on computer moderation, one would think that A) the requisite software would have been as important as cutting metal and B) keeping that aspect as simple and tamper resistant (free of tangential sub routines) as possible, would have been a perquisite in the design brief.

Like the proverbial battle plan, that too went out the window on first contact long, long ago.

Reply

Lonnie Williams December 28, 2013 at 8:32 am

The software delays are the critical keystone to this jet, but not the full story. I spent 10 years on the program developing the training for all versions of this jet so when the software lags so does just about everything else – training for sure! Now we have a number of different syllabi because of the software versions on the jets at the training facility. Then maintenance is casing their butt because the 'black boxes' are coded to work with certain versions of software…so round and round it goes where it stops…who knows?

Reply

Big-Dean December 28, 2013 at 2:05 pm

The basic underlying problem is the vendor (Lockhead) views this as a make work project and will do anything to make it last as long as possible therefore making the most dollars possible. They truly don't care about the outcome-only the dollars. The way this program is heading, it will eat up the entire DOD budget in a few years.

But on top of this is the shear incompetence of the project manager (the air force). They have repeated show their inability to mange the project and the vendor and one wonders if the air force will be giving the vendor 20+ years and 6 Trillion dollars to complete this project

Reply

Dfens December 28, 2013 at 3:50 pm

The DoD watch dogs weren't incompetent. They knew that as long as they could stay on the F-35 program they wouldn't have to go anywhere nasty and dirty to fight. They stayed all nice and safe and warm in a luxury building put up just for this big procurement fiasco and had the best of everything in that building. Come on, give them a little credit.

Reply

catherine December 28, 2013 at 4:37 pm

You are all right!!!!

Reply

JohnB December 28, 2013 at 5:40 pm

Most likely, the team of independent experts will marvel at the immaculate political engineering of the Lockheed Martin F-35 Lightning II.

Reply

Dfens December 29, 2013 at 2:50 pm

No, the experts will find some real problem. What they won't find is the root of that problem. What they won't identify is the fact that all the problems stem from the fact that the US government pays these contractors more to fail than they do if they succeed. What this "study" will really end up being is one more whitewash. It will be one more case where your money is wasted by a government that pretends to give a damn about how much of your money they waste.

Reply

JohnB December 29, 2013 at 6:30 pm

Shortly after the completion of the study, the independent experts will accept well paid positions as Lockheed Martin general-purpose advisors. They may retire with full pay twelve months later.

Reply

Dfens December 29, 2013 at 7:43 pm

Damn, you out cynic-ed me. And sadly, you're probably right. After all, Lockheed will want to be seen as trying to "correct the problems". Makes me feel like crap just thinking like they do for even a minute.

Reply

JJ Murray December 30, 2013 at 7:42 am

PLEASE make this program go away!

Reply

oblatt2 December 30, 2013 at 1:24 pm

There is an old saying at Lockheed… "money in garbage out"

The F-35 isnt a project management issue it's a fraud issue. They don't need to call some software experts they need to call the FBI.

Reply

Dfens December 30, 2013 at 4:36 pm

The FBI couldn't do anything. What Lockheed does is immoral, not illegal. And what they do is not confined to Lockheed nor is it even confined to the department of defense. Government pays contractors more to fail at all levels and across all departments. I have a friend who used to be a building contractor. He wanted to get in on a remodeling contract at the local airport. He bid $10,000 to do a job that any home owner would have paid about $10,000 for a contractor to do. His quote was determined to be "unrealistically low" (or some term like that) because all of the building contractors that normally bid on that kind of work were putting in bids in the millions of dollars range. The only real difference between the $10K and the $1M dollar quotes had nothing to do with the quality of the work or materials used. The entire difference was that the contractors familiar with government procurement practices were dragging out the work and inflating the costs of the materials. These practices are so common and so routine that they go on every day. Every day the government wastes your money on all kinds of goods and services that they buy and no one says a thing. There's nothing illegal about it. The only people who turn it illegal are the stupid people who don't understand how to legally milk the system.

Reply

platypusfriend December 31, 2013 at 2:31 pm

Has anyone here actually SEEN CODE for the F-35? ..What development methodology/approach are the developers using? What version control? Are they using Test/Behavior Driven Development? (etc, etc)..

Reply

Dfens January 2, 2014 at 9:32 am

Most of the code involves image processing.

Reply

Gdadl January 1, 2014 at 11:02 am

The solution to the F35 was the already operational F22 (a better plane than the F35 will ever be … and cheaper than the F35 when the final accounting is done on the F35). Oh .., but the dickhead in chief got rid of the F22 production line …

Reply

STemplar January 2, 2014 at 8:26 am

The F22 was not a strike aircraft. It's capabilities to do so are tacked on. I'm no F35 fan and quite frankly our 'need' for either one of these platforms is pretty dubious. Advanced legacy planes with next gen weapons could have accomplished everything either one of these systems ever will, at a fraction of the cost, and we couldn't have written the check and had them operational years ago.

The F22 and F35 were left over ideas from the Cold War with the European theatre in mind. Neither really has the specs we need at all frankly.

Reply

Bat January 1, 2014 at 11:54 am
Long Johann January 1, 2014 at 1:45 pm

I'm guessing the F-35 will have to re-architect its avionics and software before they will ever reach full capability. Perhaps they can use the FACE technical standard from The Open Group to ease the complexity problem, but they will need to add processing power to do so. That's not such a big deal because obsolescence is already affecting the original computer hardware. If we don't start developing modular, segmented, open, hardware-abstracted software capabilities, we'll never reach our goals.

Reply

meanmarine January 2, 2014 at 1:24 pm

By the time the F35 is declared operational and all the bugs are worked out the Chinese and Russians will have already stolen all our secrets and will be building the same airplane using our technology. Then we get to start all over again with a F36

Reply

Tinto January 2, 2014 at 1:27 pm

Hope the same company that did Obamacare is not working on this stuff.

Reply

Steve Dixon January 3, 2014 at 3:09 am

The biggest joke of all will be when "they" figure out how to detect "stealthy" aircraft consistently and reliably ….. then it's back to ACM (dogfighting) with SRAAM's and guns. The F-35 appears to be deficient in the Air Combat area.
I hope I'm wrong. I think the "JOINT" part of this programme is where the real problems arise.

Reply

TaxPayer March 18, 2014 at 10:06 am

Forget the "software", the main problem lay in the meeting rooms in DC where decisions are made by those that do NOT have the knowledge or background listen to those with vested interest. If you stand back and look you'll see that the issue starts with Green Hills then cascades into an uncontrollable runaway bolstered by rent seeking crony capitalism.
Start with the FCC and end with the DOD, look at what is being done and why then start over with a clean slate. In writing software for over 30 years the one thing that has enable success has been to look at the problem from a new prospective then rewrite everything from scratch to deal with past tunnel issues.

Reply

Leave a Comment

Previous post:

Next post: