EV software delays threaten to become “a disaster” for the German automakers

When I interviewed Tesla co-founder Ian Wright in 2014 for my history of Tesla, he described the software found in traditional automobiles as “a dog’s breakfast,” by which he meant that it was a motley mixture of heterogenous hardware and software, in contrast to Tesla’s elegant unified system, which controls every function of the vehicle.

“I’m looking out the window at my 2008 Volkswagen Touareg, and I bet that’s got sixty or seventy electronic black boxes, three hundred pounds of wiring harness, and software from twenty different companies in it,” Wright told me.

The fact that he used a VW as an example was a coincidence—the same observation would have been true of just about any other legacy vehicle. However, his comments seem particularly poignant today, in light of the recent revelation that Volkswagen is having serious difficulties getting its EV software ready in time for the upcoming launch of the ID.3.

Volkswagen and other automakers know perfectly well that they’ve fallen far behind Tesla, not only in battery technology, but also in the software expertise that will be mission-critical for future generations of electric, connected, autonomous vehicles. As CleanTechnica’s Zachary Shahan reported in a recent article, Volkswagen is moving to take control of its software development. It has established a new business unit called Car.Software, with a mission to bring more software development in-house.

In an article published on Volkswagen’s site last June, Board Member Christian Senger, responsible for Digital Car & Services, explained the company’s new software strategy. “Currently, up to 70 control units operating with software from 200 different suppliers must be networked in vehicles of the Volkswagen brand. We devote a large part of our energy to technical integration and rely very much on the developments of third parties. This is not a good model for the future. We need to be the ones who develop the software, set the standards and make them available to all brands and suppliers.”

“Today, our share [of self-developed software] is less than 10 percent,” Senger continued. “That is clearly too small. In the Volkswagen Group, we want to achieve a share in software development of more than 60 percent by 2025.”

“We have organizationally separated the development of hardware and software in the company, and incidentally, we’re the first car manufacturer to do this. This is important because software follows much faster development cycles. We want to form an agile Car.Software unit, and bring together more than 5,000 experts and top talent by 2025. We will bring together experts from brands and companies within the Group, but we want to focus much more on attracting top professionals from the IT and tech industry.”

A worthy goal, but, as the German newspaper Süddeutsche Zeitung recently reported, VW and the other German automakers are struggling to accomplish it. To take control of their software development, they’ll need armies of cutting-edge (read: young) software experts. However, such hipsters are in high demand, and it’s proving difficult to entice them to work in the auto industry. “It’s an absolute disaster,” an anonymous insider told the Zeitung. “We’re simply not getting the people.”

The problem is shared by all the German automakers. According to SZ, top management from the Volkswagen Group met in secret with their counterparts at Daimler to explore the possibility of working together on software development, not only to save costs, but to share their limited pools of coding skills. However, it was soon revealed that Daimler had already been making similar overtures to BMW. Close cooperation among all three brands would risk running afoul of German antitrust law, so the collaboration appears to be a dead letter for the moment.

Asked what he saw see as a top priority for software development, Christian Senger said, “safeguarding the timely production start of our cars.” With eloquent understatement, CleanTechnica’s Zachary Shahan calls this comment “a bit unfortunate based on recent news.”

Sources cited in the Süddeutsche Zeitung article held out little hope that the ID.3 would be delivered on time: “This is no longer a laughing matter. The car is far from ready for the market.” Insiders say the software shortfall “threatens to become a disaster.”

Of course, it’s nothing new for the launch of a new vehicle to be delayed. “This in itself wouldn’t be unusual,” says SZ. In this case, however, VW has little room for error. For one thing, VW chief Herbert Diess has invested personal prestige in the ID.3—it’s the centerpiece of his plans to radically remake VW as an electric automaker. For another, VW is bringing the ID.3 to market in order to comply with ever-stricter EU CO₂ standards. The company needs to have 100,000 e-autos on the streets by the end of this year in order to avoid fines.

At the moment, VW’s plan seems to be to proceed with producing the ID.3s sans software, and to store them in a giant parking lot until the necessary software is ready. SZ notes that the picture of these empty steel hulls sitting there waiting to be brought to life illustrates the situation that Germany’s automakers most fear: that they will become providers of a low-margin commodity product, while others, perhaps Google or Apple, reap the big profits that everyone expects to come from monetizing data about car buyers and drivers. German sentences are notoriously difficult to translate into English, but the Süddeutsche Zeitung’s conclusion goes something like this: “If the Germans can’t field competitive software, they will languish as mere delivery boys—a horror scenario for the proud German auto industry.”

Sources: CleanTechnica, Süddeutsche Zeitung
Image: Volkswagen AG

  • Bacila

    Hmm… Looking from an IT perspective I would see this as an opportunity for the EU wide regulation: make EU wide standards for the automotive component interfaces. There was already an attempt to make OBDII as a diagnostic standard. This time initiative can go from the automakers themselves. Otherwise it will be a disaster because development hell will go to OEM side as they will need to prepare device X with protocols for VAG, and almost or even the same device for Daimler with different protocol, then third protocol for BMW and the list goes on.

    • Роман Васильевич

      Defacto UDS IS EU (Maybe even WW) diagnostic standard ALREADY. Do you familiar with ISO14229, 15765, 22900, 22901, J2534? Maybe familiar with a ODX?

  • Marc

    The group that makes the Taycan, e-tron, e-golf etc suddenly can’t make their own software work? Yeah right! 😂

    • bytrain

      What makes you think that those cars were delivered without software problems? The main difference between the ID.3 and those you mention is that those were all small scale halo or compliance cars where software delays didn’t result in big headlines and a risk to the group’s future direction. https://www.electrive.com/2018/12/17/audi-confirms-e-tron-electric-car-delayed-until-2019/

      • Marc

        ‘What makes you think that those cars were delivered without software problems?’-Any problems would have been brought up in reviews. But if you know otherwise, prove it. As for the ‘compliance cars’ e-tron and e-golf. They are both outselling tesla in Europe. And the Taycan is selling in huge numbers for Porsche. I think you are very misinformed.

        • bytrain

          Marc, with regard to your first point “But if you know otherwise, prove it,” that is precisely why I included the link to the article stating that the e-tron was delayed almost a year as a result of software issues. With regard to your second point about the e-tron and e-Golf outselling Tesla in Europe, the numbers for 2019 were Tesla Model 3: 95,247; VW e-Golf: 28,710; and Audi e-tron: 18,483. Meaning Tesla outsold those two combined by a 2 to 1 margin. And the Porsche Taycan didn’t even break into the top 20 list. Which begs the question: Why would you consider me misinformed when you obviously can’t even be bothered to research the facts before making such false statements? And here is my source: https://insideevs.com/news/394870/plugin-sales-europe-record-december-2019/

          • Marc

            You are misinformed because you are comparing figures for Tesla that are over a year old, when the E-tron wasn’t even on sale. As for sale Taycan, deliveries where only recently made. https://uploads.disquscdn.com/images/711b39d83d7689ee9dcdb5ce5df3ff29316c72a7d49ccf86583b7aa29caca23e.jpg https://uploads.disquscdn.com/images/7584305f7095b535177d9c91e92acfd5092d8a5381ef41e20afd9e73c2c47750.png Did you think that I wouldn’t know? Since late last year tesla have been outsold across Europe as I said.

          • bytrain

            Actually, since the Tesla and VW Group data was for 2019, and 2019 was only 3 months and 4 days ago, I don’t see how you can say it was over a year old. And the data did include the e-tron as the provided link clearly demonstrated. I honestly felt badly for only using a year’s worth of data. Until you tried to make a point with only two months worth of data. If you want to restate your original assertion as “the e-Golf outsold the Model 3 in Europe during the first two months of 2020,” then, based on the table you provided, I would have to agree with you. But that’s much different from your original broad assertion.

          • Marc

            It actually started last year around November in Norway and the Netherlands. Due to the fact that the E-tron became more widely available. March figures are similar. In Norway the satisfaction rating for Tesla has dropped massively due to all the problems owners have been experiencing which also didn’t help. https://uploads.disquscdn.com/images/ba32ae031360d4658e24b309ac489fed816adfb4278b22775a95848ff5bedc7a.png

  • Stuart McColl

    Here is the most important line in this article >>> We have organizationally separated the development of hardware and software in the company <<< no wonder … read the methods of Elon Musk at Tesla or SpaceX … and that is where Elon would say the problem is primarily. I like the idea of setting the standards as they say they are trying to do … but they'll need to keep hardware and software folks as close as possible … once a standard is set … and still while that standard is delivered … you can release hardware to be made under a certified architecture … but you can't categorically separate the 2. Hmmmm … should I buy VW stock ??? Ahhh … after reading this … ahhhhhh …. NO !

    • Роман Васильевич

      Unfortunately and no wonder VW automotive electronic architecture is a peice of sh…

  • Stuart McColl

    Also … I’ve heard directly from a friend on mine that works at Tesla in Germany … Germans have a problem challenging the boss … even when the boss is clearly wrong. In certain situations maybe that works … but not in a techno leveraged fast moving must catch up with Tesla auto techno scenario. The people in these companies like VW are not disrupters, so the best ideas won’t make it to the top, where they can be executed on. Read Steve Jobs book … shouting brawls in Apple often had to take place with workers shouting back and forth with Steve himself to in the end find the best answer to a problem. Similar with Elon at times in his early days.

    • Роман Васильевич

      I dont undestand people when they doesn protect their opinion. Especially when we talk about a hazardous systems that CAN do a harm to lots of peoples. I Live in Russia and its a big problem in our country. Nobody wants to solve the problems and to talk about it. So im a SW development teamlead and I WANT to know about a problems in a SW development process in my team, we spent a lot of time for feedback, its a very important. So if we talk about me, there is no need to challenge me (as a boss) to describe some unwanted behavior, e.g.

  • http://www.lifefulfilling.com/ Roger J. Zamofing

    In German “Hochmut kommt vor dem Fall” (greed leads to a sorry end). This is the price to pay if you ignore any attempt to give constructive feedback or try to consult for an innovative production or e.g. have a serious project for a new type of very simple combustion hybrid engine as a perfect bridge technology toward better electric vehicles.

    We tried already in 1994 in Hannover with some of the board members present and after asking 7 questions to 8 systems qualities to give advice on personel development towards a more innovative cooperation. The only arrogant thing to speak about, was the “trick” we used to do our job but not the results showing terrible weakness in 3 of 8 systems qualities. Lies about reality are at the beginning of any (thinking-) catastrophe and the arrogance of closed systems will do the rest, so the “Thousand-Year Reich” is sinking due to the top management people and not the workers or engineers doing a good job.

    The same “old” qualities however lasted till now at present times which seem to still stem from the questionable personal values of the founders families. But as long as everybody was buying VW that obviously was not a problem to them. Among the weaknesses was also what Stuart McColl writes about. Too many just followers and not so many responsible thinkers with personal integrity. Everybody was in fear to say something aginst a boss in command and that in NO SYSTEM can go well for a long time. If they do not learn from failures now, they will never… Elon Musk was to the industry what Apple was to Nokia, Motorola and Ericsson. Also they did not wanted see what was actually happening to them and others will size the chances, e.g. Nio, Byton, BYD, Geely, Tata and many more to come.

    The same hierarchical problems however we saw in consulting at Mercedes-Benz in the factory producing the S-Class (Top model of the line) once, where the workers already knew about some electronics problems at that time and had also the SOLUTION ready worked up in their spare time. They only had 4 or 5 black boxes just in one door to handle all functions and this was not working as designed by the engineers. The workers did not feedback because they claimed not being responsible for the ones having their private parking lot and white collars, earning 10 times more then themselves.

    • Роман Васильевич

      Hello! Im sure your opinion is very close to reality, you are right! Im working in an automotive company that develops and produces a lot of automotive ECUs and we have actually the same problem, we have “a couple” of very good engineers with a small salary and a big PM department. I think it s a common problem in a big corporations, where a simple engineer have no voice.