Saturday, September 26, 2015

Dieselgate: Night of the Long Knives

Link to update

As could have been forecast VW tried to cast a few engineers down the well of forgetfulness, just a small group they would have you know, and they have been suspended

Bernd Osterloh, chairman of VW’s work council and a member of the executive committee, said: “A small group has done damage to our company. We need a climate where mistakes are not hidden.  Müller and Osterloh were speaking at a press conference to unveil the new boss of the troubled German carmaker.
Some of the heads were near the top too, with the directors of development (chief engineers) at VW, Audi, Ulrich Hackenberg and Porsche, Wolfgang Hatz being given their leave as well as the VW chief in the US

This alone would be taken with a small dose of salts, but news from California has it that testing there was tightly monitored and controlled by a number of high ups in the VW management team, closely linked to the recently retired CEO Martin Winterkorn.  It looks like the California testing team is leaking early to avoid being pushed in front of the diesel bus

But electron trails extend far today, and Bild am Sonntag (paywall) has a letter to VW from Bosch, the supplier of VWs engine control software and emissions systems, written in 2007 warning VW that it would be illegal to use Bosch's software to control emissions systems in cars that they sold, but that it was given to VW for testing purposes only.

This clears up the question of where the software came from.

What happened is also a bit clearer.  The diesel engines in question were developed in 2006 to improve VWs performance in the US market.  The development team wanted to use the AdBlue system to control NOx emissions which would cost  ~ 300 for each car.  The pencil pushers said no.  Diesels were expensive enough.

At about that time Martin Winterkorn took over at VW which was having a tough time and his job was to control costs.  The original design team was sidetracked.  Hackenberg and Hatz took over, two of the development directors asked to leave yesterday.


JohnMashey said...

Deja VW

There are certainly people who know about software test gimmicks here :-))

In the 1980s, there were problems with computer performance benchmarking, quite analogous to this VW mess. Competition was fierce, with many $ at stake.

A typical benchmark was source code, which a compiler translated to a binary program to be run and timed.
Computer companies used internal suites of real programs for their own measurements, but the only widely-used-and-reported benchmarks were fairly simple programs, some rather unrealistic.

1) Some people diddled their compilers to generate wonderful code for cases almost never found in real programs, but made a big difference for oft-used industry benchmarks. That's almost Ok, just unrepresentative, although occasionally they went too far, and the method actually broke real programs.

2) But at least one company went further, not just recognizing program constructs, but checking source code for a specific unique input string at the front of the crucial source. Then, they inserted hand-optimized code that the compiler could not have done.
This is a close analog of VW.

In computing, the answer was creation of industry-standard benchmarks gathered from real, substantial programs, and tighten rules around them to inhibit gimmickry.

One such effort was SPEC, founded in 1988 by {Apollo, HP, MIPS,Sun}, but later joined by most computer companies.
It is still going.
That may have seemed like foxes guarding the henhouse, but as I explained (i was a cofounder), foxes can be very good at watching each other :-)

Anyway, when big money is involved, people cheat simple tests, and now that cars are really computers with wheels, it's a lot easier. Tests have to be realistic.

Russell Seitz said...

The Add Blue solution is excellent- and offers refill synergy with windshield washing spays

Now about vaping...

William Connolley said...

> Müller... pledged ... “maximum transparency” in an investigation into how the company cheated emissions tests on diesel cars. The new VW boss did not reveal how many staff had been suspended or who they were.

Oh dear. That's a rather poor start for maximum transparency.

Florifulgurator said...

Pencil pushers and bean counters wreaking havoc. I've seen that in almost every company I worked for here in Germany, big and small. "Business economists" with not the slightest feel for numbers. All boxes have same size. They can add and substract but know not what. I'm sooo fed up with these boneheads.

Is that typical German?

Brian said...

These people who've been cut loose will not feel much loyalty, esp. when the prospect of criminal sanctions gets closer.

Seems like information is already starting to shake out to various media, I wonder if these people are the source.

The only ones who will escape this net are the ones who were smart enough to anticipate this happening, and that doesn't seem to be a large set.

Fernando Leanme said...

The VW problem is similar to the BP problem causing Macondo, the DOW problem at Bophal, the TEPCO at Fukushima, or the numerous cases seen in the USSR (Urals dead zone atomic blast, Chernobyl, Sverdlovsk Anthrax event, etc).

The root causes are in upper management morals (or lack thereof) which creates a flawed rewards and safety culture, which in turn enables people to design, execute, and operate irresponsibly and with serious negligence. I used to hold a job which involved trying to keep wayward managements and assorted assholes from breaking rules or behaving incompentently. It paid well but it was really stressful. Looking at this problem for many years I've concluded the key is to have private interests who know they'll be destroyed if they screw up, and regulators who can't be corrupted so they keep a close eye on these private entities.

I analyzed very closely the performance by state owned companies in the USSR, where this type of problem was huge. It was impossible to create independent regulators because the state owned the companies and also the regulating bodies. In other countries the problem is corruption. And in some cases its broad incompetence caused by a cozy political relationship between regulators and regulated outfits.

One place I think we may see a monster problem is Brazil, where the government and Petrobras have a cozy relationship, we know corruption has been the norm, and they are pushing the technology envelope producing very deep high pressure reservoirs in deep water. I would expect a giant fire and explosion offshore Brazil sometime in the next 10 years.

EliRabett said...

It is truly depressing how those private interests can ignore such risks as long as the profits roll in. They always think they can buy their way out or sell out at the optimal time.

So no Fernando, only the risk of losing the business and going to prison works and the later has vanished in our kleptocracy.

crf said...

Almost surely the upper management knew about and approved of this cheating program. It's also highly likely that they had only planned to use this program as a very short term stopgap until their diesel technology was refined enough to pass emissions testing without cheating. (After all, other car makers apparently were meeting the emission requirements with their diesels.)

But this proved much harder to do, financially and technically, than VW had hoped. And then they couldn't stop the cheating program, as the decrease in performance would reveal the scheme.

Thus they were in a race: trying to develop and unveil new compliant diesel engines before regulators discovered their scheme.

Donald Strong said...

It appears that the German auto industry is unlike US banking, in which virtually nobody sang after the crash.

Everett F Sargent said...


Are you actually trying to be an appeaser for VW?

If VW did not have something on the drawing boards or in the planning stages with respect to their diesel engines meeting current/future emission standards sans the FIX, which internal documents would clearly show (at least for the next 2-3 model years), VW should get fined like 10X whatever the current law allows.

I don't think model years 2009-2016 counts "as a very short term stopgap" that's like EIGHT years of "do nothing except cheat" diesel engineering.

Much more likely, internal documents will show that VW fully intended to Keep on Truckin' ...

VW diesels should have a permanent ban from future sales in the US and all current VW diesels in the US should be crushed and then replaced with a fully compliant petrol model worth 3X whatever the MSRP was on those original diesel VW's.

There are absolutely NO excuses allowed for this type of behavior.

E. Swanson said...

EFS Said:
VW diesels should have a permanent ban from future sales in the US and all current VW diesels in the US should be crushed and then replaced with a fully compliant petrol model worth 3X whatever the MSRP was on those original diesel VW's.

Great, lets kill everybody in the house, just because we know there's a bad guy hanging out inside. Sound familiar? It's the typical US drone attack scenario.

Given that those cars already on the road could be repaired by a simple upgrade of their software via flashing the ECU, why should the owners who have already put their hard earned money into the cars be expected to suffer further? Of course, VW should be held accountable and would be expected to pay some stiff penalty in addition to the cost of the repair. Doing so would be much more than the penalties (not) leveled on those banksters who caused the near meltdown of the world financial system in 2008 and who appear to be again using similar schemes to siphon money out of markets. As long as society allows a philosophy of "Greed is Good" and "A crime isn't committed until someone is caught", "The Rule of Law" will continue to be violated wherever possible.

EliRabett said...

Don't think that a software update will be enough because it will kill mileage and power. If they can retrofit an AdBlue system that might work

crf said...

Everett F Sargeant,

As I said, when VW began cheating, it was on the expectation that it would be for a very short time: just until they had sufficiently developed their technology to meet emission standards (in a cost-effective way).

However, they severely underestimated the technological hurdles they had to overcome. But they still couldn't end the cheating software problem, since that would expose them.

I'm sure that every month that auto industry regulators failed to discover the cheating gave VW more confidence that they would be able to "beat the clock" in finely getting their diesel tech to the level that they could retire the cheating software.

(I'm not trying to excuse VW: merely trying to explain why such a cheating strategy might have been given the go ahead in the first place, and why they had to continue it.)

Everett F Sargent said...


It is very likely that the AdBlue (aka SCR) will not be enough for VW.

The two ICCT reports the EPA statement and other sources have identified the three US Vehicles ...

“Vehicle A” was a 2012 VW Jetta (LNT)
“Vehicle B” was a 2013 VW Passat (SCR)
“Vehicle C” was a BMW X5 (SCR)

From the 15-car ICCT report that means that (from Table 2 page 4) ...

“Vehicle H” was a 2012 VW Jetta (LNT), total trips = 13
“Vehicle F” was a 2013 VW Passat (SCR), total trips = 15
“Vehicle B” was a BMW X5 (SCR), total trips = 8

Make "M1" = BMW and "M2" = VW

M@ had five vehicles, four had SCR (SCR or AdBlue) and one had LNT

ICCT US three car report ...

ICCT 15-car report (AFAIK includes the three car data)...

Also ICCT has a very recent report (with some background materials) titled ...

NOX control technologies for Euro 6 Diesel passenger cars

So, in summary, one of the VW's had an SCR (or AdBlue) system and in "real world" tests (e. g. avoiding the software FIX) it failed miserably "by a factor of 5 to 20" (page ii of the US three car report).

VW is still DAA.

Everett F Sargent said...


You are still making (the exact same) excuses for VW based on nothing more than idle speculation.

I read what you wrote the 1st time.

If you commit the crime you must do the time and you must pay the fine.

JonnieG said...

@Everett F Sarget re; crf comments: It's pretty clear to me that crf is offering an explanation of may have happened in the VW thinking based on available facts and is not making excuses for VW. In fact crf says as much. I, for one, see crf's point. This faulty logic is a theme common with embezzlers: I was gonna's pay it all back when I got back from Las Vegas. Logical? In the narrow context of the excuse, yes - the Universe does this all the time at the quantum level. In the larger (human) context? Illegal.

Brian said...

crf has an interesting speculation, trying to figure out some logical reason for VW to do what it did (setting aside the ethics for a second).

I suppose it's possible. We'll find out in a year or two, maybe sooner.

I'll offer two other possibilities:

1. the European testing systems are made to be gamed, and then some bright lad decided to take that to the nth degree in the US.

2. there is no logical explanation. Some group of people within VW just decided to take the easy way out. It made no sense the first year they did it and made less sense each following year as more and more people became involved, until finally in May 2014 it started to break open, hit disastrous levels and still they soldiered on, with still more people involved.

These two possibilities overlap.

Blogger profile said...

"In the larger (human) context? Illegal."

Unless you're a bank...

Blogger profile said...

"Great, lets kill everybody in the house, just because we know there's a bad guy hanging out inside. Sound familiar? It's the typical US drone attack scenario. "

Of course, if it were GM, the cry would be entirely different.

If it were the EU doing this to GM, the cry would be used to proclaim this a witch hunt against the USA by the evil UN, solely a money grab to shore up their defecit.

Blogger profile said...

"So no Fernando, only the risk of losing the business and going to prison works and the later has vanished in our kleptocracy."

Although it would be punishing shareholders for things they do not know is happening. I would counter this with the fact that shareholders should be demanding more transparency on what the people who they employ via their shared ownership from the company than they get, and punishing them for NOT CARING that they don't know (which may be merely for reasons of "plausible deniability") would be appropriate to make it a requirement for more transparent operations in exchange for their shared ownership. And if not, no longer a plausible excuse.