These days we all pay a lot of attention to claims made about gains in fuel economy offered by this gizmo and that widget. Almost as often, sadly, we’re talking about losses in fuel efficiency brought on by efforts to rid the air of evil emissions. Those losses are always readily verified by real-world experience, the day-in, day-out reality of trucking in 2013. They’re real, they’re credible.
Are the literally countless claims about improvements in fuel economy equally real, equally credible?
Hmmm… well, maybe.
Sure, the gains can be verified the same way losses have been seen in the last decade — by thousands upon thousands of real-world miles. And I’m talking 100,000 or even 200,000 miles or so. Not much good to a marketing person with a new product to introduce. Or a truck owner trying to decide how to spec his next horseless carriage.
Engineers will say that computer modelling has come so far in recent years that it can reliably predict how well a given truck or engine or component will perform in efficiency terms without putting a single wheel on the road. They’ve mapped, down to the last minute detail of grade and elevation and pavement type, all the main roads in the United States, and a lot of them in Canada. Given a certain spec and gross weight, they can sit at their computers and tell you accurately which gear that truck will need to crest a specific hill in Pennsylvania or a mountain pass in British Columbia. They can inject weather and traffic conditions and they can chart the fuel-mileage difference between one tire pressure and another on that truck.
That’s one approach. Another is the good old test track using the industry-standard J1321 test protocol that’s used to define what is and isn’t going to be SmartWay-approved. Otherwise known as the SAE Type II test, approved by TMC (the Technology and Maintenance Council), it’s a rigorous procedure that aims to separate the men from the boys on the fuel-economy battlefront. And in the last few years aerodynamic devices have been increasingly common test subjects, which is a topic I’m about to dive into.
But are test-track results always credible? I think they are, far more often than not, at least in terms of comparing one spec to another. However…
WHY AND HOW COULD RESULTS DIFFER substantially if the testing of a given product is done to the same standardized regime? Honestly, I don’t know, at least not yet. The issue arises in the context of a story I wrote late last week.
I’ll get to the controversial bit in a minute, but first let me offer the story at large starting with some detail on PIT, which isn’t terribly well known at this point. That will change because it’s quite unique, the only organization of its kind in North America.
Based in Montreal, PIT was formed in 2008, some time after Claude Robert had gone to PIT’s mother ship, FPInnovations, with a desire to test some componentry for use in his fleet, Transport Groupe Robert. Other Quebec fleets were soon involved. Things grew from there and now, twice a year, PIT takes over the Blainville track for a full and intense week to do fuel-economy testing of trucks and components. They call the event Energotest.
PIT is unbiased and neutral, a testing organization that aims to help manufacturers evaluate and refine prototypes but also to assist fleets in selecting the best technologies to reduce costs and environmental impact. It works in co-operation with the U.S. SmartWay Transport Partnership, Natural Resources Canada, and Environment Canada. The latter has selected PIT as the benchmark facility for testing green transportation technologies.
Membership in PIT has grown to include 26 fleets and 16 municipalities, as well as government agencies. Fleets pay an annual membership fee based on their size, as little as $12 per vehicle with a $5200 minimum. Manufacturers pay $18,000 to have a product evaluated. Among its customers are the likes of Shell and Hendrickson. Fleet members include Robert, Bison Transport, and Praxair from the U.S.
PIT’s parent, FPInnovations, is among the world’s largest private, not-for-profit forest research centers, helping the Canadian forest industry develop technologies that promote efficiency and sustainable development. PIT is a logical extension of its work developing technologies for logging trucks.
DURING ITS BI-ANNUAL ENERGOTEST event in Blainville two weeks ago, PIT announced the results of performance evaluations on trailers equipped with side skirts and undercarriage aerodynamic devices. The skirts won.
The test results show that trailers with side skirts consumed an average of 6.69% less fuel than similar vehicles without skirts. Trailers with undercarriage aerodynamic devices, often called ‘undertrays’, consumed 1.43% less fuel on average than similar units without the deflectors.
“The goal of these trials was to identify the real value of each technology so fleet operators can focus their implementation efforts where they get the best value and can more easily justify their capital investment,” said Yves Provencher, director of PIT. “Our controlled test-track fuel-efficiency studies accelerate technology implementation and provide the commercial vehicle industry with the information it needs to make sound technology choices.”
Before you ask, and some folks already have, PIT won’t release specific results for any of these products.
The PIT testing was performed in accordance with SAE J1321 Type II test procedures as always. For each test, unmodified control vehicles and test vehicles had the same configuration, were coupled to the same trailers for base and test segments, and maintained load weights the same throughout the entire test period. All vehicles were set to manufacturer’s specs.
As the test protocol demands, fuel consumption was measured by weighing portable tanks before and after each trip. The testing consisted of a baseline segment using non-modified vehicles followed by a segment using the control vehicle and test units equipped with the aerodynamic devices. For baseline and final segments, results were presented as the ratio between the average fuel consumed by the test vehicle and the average fuel consumed by the control vehicle.
AND NOW THE CONTROVERSY. I wasn’t surprised to get a call this past Monday morning from Dave Tyrchniewicz of SmartTruck Canada in Winnipeg, distributors here of the American parent’s trailer-undercarriage air deflectors and other aero devices. He was disputing PIT’s test findings. And no wonder.
SmartTruck claims a 5.5% fuel saving for its basic UT1 undertray (10.5% for the most elaborate UT6 setup), yet PIT testing showed a gain of no more than 2.2% for any of the undercarriage deflectors. And possibly as low as 0%.
As discrepancies go, that’s pretty big.
Now, while I had no wish to get in the middle of a pissing match, I had no choice but to delve a little deeper. Aside from simply being very curious, I owed it to you readers to find an explanation if I could. No luck so far, but I’ll keep pursuing it for sure.
I invited SmartTruck to write me something that would explain how they arrived at their 5.5% figure and I tried to reach Mitch Greenberg, chief commercial officer at SmartTruck Systems, by phone. To this point, I have a written statement from him with the promise of more to follow.
I also arranged a call with PIT director Yves Provencher, a conference call in fact. I dragged along colleagues Steve Bouchard, editor of sister magazine Transport Routier, and Today’s Trucking contributor Jim Park. Jim also writes for HDT magazine in the U.S., as I do, and he had also penned a piece on the PIT test report for HDT’s website, truckinginfo.com. Both Steve and I have spent time on more than one occasion at the Blainville track to witness Energotest trials.
HERE’S WHAT MITCH GREENBERG SAID in his e-mailed statement, unedited:
“SmartTruck has conducted multiple rounds of testing over several years using numerous testing methods to determine the factual fuel savings of its UnderTray systems. These tests always strictly follow all U.S. EPA-recognized testing guidelines and addendums with no exceptions or shortcuts. EPA requires all protocols be followed without exception or variation, or the test results are rejected as invalid and erroneous. This stringency enables the testing methods to minimize variability and any possible compromises of the testing data. Before releasing any product for sale, SmartTruck utilizes SAE Type II (J1321) testing required by EPA’s SmartWay Program, coast down testing as required by EPA’s Heavy Duty Vehicle Fuel Efficiency Regulation and computational fluid dynamic (CFD) testing using world class computing systems — with all rounds of all types of testing providing the same statistically significant fuel savings results. In addition to testing, SmartTruck also has a wide array of customers who have verified these fuel savings results with their own real-world experience with their UnderTray systems.”
I can’t tell you anything about the specifics of all that testing, except to say it was apparently done by a third party. Or third parties plural. Of interest here is the fact that SmartTruck already had SmartWay and CARB approvals before it contracted with PIT in 2011 to test its undercarriage deflectors. It wanted Canadian tests to help with its entry into this market.
Implicit in SmartWay approval is a tested fuel-economy improvement of much more than something in the 0% to 2.2% range reported by PIT. I believe, in fact, it would have had to be 5% or more.
My conference call with Yves Provencher revealed that PIT actually tested SmartTruck’s UT1 undertray system again this past summer, achieving the same results as in the first trial. It did so because some of its fleet members requested a second look.
Provencher led me through the stringent test procedures they employ and said PIT is using the latest SAE Type II protocol, tougher than the one SmartWay requires. Among the differences is the fact that the two trucks, one modified with the component being tested and one in stock trim, now have to be identical. Previously they simply had to be substantially similar.
As I wrote earlier, I have no wish to get in the middle of any dispute between PIT and SmartTruck so I’ll leave the specifics alone from here on. I believe PIT to be above reproach and I’ve spent enough time with its people, at the track and elsewhere, to see how seriously they take their work. I don’t know SmartTruck well at all, but I have no reason to doubt their sincerity. Nor do I have any reason to doubt the veracity of their fuel-economy claims.
Which leaves me with a conundrum: how on earth can these test results be so different?
In fact, I’m left with the makings of an altogether new article; a broad examination of test procedures in general.
Have they just been called into question? Or is this just an isolated anomaly? Stay tuned.
(I should note before I leave this subject that Montreal-based AirFlow Deflector is also SmartWay-approved, and Airman Systems promises a 5% fuel-economy improvement or your money back.)
This blog originally appeared in The Lockwood Report: a bi-weekly newsletter on products, services and other related topics. If you’d like to receive The Lockwood Report, you can sign up here.