[Stoves] Differences in stove testing — was Re: ETHOS 2017 agenda and logistics

Cecil,

Your reply (below) was off to such a good start, agreeing to not insist on the same tests to compare apples and oranges (char-making stoves and non-char-making stoves, which might be called ash-making stoves because that is what happens if the char is consumed and the char’s energy is released.)

But then you revert back to the old arguements that rely on equating fuel with enegy.  Those arguements simply do not hold up. 

Analogy:  If a person is thirsty but needs to know if what is in each cup is safe to drink, the tests for a cup of Ice tea and a cup of cold milk should be different, but should answer the necessary questions.  (But let’s not now  discuss tea and milk, okay?)

I heard (from John Mitchel, I think) that the testing protocols are now with some adjustments for char-producing and for ash-producing stoves.  If I understood him correctly, this is in the protocols that are moving forward.  John will be at ETHOS and he (and Ranyee and others) can verify this aspect.

I hope that we can soon stop using “char issues” to condemn the WBT.

**********************
ON THE OTHER HAND, Concerning OTHER issues about the WBT, those should be addressed also.   And regarding that, Xavier  message (below Cecil’s) raises important questions.   For them, I have not heard complete answers yet.

Xavier wrote:     (anyone wanting more info should read his full message with a reprinted abstract of serious studies.)

• do you contest the role of thermodynamic uncertainties (viz. variable steam production and boiling point determination) on results repeatability? Can you ensure there are no uncertainties? Of if there are, can you ensure they have no effect on results repeatability? How?
• do you have an answer to the questions about the rationale of some calculations raised by Zhang et al.?
• do you support the statistical approach recommended by this standardised laboratory-based test to evaluate, communicate and compare performances and emissions of tested stoves, i.e. using the arithmetic average of three replicate tests? How do you guarantee this statistical approach ensure good comparison of stove performances?

I don’t need to be a scientist myself, to understand there is something wrong when I hear these researchers sounding the alarm(s).

From the reprinted abstract, what concerns me is this:

At 99% confidence level, only 15% of all the supposed ‘‘improved” stoves emerged as real ICSs at most. When enough statistical information is provided from WBT results, only the Stove with fan model of cookstoves seemed to reveal real improvements with a probability greater than 93%.

That statement lacks precision about “Stove with fan model”.   But reading between the lines, I interpret that as favorable news for TLUDs with forced air.   And that will probably favorably relate to TLUDs with natural draft WHEN SUFFICIENT NUMBERS ARE TESTED AND COMPARED.

And that same quote is UNfavorable news for most of the ICS “improved” stoves that probably should not be getting so much support or being counted as being progress and accomplishments. 

What could be useful is if FULL access to test DATA (not summaries) could be available for analysis and even some RE-processing.   The problem with this is that testing centers do not release the data without written permission from the maker of the stoves.   I am not sure that GACC or anyone could change that policy.

You will hear more about this at different sessions at ETHOS, including my Saturday evening presentation at ETHOS (with eventual postings to the Stoves Listserv.)  So please stay tuned!!

ETHOS is this weekend.  Some participants might not even see this message (because they are in transsit, etc.)

If you see bright lights in the sky this weekend, that might be from fireworks at ETHOS!!   🙂

Paul

20170127092930.5382229.36909.32846@gmail.com” type=”cite”>

Xavier, Ron, et al:

Your response to Ron is formidable. ‎To my mind, a veritable  tour de force. My pre-scientific intuition tells me that TLUD stoves producing charcoal are a separate category of stoves which should only be compared to other charcoal producing stoves. It’s apples and oranges, apple sauce vs OJ or AJ . We don’t advance the science of apple stoves by applying performance standards which are uniquely appropriate to orange stoves. Or do we? Historically the WBT emerged from Baldwin’s efforts to conceptualize and measure the efficiency of a stove in terms of energy in and useful work outputs. The challenge here is meta stove science which takes us to epistemology – how we know what we claim to know. 

If Ron wants to invent and promote and measure the efficiency of stoves producing copious amounts of biochar and Crispin wants to measure the energy in and work outputs of stoves that make little or no char, then we have no choice but to ask the stove users of the world  to pick the type of stove they want. The appropriate stove test goes back to what kind of stove the costomer wants or traditionally knows about or can be motivated by advertisers, regulators, subsidizers, visionaries, etc to purchase and use. 

I clever anthropologist acquaintance likes to say: “anything you can do I can do meta”. So, what is the meta between Crispin and Ron and the GACC and Aprovecho and the many different types of biomass combustion and work producing devices?? It is simplifying and reasonable to begin with the cook/operator

and purchaser in terms of the stove work‎ desired (cooking experience and space heating), the preferred or available fuels, and the customers willingness to buy or fabricate the stove. 

We stove researchers should start with these givens. Surely it is our job to devise tests of stove performance that assess stoves from the perspective of the primary customer’s best interest and their preferred stove functionalities. Like the old Consumer’s Digest (does it still exist?) we compare stove performance as objectively as possible in terms of a finite number of dimension of stove performance which customers value. 

We may even get groups of stove users to rank (prioritize) different stove functions and also to rate stove performance relative to a baseline traditional stove, and arrive at an overall performance rating comparing end users perceptions and performance scores of particular stoves that are valid and predictive of stove buyer’s choices in the market and also predictive of their propensity to actually use a given stove product. 

If stove scientists are constructing formula that privilage particular stoves – char making stoves or rocket stoves – in an effort to promote their preferred stove products, functions, and testing protocols such actions create biases in favour of one or another type of stove into our common and eventually universal stove performance testing procedures.  Surely it is illigitimate and ultimately counterproductive to rewrite the input output equation for calculating the energy efficiency of stoves in general to accommodate the pecularities of particular stoves. The char produced by the TLUD may have value but it is not permissible to retroactively deduct this output energy from the energy input into the stove! 

We must do our best not to permit our search for objective and universal tests of stove performance to become contaminated with “stovangelism” where we use stoves, subsidies, climate hysterias, and political fantasias to over power and rewrite the priorities, values, and world views of billions of impoverished stove users around the planet. 

It is fine to emphasize the potential benefits of a stove that produces buckets of charcoal for sale or other uses. It is not fair to claim a higher efficiency for a char producing stove by subtracting the energy value of the char from the denominator….because it takes every bit of  the original input to produce the char and also do the cooking and heating. The input of fuelwood does not shrink because some of it turns into charcoal that may or not be useful and valued output. 

Surely we can discover meta testing procedures and inclusive formats for reporting fairly on the strengths of different types of stoves. Nikhil has been calling for the stover of the world to unite and create a common technical language of agreed upon terms , metrics, and testing protocols. 

I recommend that the exponents of particular stoves and testing protocols huddle together and describe how different types of stoves are to be tested in all dimensions of stove performance in the lab and in the field. Once we have the TLUD, rocket, liquid‎ fuel, coal fuel, etc stove test protocols, formulas, and technical terms available we will look for and find a unifying language of terms and operational tests.

That’s what I want!

Cecil the Cook

Sent from my BlackBerry 10 smartphone.

From: Xavier Brandao Sent: Thursday, January 26, 2017 5:38 PM To: Discussion of biomass cooking stoves; adam@instove.org Reply To: Discussion of biomass cooking stoves Cc: ederby@winrock.org Subject: Re: [Stoves] ETHOS 2017 agenda and logistics

Dear Ron,

There are a lot of points of discussion.
Some of the questions are also addressed to me, forgive me and let me know if I don’t answer all the questions you had for me.

Can you (anyone) report on how well the WBT has supported your own internal testing.  Can you think of any approach better than heating/boiling water to come up with fuel consumption comparisons between stoves?

The Heterogeneous Testing Protocol. From our testing team at Prakti, it is a flexible protocol, easy to use and it can perform any cooking task.

I would also note that if the three (?) tests are very different, this could indicate a problem with the stove – not the test or testers.
I don’t see how a stove model who seem to be mass-produced, each unit being exactly the same, can give 3 very different test results.
See the picture here:
www.primestoves.com/img/manufacturing/small-03.jpg

[RWL:  Xavier seems unconcerned about the main issue (the “denominator equation”) separating Crispin and myself – and his reason for unhappiness there is still a mystery.  I still do not understand any detail of Xavier’s concerns – and have earlier responded on each of about 7 cites he sent me.
I am not unconcerned. As I said, I think the denominator equation is an important question, and it is good that you are discussing it with Crispin. There is progress, I believe, in the discussion.
Since I am not a scientist, there is not much I can do or bring to that discussion. There are other important questions being discussed on this list, about health impact, fuels, TLUDs, and many other subjects. I am happy to see them take place, but I cannot contribute much.
Now, do I think the various issues with the WBT are far more important then the denominator equation question? Yes I do.
On the topic of whether or not we should keep the WBT, knowing of all these issues, I believe I can contribute. Because this discussion is important to project implementers, business managers, decision-makers. People like Vahid and Camilla depend directly on the testing protocols in place to run their business successfully.

I still do not understand any detail of Xavier’s concerns
I thought I was clear, but maybe I didn’t express myself very clearly.
To me, it is very simple.
There is a growing number of practitioners complaining about the variability with WBT results.
There is a growing number of studies pointing at intrinsic flaws inside the WBT protocols, both on metrics and repeatability. The studies tell that it is impossible to know really how a stove performs, because of the margin of error.
When I make a stove, I want to know if it is performant. I, unfortunately, have to test it for that. A testing protocol which results are as uncertain as the lottery is of no use to me.
How could I not be concerned?

This, below, this is what concerns me:
Long version:
” However, different authors have been raising doubts about the consistency of WBT results, focusing in particular on three issues: (i) L’Orange et al. [6] highlighted the role of thermodynamic uncertainties (viz. variable steam production and boiling point determination) on results repeatability; (ii) Zhang et al. [7] raised questions about the rationale of some calculations and about metrics terminology; (iii) finally, Wang et al. [8] criticised the statistical approach recommended by this standardised laboratory-based test to evaluate, communicate and compare performances and emissions of tested stoves, i.e. using the arithmetic average of three replicate tests.”
“The results suggest how considering only the mean values of the outputs of the WBT and neglecting intrinsic uncertainties of the results may lead to make large errors and misinterpretations regarding the ICSs’ performance. Indeed, for all the three Classes analysed, at 90% degree of confidence, the percentage of ‘‘improved” stoves obtained by considering the mean values of the WBT is among 3 and 6 times higher than the percentage resulted from this analysis at least. At 99% confidence level, only 15% of all the supposed ‘‘improved” stoves emerged as real ICSs at most. When enough statistical information is provided from WBT results, only the Stove with fan model of cookstoves seemed to reveal real improvements with a probability greater than 93%. This work shows how neglecting the epistemic statistical uncertainties originated from WBTs – as done by a large portion of the literature, which reports results from few lab-tests replicates without sufficient statistical information – might lead to misinterpreted evaluations of ICSs’ performance, with potential negative impact on beneficiaries.”
The short version is enough to feel very concerned: “This work shows how neglecting the epistemic statistical uncertainties originated from WBTs – as done by a large portion of the literature, which reports results from few lab-tests replicates without sufficient statistical information – might lead to misinterpreted evaluations of ICSs’ performance, with potential negative impact on beneficiaries.“

I haven’t seen your answer to the critiques raised by the studies.

Ron, maybe you are able to answer the many questions all these authors are raising in their researches, so I would like to re-ask you these questions:

• do you contest the role of thermodynamic uncertainties (viz. variable steam production and boiling point determination) on results repeatability? Can you ensure there are no uncertainties? Of if there are, can you ensure they have no effect on results repeatability? How?
• do you have an answer to the questions about the rationale of some calculations raised by Zhang et al.?
• do you support the statistical approach recommended by this standardised laboratory-based test to evaluate, communicate and compare performances and emissions of tested stoves, i.e. using the arithmetic average of three replicate tests? How do you guarantee this statistical approach ensure good comparison of stove performances?

I don’t need to be a scientist myself, to understand there is something wrong when I hear these researchers sounding the alarm(s).
When you are an administrator running a hospital, and both researchers and patients tell you that one drug is harmful, and you hear nothing from the supporters of that drug, I believe your role is to listen to the alarms and stop distributing the drug. You don’t need to become a chemist yourself, get a PhD and understand everything about the inner workings of the drug to make a decision.
This is the precautionary principle.
The GACC is the closest we have from an administrator.
There’s a song which says: “inaction is a weapon of mass destruction”.

Best,

Xavier