Resistance to Instrumentation, Smart Water Grid, Water Technology Innovations

Resistance to the effective use of instrumentation

Hi Fellow Water Quality and Water Security Blog Followers,

This week’s new guest blogger is Oliver Grievson, Flow Compliance & Regulatory Efficiency Manager at Anglian Water Services and fearless leader and moderator of the Water Industry Process Automation and Control (WIPAC) group on Linked in.

A few months ago the Water Industry Process Automation & Control LinkedIn group discussed the subject of Why the Water Industry does not use Instrumentation to its full effectiveness. On the back of this a group paper was produced discussing the subject.

Over the next few months, on a fortnightly basis, this blog will investigate further the different reasons that the instruments that are used daily in our water and wastewater treatment works are not used to their full effectiveness.

The first thing to discuss is,

“What is the definition of an instrument in terms of the works that treat our water & wastewater?”

This can simply be defined as:

“A device that measures the performance or state of a part or whole of a  treatment process”

This could literally be how a pump or blower is performing or how the process in its entirety is performing.

Let’s for example take the modern activated sludge plant as an example of the state of instrumentation within the industry at the current time. It is very typical to have a number of instruments including flow meters, pressure sensors, blanket detectors, and solids monitors. In complex wastewater treatment works they will control the movement of bellmouths, and monitor the levels of solids concentrations, control of the blower speeds based on the dissolved oxygen concentration and on occasion there will be some automated control of the solids wasted from the process.

However it is also very common for manual samples to taken, the automated control switched off and the dissolved oxygen concentration control system switched off.

There is also a step further that could be taken, the benefits of advanced process control has the ability to reduce the power consumption, keep the treatment works controlled so that effluent concentrations are controlled to a selectable level and minimise the chemical use (as required). Additionally more advanced control systems can be used for preventative maintenance and ensure the reliability of the mechanical elements of the treatment works. The question is, if there is so much potential, why is it not used?

This can be summarised in a few simple reasons, future blogs which will be investigated in turn:

(a)  Instrument reliability – There is resistance to the use of instrumentation to its full effectiveness as instruments are perceived not to be reliable. In some cases this is true, insofar as an instrument can be badly installed or installed in the wrong place and the instrument reliability is compromised. In other cases it is because an instrument is poorly maintained

(b)  The threat of instruments – The perceived threat that instrumentation and automation will be used to rationalise the workforce and not perceived as the tool for operators to be more effective and be able to operate rather than be limited to analysis samples for the majority of their working time

(c)  Over-design of the automation system – The use of instrumentation so that the system is over-complicated and un-operable, the relationship between the engineer and the operator.

(d)  Poor use of current data and poor data management – Instrumentation that is currently in place at treatment works normally feeds through to a SCADA system. This data is rarely used to its full effectiveness. The vast majority of data that the instruments produce is generally not used. The data that is used is not used effectively leading to “data richness but information poverty”

(e)  A lack of understanding of what instrumentation can achieve – There is generally a poor knowledge over what instrumentation can achieve in order to deliver process control/advanced process control. A poor integration of the current instrumentation has led to the loss of the vast majority of data and information that instrumentation produces. This has led to poor efficiencies in current process control and the inability to take the step further to utilise instrumentation to its full effectiveness.

(f)   Lack of trust in instrumentation – Instrumentation isn’t trusted from the operator level, to the corporate level or at the regulatory level. Instrumentation cannot be used for regulatory compliance.

All of these factors have led to the instrumentation that is used on a day to day basis at the water industries treatment works to be used inefficiently in terms of what it can and cannot achieve and the professional water and wastewater treatment operators left with only enough time to conduct manual sampling and barely enough time to actually operate the treatment works that they run.

Over the next few weeks each of these subject areas will be investigated and the blog series will investigate what can be done and how it can be realised.


About noahmorgenstern

Entrepreneurial Warlock, mCouponing evangelist, NFC Rabbi, Innovation and Business Intelligence Imam, Secular World Shaker, and General All Around Good Guy


2 thoughts on “Resistance to the effective use of instrumentation

  1. Brilliant insights are usual from Oliver! The first step in curing the patient is to accurately diagnosis the illness which is what Oliver is doing here and in the WIPAC group.

    Thanks, Oliver.

    Posted by John B Cook | January 4, 2012, 1:23 pm
  2. I wanted to add a discussion thread from the Water Security group based on this post to encourage a wider discussion here:

    Steve Cooper • Some interesting points made there although im not entirely sure i agree with all of them. It all depends what you are implying by effective use of instrumentation. Are you aiming for fully automated sites? or just looking to improve things slightly where there is no current instrumentation? The paper is a bit of a mixed bag for me, one one side it appears to be pushing for full automation, yet in the next breath its implying there are sites where controllers spend 99% of their time sampling, which certainly in the developed world won’t be true. However if we are looking at under developed countries, they are less likely to be even considering full automation.
    Also, in either case, and I believe its missing off the white paper, is cost. Instrumentation isnt cheap, if you are a large water company with alot of treatment works, rolling out lets say an automation programme across all your sites will be very expensive.. millions. On the other side of the coin if your site is so deviod of instruments that you have an operator sampling all day, the chances are that the upgrade to your plant of a nice shiney new inline pH meter for 2-3k will also be low down on your priorities list.
    Its an interesting paper, just needs to be more precise in who its targeted at. I would however agree that alot of the points on there in a general context, here is a mis-trust of instruments, and with good reason, they do tend to drift out, they get clogged they get covered in by-products and thats just on clean water and section F on the paper is essentially born out of section A.
    I also find the following statement a bit ‘wishy-washy’………….
    The data that is used is not used effectively leading to “data richness but information poverty”……………
    It’s a statement that say alot, without really saying anything, a properly designed plant wont be collecting data/information that it doesnt need.
    2 days ago• Like• Reply privately• Delete 1

    Oliver Grievson • Steve,

    Glad you enjoyed the paper, at least in parts. Let me clarify some of the points i made.

    In terms of the levels of i strumentation there will of course be a cost benefit analysis with i would say a sliding scale on the levels of instrumentation depending upon the complexity of the works. For example Davyhulme i would expect to be pretty well fully automated as its a complex dual stream asp with tertiary baff where as Irthington with a single biological would have very little.

    When you look at the cost then again you would look to install eithervon a major scheme or as required. For instance at Wellington Dock, the new works for Liverpool will be heavily automated, when you look at this as part of the scheme it will be a fraction of the scheme cost. Retrospective installation doesn’t have to be “millions” a recent business case that i built around mixed liquor monitors had a cost of 60k and a payback of 15 months for 8 treatment works. Poor MLSS control being the most common treatment works problem and despite being in mixed liquor is relatively easy to maintain and cleaned ( but yes it needs maintaining,cleaning and calibrating).

    Finally on the point of data richness and information poverty and using the MLSS analogy. How many plants measure MLSS by instrument, trend it and collect the data and do no more. MLSS can of course be combined with other information to give the total mass of biomass and sludge age, important when controlling the works at the best efficiencies and keepin aeration demand down and preventing overuse of assets.
    2 days ago• Like1

    John B.
    John B. Cook, PE • @Steve,

    Data richness and information poverty–a spot on statement. The industry is very good at collecting copious amounts of data but not converting the data into information, which hopefully leads on to knowledge. Think of data as musical notes. Think of information as a score of music. Think of data as amino acids. Think of information as DNA. We typically collect data because it is very easy to collect and store. But it is never transformed into information, using Shannon’s definition, because we never take the time or don’t have the skill to adequately take the next step in the process of understanding what the “data is telling us”. That is when it becomes information–when it is telling us something about the process and we can assimilate this information to increase our knowledge of the operation and use it to its best and highest use, namely, in making informed (information) decisions. We have then attained a higher level of knowledge in our decision-making capacity.

    So the statement, “Data richness and information poverty” is a simplified, but not simplistic, way of thinking about what is a real problem in the industry. If we have a great alphabet and great words, but those words are never used to communicate information, they are practically useless. That is what Oliver is saying. I hope this helps to clarify this important idea.
    2 days ago• Unlike 1

    Noah Morgenstern (1,000+) • @John and Oliver

    Elegantly put gentlemen.

    @Steve, and everyone else for that matter – Do you have examples of the pricing, recovery cost, and ROI analysis for asset management strategies that you can censor and provide us with to show us these pricing problems? Do you take operational cost savings as a result of instrumentation or the optimization that some automated processes could provide to other aspects of the facility?

    Maybe the problem is the way we look at NPV analysis and model cash flows [with marginal costs/incremental costs included].

    What do you think?

    2 days ago

    John B.
    John B. Cook, PE • @Noah and Oliver,

    Perhaps this is the genesis for a group discussion on how best to demonstrate financial benefits to advanced instrumentation/automation. I think that surely one of the problems is knowing in advance what the real savings will be. This is where theory and experience by the engineer and operator come together, but it would be great to have “an anthology of case studies” to use as support. It is well known that, generally speaking across all industries, there is a reluctance to try a different approach unless it is “tried and true”. Hence, the case studies demonstrating the advantages and disadvantages of a new technology. There are very few “early adopters” of any new technology or approach; hence, the importance to document their experiences. Just a thought, and perhaps it’s already been done somewhere.
    2 days ago• Like

    Follow Fred
    Fred Cohen • So in building your recent business case for $60K with a 15 month ROI for 8 works, how much did the study cost? Please include all the time and money spent by the employees and others who participated (their time may be paid for but it is not free). How did you calculate the lifecycle costs of this effort? Was this just collection of the more data or did it include storage, analysis, backups, etc. What were the liabilities introduced by the added collection? How was the data retention and disposition program effected? What were the regulatory implications? The list goes on and on.

    Here is my point. People don’t like to change things that are working – or even things that are not completely broken. Even if I can break even in 15 months for a $60K investment, if I am making $6M/y, would this be the best investment of my time and effort? In my own very small business, I would not likely consider a $60K investment for break even in 15 months to be a good business decision – and that assumes 100% likelihood of full return with no other added costs or consequences. Among other things, that’s because there are a lot of other ways to make $60K or a lot more in 15 months. Strangely, a $60M investment for 15 month full return at a lower likelihood of success is probably a good business investment (although not for me – I don’t have $60M).
    2 days ago• Like

    Noah Morgenstern (1,000+) • @ Fred

    valid points, all of them. Taking these into account what would be the business environment which would be conducive, in your opinion, to this type of purchase?
    2 days ago

    Follow Steve
    Steve Cooper • @ Oliver – now we’re getting a bit more specific. My main point with the paper was that its too broad a statement to fit all instances. I dont believe across the world there is a resistance to effectively use instrumentation, however each of the statements made can be applied to individual circumstances.
    You’re right with the small site requiring less instrumentation, but thats because its probably not needed, not because there is resistance to use it.

    Was the business case you put together put in to action and the project installed? Im not having a dig, im just wondering if it wasnt what were the reasons for not implementing it? We need a few people to mention a few examples as to why their projects for improved automation/instrument upgrades etc were knocked back.

    Fred had made a good point, people dont like to change things that are working, even if some of the tech is 20 years old. Things improve as new works are built and process overhauled but once again cost will play a big factor and the instrumentation used will be the bare minumum to get the task identified in thee original design spec completed.

    The collecting of information and not using it , again doesnt really sit with me as a resistance to use it. For example trending has been mentioned, trending is usually supplied to keep a historic record of whats happened. Handy to look back on if something does go wrong.The fact that no one is using your mlss trend to calclate total mass of biomas is unlikely down to a resistance to use intrumentation but more likely down to the fact that people arent aware that you can actually calculate it. I wasnt until you mentioned it, maybe i should look in to it at our place?

    Going back to Freds point, essentially if it aint broke dont fix it, then when it comes to investing in new instrumentation to uprade a works which technically is running ok, it’s unlikely to get money for that project if there is a works down the road which is failing and in massive need of civils investment.

    We could argue forever and give specific examples to prove and disprove each point. My general opinion is that there are alot of valid points in there, however i dont believe there is a mentality out there to intentionally not use instrumentation effectively. Just my opinion.
    Good point of discussion though none the less.
    1 day ago• Like

    John B.
    John B. Cook, PE • @Steve,

    I would further elucidate Oliver’s points by stating that in working with other industries–whether polymers or auto or metals–they would not be competitive for very long with a resistance to continual improvement, a part of which includes automation. The idea of something simply still working is not acceptable if the profit motive and stiff competition are involved. Businesses which retain the status quo don’t stay in business for long. Imagine Henry Ford being resistant to auto-making because buggy whips “still work”. Yes, they do this work. The great majority of private industries don’t have the luxury of simply raising their prices (rates) when expenses increase. Very often, it’s improve quality or lower expenses, or usually both to retain market share.

    The entire quality movement, which began in the US and was transported to Japanese automakers, would be nonexistent, and the quality of automobiles would be considerably poorer than they are now, if not for this philosophy. As one of the quality gurus stated, and I can’t recall who: “If it’s not broke, we haven’t looked hard enough.” Many industries, aside from the water industry, would find themselves on the ash-heap of history without incremental improvement, and that includes sophisticated levels of automation and robot technology–vastly sophisticated.

    I think what Oliver is trying to determine is, Why the resistance if a new technology improves a process and reduces expenses? Why are we so averse to change in the water industry? I think those are the chief questions he is trying to answer though Oliver is certainly able to speak for himself.
    1 day ago• Like

    Follow Steve
    Steve Cooper • Lets not go there with cars John, if the americans began the quality movement.. where did it all go wrong?
    Anyway.. back on topic… and i agree with what you are inadvertantly alluding to, that in the water industry the fact there is no competition, this could well be a driving factor. The industries you mentioned are all in highly competitive environments, of course they have to be as efficient as possible whilst continually looking to design new products. While the water industry should always be striving to be efficient, the product doesnt change for us, it just has to meet standards. We dont have to produce the next gen ‘i phone’ which is faster and better.
    Like i have said previously.. i dont believe we are adverse to using new technologies, Of course we want better and more efficient process, of course water companies are always looking to optimise.
    Not knowing what can or cant be done with instrumentation may be a factor, but it’s totally different to having a definite resistance to change. Maybe thats the answer, maybe without constant R&D and being up to date with all the latest technologies is the reason why in the developed world we dont have the most efficient and shiniest kit running our plants. But these 2 things i’ve mentioned (competition & knowledge) could definitly be contributing to the perceived perception of resistance.

    Can i ask where the general perception came from that water companies are making a conscious decision NOT to use instrumentation to become more efficient?
    1 day ago• Like

    John B.
    John B. Cook, PE • @Steve,

    I’ll honor your request and not go through the history of why the quality movement began in the US and was transported to Japan, especially the auto industry, but it had a major impact upon US auto sales. Consider GM, for example….

    I’ll let Oliver respond to your last question, but it has been my consistent observation that there is resistance to instrumentation in general, but especially to automation in the water industry. Again, I only have other industries to compare the water industry to. And from my consulting experience, I have observed many examples of ways to improve operations to produce a higher quality effluent or finished water, but as long as the regulation is being met, as you stated earlier, “If it’s not broke, don’t fix it.” The latter statement is a good summary statement of where many engineers and operators are beginning. And if that is the beginning, a priori assumption, then there isn’t going to be any motivation to change for the sake of continual improvement in quality, saving expenses, etc. But that is only my personal experience based upon 35 years. Oliver is a youthful man, and obviously reached his conclusions based upon a different reference point, so I’ll defer to him.
    1 day ago• Like

    Oliver Grievson • Steve,

    I don’t think its a concious decision not to use instrumentation efficiently, it just isn’t in several different ways. Firstly you have operators who have instruments installed and still do manual sampling every day when a once a fornight check on the instrument will do. Secondly you have data from instruments that isn’t used and not converted into information.

    Is it a concious decision? The operator and the manual testing you could argue it is. The inefficient use of what you have and not converting data into information probably not.
    1 day ago• Like

    Follow Steve
    Steve Cooper • Looks like were all starting to agree, which is nice. Well, on the part about it not being a conscious decision at least. I htink the question could better be re-phrased, I think what we are looking at is the lack of external motivation/pressure to change.
    with regard to intrument information, yep maybe there is a bit if ignorance in not knowing what that all singing all dancing shiney piece of kit can actually do. However when plants and processes are designed, far cleverer people than me will spec the needs, and absolute minimum that is required to get the job done will be built.
    I dont agree with checking instruments once a fortnight however.. a daily manual sample is what i would suggest..and is probably a regulatory requirement i should imagine (not checked). You cant have incorrectly treated water going out of the gate for 2 weeks because the chlorine dosing control analyser has drifted and no one has noticed. An extreme example i know, but i believe a valid one!

    John, relax mate i was only winding you up about the american car industry.. you cant argue though that the japanese do build better cars though these days 😛 have a good weekend all!
    21 hours ago• Like

    John B.
    John B. Cook, PE • @Steve,

    Not a problem. I was using the point of how the quality movement infused excellence. The good news is that now the US auto industry has embraced the quality tools the Japanese had been using for years, so I think an American-made car is a very well-built car, indeed. You’ll get no argument from me. I drive American!

    The question of discrete sampling versus using instrumentation to acquire data is a completely different issue and I would refer anyone interested to an earlier post on the necessary frequencies to accurately capture behaviors using Shannon’s Theorem. A daily manual sample might, in some States, be a regulatory requirement, but the percent in the regulatory community who have expert knowledge of this is, well let’s just say at the risk of offending, smallish.
    21 hours ago• Like

    Oliver Grievson • Steve,

    Having been oneof those people who (a) design (b) commission and (c) spec i know that things get missed and the arguement between constructability and operability doesn’t quite gell together. There is a need to be aware of what all the instruments can do in isolation which is where things work but also where they can work together and what data from the instruments can be turned into operation for the operator who on a site that isn’t manned 24/7 at 3 inthe morning, or for the ROCC operator who is looking the site remotely.

    On check sampling frequency there has to be a criticality function, the example of chlorine that you give is valid, however there isn’t one instrument there but three, always cross checking for drift so the chances of one sensor drifting are good when move to three the chance decreases to minimal, however i think you will still find operators check everyday. There isn’t regulatory requirement though, although there is certainly a duty of care and the implications of failling to disinfect are to great not to do a manual check, certainly on manned sites.
    8 hours ago• Like

    Follow Fred
    Fred Cohen • When I last did a security assessment of a water system, I felt a great deal of comfort in the fact that they did manual sampling and chemical analysis in their own lab every day. In order for redundancy to be effective, it nominally should be separate and different. The sensors that were in place should – likely will – detect changes to put the water out of spec in near-real-time. However, when the computers that do the analysis, sensors that gather the data, reporting system that presents the results, or people that manage and operate the system don’t work properly, whether through malice or accident, the independent manual pulling of water samples and testing with chemicals provides assurance that it requires a collusion of at least (in the last case) 3 people to cause water to go bad. Please note that the public also acts as a sensor (the water smells/tastes funny does get reported). The sampling rate has a lot to do with the volumes involved – How many trucks of what do you have to pull up to the reservoir with and pour into it before it has enough effect to cause an actual public safety hazard and how is this detected?

    Returning to Noah’s question: “Taking these into account what would be the business environment which would be conducive, in your opinion, to this type of purchase?”, the analysis of business decisions is generally taken based on a set of specific alternatives. Given the set of identified alternatives, “Which is the best bet?” (literally a bet) is a complex question. It has everything to do with strategic vision, alignment with the business, short- and long-term costs and consequences, cash flows, and so forth, all of which have to be sensible in order to make a decision in favor of an investment. And of course simply sitting on your money can be a big problem – as people with lots of money all over the world are starting to recognize. It rarely gains value in pure safety. As a rule of thumb, businesses won’t make executive decisions about things that aren’t material (let’s say a 5% difference in the overall balance sheet) relative to the company. Smaller decisions are left to the next level in the hierarchy, and so forth down to the janitor who may have the ability to work with the local facility manager to decide to change the brand of cleaning supplies for that particular facility. The risk of the change is that it doesn’t work out and the janitor loses their job – and this goes all the way to the top (executives can lose their job for bad decisions as well).

    So the first criterion for change is that the people at risk feel as if the risk of losing their job is less than the benefit of making the change. And what exactly is the benefit to the decision maker for making a change in instrumentation? If suggested by management as a strategic direction for the company (we will be the most high tech / best operated / best priced / whatever water company in the area / state / nation / world) with proper incentives (employees who make improvements in these areas are rewarded even when the changes don’t work out perfectly), such a change may be considered and may even take place.

    Influencing organizations to change is a complex field in and of itself. I hope this brief introduction is useful to understanding, but don;t take it to your management and use it as a basis to suggest a change without considering the risks to your job by trying to tell management how to manage.
    3 hours ago• Like

    Follow Steve
    Steve Cooper • Well said Fred, some very good points in there.

    Oliver.. again i think were getting too far in to the specifics.. with individual examples of triple val.. which incidently in my opinion isnt really that effective in monitoring drift.. Yes its good if one of the sensors develops a fault.. However, triple val installations tend to come from the same sample point and if the sensors are in a poistion where they could be affected by lime build up, or managenese, or whatever, then all 3 are subject to these parameters and will tend to drift out together. They may all under read, will never hit their alarm point as they are under reading and without manual comaprisons as a rough guide no one will be none the wiser. Yep i know its a loose example but as ive said before were getting in to splitting hairs territory. I assume this post was put up to gain opinions… ive given mine ( more impotant projects/more important regualtory drivers/a lack of understanding with what tech is out there and what i can do/lack of competition etc etc etc) and i believe the way the question is phrased isn’t necessarily true, there are many factors which can make it SEEM like there is a resistance, but i dont think there is a clear conscious decision not to change.
    1 hour ago• Like

    Noah Morgenstern (1,000+) • @Fred – great points. When would you see a manager looking to brand their utility as most hi-tech/best operated/etc? Is this all based on perceived value of this in the PR arena? Will it take a progressive general manager or board member to push this type of agenda or a PR mess someone has to clean up for political gain? What about the the rising issue of workforce changes [less interest, less skilled, organization knowledge being lost, etc] would this be a driver? Do you see the awareness or importance of integrated water management/smart water networks as a driver leading to a management push for instrumentation, APC. etc?

    @Steve – thanks for your views its making for a great discussion. Id urge more people to jump in as well. Re: your points on regulatory drivers – Do you see legislation being changed towards the issue of instrumentation and APC? In your opinion, with the work of SWIG and others like them do you see this changing opinions? How would you best go about it? Re: a lack of understanding of what tech is out there – outside of trade shows, expos, etc are facility managers or operators actively looking or keeping up with technology trends, or do you see it as more of the manufacturers that need to be more progressive in reaching out with education on their technologies [or maybe this would come across highly disingenuous]. What is your opinion on industry drivers that will lead to a quicker uptake of instrumentation, APC, etc? Is the push towards smart cities specifically smart water management have any relevance yet or this is just a nice concept in your eyes?
    42 minutes ago

    Follow Fred
    Fred Cohen • Noah asks: “When would you see a manager looking to brand their utility as most hi-tech/best operated/etc? … ” I think PG&E is an example from the power industry. Look at theior Web site. They want to lead and be the best in some ways – and it is used internally as a basis for proposing actions and making decisions. Now recall that management vision does not always meet reality… PG&E’s years of gas pipeline issues as an example.

    I don’t see this in the water arena unless and until an incident or inability to deliver safe water in adequate volume becomes a public issue. But I don’t run a water company, so I am not really the proper authority to ask. Small changes for constant upgrading and improvement may be doable by some internal folks who have the support and ear of management, but the vast majority of water systems are relatively small and have few people and little in the way of resources for technology advancements. After all, as long as you get clean water at your tap, why do you care whether they use one measurement scheme or another? If they said (to the 100,000 families and 5,000 businesses they service) that for an added bill of $1/year – within 15 months, they will be able to reduce the bill by $1.20/year, would that really make all the difference?

    Posted by noahmorgenstern | January 7, 2012, 5:03 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: