Computational methods

Data science involves the combined application of statistical and computational methods with domain knowledge to understand, visualize and make predictions, and thus derive value from data in that domain. It encompasses data analysis, Big Data and machine learning.  

Big Data has an important role to play in ensuring all the sustainable development goals, and not only that for water. Real-time data from water and environmental sensors over a timescale of seconds, minutes and hours can generate big data and will need appropriate methodologies and models to be developed to derive insights and value from them. 

Big Data is huge in volume and grows exponentially. It is distinguished from normal data by its four principal attributes, known as the four V’s-  volume, velocity, variety, and veracity. Simple examples could be data from all the sensors in the aircraft, or the number of messages on a social media platform.  

Volume-  Data on the order of terabytes (1012), petabytes (1015) and larger require different information processing systems to normal data. 

Velocity– The rate at which data is created, stored, processed and analyzed. 

Veracity- Measures how accurate and reliable the data, such as the amount of bias or uncertainty, eg. noise in the readings from IoT sensors.   

Variety– Structured data can be classified into fields such as names, addresses, water quality values. Unstructured data cannot be classified easily into fields such as satellite images, paragraphs of written text, photos and graphic images, videos, streaming instrument data, webpages, and document files. Semi-structured data which is a type of combination of structured and unstructured data is also possible. 

Machine learning is used for predictive data analysis in an automated way. It is based on pattern recognition and enables computers to learn without being programmed to perform specific tasks. Using machine-learning algorithms, computers can learn from data and make predictions in the same way as humans learn from experience. As machine-learning models are exposed to new data, they can adapt, and their accuracy is improved as they learn from previous computations to produce more reliable, repeatable decisions and results.   

Artificial neural networks (ANNs) are used throughout the water industry for finding the complex and nonlinear relationships between input and output parameters.  ANN contains interconnected set of neurons allowing computers to learn in an analogous but simplified way to humans by learning the hidden patterns between known input and output data points, which is known as training. During training, the weighting factors attached to the output of each neuron are optimized. When new input data of a similar type to the trained data is presented to the ANN the output data can be predicted, eg. the input data could be the water level in a lake over the last few years, and the prediction could be made for the level next year.