Skip to main content

Posts

Lessons I have learnt during E3SM development

I have been involved with the E3SM development since I joined PNNL as a postdoc. Over the course of time, I have learnt a lot from the E3SM model. I also found many issues within the model, which reflects lots of similar struggles in the lifespan of software engineering.

Here I list a few major ones that we all dislike but they are around in almost every project we have worked on.

Excessive usage of existing framework even it is not meant to Working in a large project means that you should not re-invent the wheels if they are already there. But more often, developers tend to use existing data types and functions even when they were not designed to do so. The reason is simple: it is easier to use existing ones than to create new ones. For example, in E3SM, there was not a data type to transfer data between river and land. Instead, developers use the data type designed for atmosphere and land to do the job. While it is ok to do so, it added unnecessary confusion for future development a…
Recent posts

Publication graphic generation workflow

Preparing graphics for journal publication is an important step before we submit the manuscript. And sometimes this process is not as smooth as we expected. There are several problems we often encounter: We generally generate more than enough figures and in the end, we only need a few;We also use different format for different purposes. For example, we use jpg/png for debugging. And we use svg/postscript for high quality production. A conversion is usually required.Journals usually prefer 600dpi high resolution figures. So postscript format might be the best option.Indexing is important for final upload.Subplot makes it even complicated.We use more than one tools as well. I use IDL/Python for plotting, but I also use GIMP/Snagit/Inkscape for some processes. Keep the process consistent is not easy. So maybe we need a clear road map so we won’t get lost easily. Here are my plans: Produce postscript/svg when possible using DrawIO/Python/IDL;If we need subplot, produce them simultaneously;…

A revisit of spatial discretization

Discretization by definition from Wikipedia: In applied mathematics, discretization is the process of transferring continuous functions, models, variables, and equations into discrete counterparts. This process is usually carried out as a first step toward making them suitable for numerical evaluation and implementation on digital computers. Now we add “spatial” to the term. The first intuitive definition is the discretization of functions in the spatial domain. There are two elements in this description: functions and spatial domain. For functions, we often refer to integral or ODEs/PDEs in numerical simulations. If these functions involve with gradient information, then they depend on spatial domain, which is how gradient is calculated. For spatial domain, we often refer to mesh or grid. And mesh can generally be classified into structured and unstructured grid. In practice, we have spent great effects on both aspects of the spatial discretization: mesh and corresponding function s…

Evaluate the stream networks from watershed delineation

Recently I asked a question on the GIS Exchange site:
https://gis.stackexchange.com/questions/315910/quantitatively-evaluating-quality-of-watershed-delineation-stream-line-results

I also asked the question on Research Gate:
https://www.researchgate.net/post/How_to_quantitatively_evaluate_the_quality_of_the_watershed_delineation_stream_line_results

The reason is that we have developed a watershed delineation model and we need to evaluate whether our model performs better or not than the previous method.

So we set out trying to find ways to evaluate the results.
The first thing as a watershed hydrologist will usually do is to look at the stream segments. If they match up with actual stream lines then it means the model is not bad.

However, both our method and the previous method can produce similar stream segment results. The the question is how can we say which one is better than the other one.

So we did some research online, and most publications used visual results as proof. Basically,…

SWAT-MODFLOW stream flow routing problem

Surface water hydrology and groundwater hydrology are coupled together in natural ecosystem. While it is easy to say so, it is not easy to model both of them simultaneously.
In my earlier posts I have covered a lot on related topics including MODFLOW, PRMS and GSFLOW.
For example:  https://www.changliao.us/2018/03/paper-discussion-002.html https://www.changliao.us/2018/03/paper-discussion-001.html https://www.changliao.us/2015/09/integrated-groundwater-and-surface-water-01.html https://www.changliao.us/2016/04/surface-water-hydrology-modeling-002.html
Today I will discuss some other issues related to SWAT-MODFLOW. SWAT is another widely used surface hydrology model and there are ongoing efforts trying the couple MODFLOW with SWAT. https://swat.tamu.edu/software/swat-modflow/
Unlike PRMS, SWAT in general does NOT use grid based approach to run simulation. Instead, SWAT uses subbasin and HRU to represent the watershed. And this difference may cause a list of challenge for us. I will discus…

Tips for ArcSWAT issue

I have to use ArcSWAT to prepare some SWAT model simulation inputs. And the experience wasn't exactly good (maybe we need a different tool to do this the right way).
Below I listed a few issues I have encountered and potentially fixed:

Failed to create raster dataset. This issue might be related to permission control on Windows. Based on my experience, it is best to create the project under the root directory using the ArcSWAT interface. Do not create the directory outside using other method (WSL mkdir caused this error multiple times on my Windows 10).


Object reference not set. This issue is often related to the above issue. If the DEM was successfully, you should not receive this error.

SWAT check error: SWAT check version may not be compatiable with the latest SWAT version. For example, check this discussion: https://groups.google.com/forum/#!topic/swatuser/aSsSeJVIrvU
I will keep adding related issues until the project is finished.

The hydrology in the real world

Below is a region of interest from Global Lakes and Wetlands Database (link).



My question is how can we represent them in the Earth System Model?
To understand the challenge, we need to know how hydrology features are represented in current modeling work.
In an ideal scenario when there is no lake, stream network are the dominant hydrology features. To date, most existing hydrology simulations are conducted at relatively large spatial domain. As a result, the spatial resolution of such simulation is limited to hundred to thousand of meters. Sometimes it is impossible to capture some features under this resolution.
On the other hand, the land is never homogeneous and lots of local depressions are visible. These local depressions are removed in most hydrological model.
When there are lakes, the situation gets complicated. There are several challenges we will face: A lake may changed its boundary depending on water storage;A lake may or may not have an outlet;A lake itself is a local dep…