{ "cells": [ { "cell_type": "markdown", "id": "4198e5ab", "metadata": {}, "source": [ "# **Subset CONUS and do a ParFlow Spinup**\n", "\n", "To launch this notebook interactively in a Jupyter notebook-like browser interface, please click the \"Launch Binder\" button below. Note that Binder may take several minutes to launch.\n", "\n", "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/hydroframe/subsettools-binder/HEAD?labpath=subsettools%2Fconus1_subsetting_spinup.ipynb)\n", "\n", "This notebook will subset a HUC8 from the CONUS1 domain as in [this](https://hydroframesubsettools.readthedocs.io/en/latest/example_notebooks/conus1_subsetting_transient.html) example. However, it does not subset climate forcing data as it is an example of model initialization or spin up. So, it only runs with ParFlow. The template runscript used is conus1_pf_spinup_solid.yaml which has the necessary timing keys to run with a growth timestep and a longterm recharge mask (PmE) as opposed to a constant time step and time varying climate forcing data. The example notebook is not set to run for the entire period it would normally take to spin-up a model, you are encouraged to do your own evaluation to determine if your model is at steady state. \n", "\n", "### This notebook has two principal sections: \n", "1. Subset all static inputs from a CONUS1 run stored in Hydrodata \n", "2. Load and alter a reference run to spin-up your subset ParFlow domain" ] }, { "cell_type": "markdown", "id": "1f0f4110", "metadata": {}, "source": [ "### Import the required libraries" ] }, { "cell_type": "code", "execution_count": 1, "id": "0c960423", "metadata": { "tags": [] }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import os\n", "from parflow import Run\n", "from parflow.tools.fs import mkdir\n", "from parflow.tools.settings import set_working_directory\n", "import subsettools as st\n", "import hf_hydrodata as hf" ] }, { "cell_type": "code", "execution_count": 2, "id": "b2285632-446a-47b4-8690-aa6ea6db1f06", "metadata": {}, "outputs": [], "source": [ "# You need to register on https://hydrogen.princeton.edu/pin before you can use the hydrodata utilities\n", "hf.register_api_pin(\"your_email\", \"your_pin\")" ] }, { "cell_type": "markdown", "id": "dd0f9348", "metadata": { "tags": [] }, "source": [ "### 1. Define variables to access datasets in Hydrodata to subset and define write paths" ] }, { "cell_type": "markdown", "id": "ed93b3b2", "metadata": {}, "source": [ "#### We will be testing with the Upper Verde watershed for this example\n", "- HUC: 15060202\n", "- Size: 6496 km^2 (ni = 112, nj = 90) " ] }, { "cell_type": "markdown", "id": "808c9620", "metadata": {}, "source": [ "#### Set your variables to specify which static data you would like to subset in Hydrodata" ] }, { "cell_type": "code", "execution_count": 3, "id": "dfbdea78", "metadata": { "tags": [] }, "outputs": [], "source": [ "runname = \"spinup_test\"\n", "\n", "# provide a way to create a subset from the conus domain (huc list, lat/lon bbox currently supported)\n", "hucs = [\"15060202\"]\n", "\n", "# provide information about the datasets you want to access for run inputs using the data catalog\n", "start = \"2005-10-01\"\n", "wy = 2006\n", "grid = \"conus1\"\n", "run_ds = \"conus1_baseline_mod\"\n", "var_ds = \"conus1_domain\"\n", "P = 1\n", "Q = 1\n", "\n", "# set the directory paths where you want to write your subset files\n", "home = os.path.expanduser(\"~\")\n", "base_dir = os.path.join(home, \"subsettools_tutorial\")\n", "input_dir = os.path.join(base_dir, \"inputs\", f\"{runname}_{grid}_{wy}WY_spinup\")\n", "output_dir = os.path.join(base_dir, \"outputs\")\n", "static_write_dir = os.path.join(input_dir, \"static\")\n", "mkdir(static_write_dir)\n", "pf_out_dir = os.path.join(output_dir, f\"{runname}_{grid}_{wy}WY_spinup\")\n", "mkdir(pf_out_dir)\n", "\n", "# Set the PARFLOW_DIR path to your local installation of ParFlow.\n", "# This is only necessary if this environment variable is not already set.\n", "# os.environ[\"PARFLOW_DIR\"] = \"/path/to/your/parflow/installation\"\n", "\n", "# load your preferred template runscript\n", "reference_run = st.get_template_runscript(grid, \"spinup\", \"solid\", pf_out_dir)" ] }, { "cell_type": "markdown", "id": "aa7ab970", "metadata": {}, "source": [ "### 2. Get the desired ParFlow i/j bbox from user provided geospatial information " ] }, { "cell_type": "code", "execution_count": 5, "id": "9d50bbca", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ij_bounds returns [imin, jmin, imax, jmax]\n", "bounding box: (375, 239, 487, 329)\n", "nj: 90\n", "ni: 112\n" ] } ], "source": [ "ij_bounds, mask = st.define_huc_domain(hucs=hucs, grid=grid)\n", "print(\"ij_bounds returns [imin, jmin, imax, jmax]\")\n", "print(f\"bounding box: {ij_bounds}\")\n", "\n", "nj = ij_bounds[3] - ij_bounds[1]\n", "ni = ij_bounds[2] - ij_bounds[0]\n", "print(f\"nj: {nj}\")\n", "print(f\"ni: {ni}\")" ] }, { "cell_type": "markdown", "id": "c0362322", "metadata": {}, "source": [ "### 3. Make the mask and solid file\n", "You only do this if you providing a huc or list of hucs. Otherwise, the reference run provided is for a box domain." ] }, { "cell_type": "code", "execution_count": 6, "id": "9c3aec38", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Wrote mask.pfb\n", "Wrote solidfile and mask_vtk with total z of 500 meters\n" ] } ], "source": [ "mask_solid_paths = st.write_mask_solid(mask=mask, grid=grid, write_dir=static_write_dir)" ] }, { "cell_type": "markdown", "id": "1567e0ff", "metadata": {}, "source": [ "### 4. Subset the static ParFlow inputs\n", "Two options to subset static inputs. \n", "1. subset_static(): This function when provided with a variable dataset hosted on hydrodata will subset all static inputs required to do a baseline run. Note that the function will raise an error if any of the requested variables do not exist in the dataset, so we need to modify the default variable list and remove \"mannings\" and \"flowbarrier\". Pressure is the steady state pressure.\n", "\n", "3. subset_press_init(): This function will write the subset pressure of the last hour in the last day before your start date in the provided timezone. If no such pressure file exists in the hydrodata run dataset specifed, no file will be written. The function assumes UTC of zero as the default, but allows the user to override. " ] }, { "cell_type": "code", "execution_count": 7, "id": "ec16ed87", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Wrote slope_x.pfb in specified directory.\n", "Wrote slope_y.pfb in specified directory.\n", "Wrote pf_indicator.pfb in specified directory.\n", "Wrote pme.pfb in specified directory.\n", "Wrote ss_pressure_head.pfb in specified directory.\n" ] } ], "source": [ "static_paths = st.subset_static(ij_bounds, dataset=var_ds, write_dir=static_write_dir,\n", " var_list=(\"slope_x\", \"slope_y\", \"pf_indicator\", \"pme\", \"ss_pressure_head\",)\n", " )" ] }, { "cell_type": "code", "execution_count": 8, "id": "33d3e618", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "UTC Date: 2005-10-01 00:00:00\n", "Wrote /home/ga6/subsettools_tutorial/inputs/spinup_test_conus1_2006WY_spinup/static/conus1_baseline_mod_2005.10.01:00.00.00_UTC0_press.pfb in specified directory.\n" ] } ], "source": [ "press_init_filepath = st.subset_press_init(\n", " ij_bounds, dataset=run_ds, date=start, write_dir=static_write_dir, time_zone=\"UTC\"\n", ")" ] }, { "cell_type": "markdown", "id": "99b69c80", "metadata": {}, "source": [ "### 5. Set up a baseline run from a reference yaml\n", "This function will return the correct template yaml file to do your run based on the grid, if you're doing spin-up and if you're using a solid file with the necessary keys changed to spin up your subset" ] }, { "cell_type": "code", "execution_count": 9, "id": "ea1f0e8e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "New runname: spinup_test provided, a new yaml file will be created\n", "No forcing directory provided, run.Solver.CLM.MetFilePath key not set\n", "ComputationalGrid.NY set to 90 and NX to 112\n", "GeomInput.domaininput.InputType detected as SolidFile, no additional keys to change for subset\n", "Updated runscript written to /home/ga6/subsettools_tutorial/outputs/spinup_test_conus1_2006WY_spinup\n" ] } ], "source": [ "runscript_path = st.edit_runscript_for_subset(\n", " ij_bounds, runscript_path=reference_run, runname=runname\n", ")" ] }, { "cell_type": "markdown", "id": "9f742051", "metadata": {}, "source": [ "### 6. Copy over your static files to your run directory\n", "You may only need to do this once, or you may want to copy subset static files to different run directories " ] }, { "cell_type": "code", "execution_count": 10, "id": "71ae6dd2", "metadata": {}, "outputs": [], "source": [ "st.copy_files(read_dir=static_write_dir, write_dir=pf_out_dir)" ] }, { "cell_type": "markdown", "id": "2c1fe950", "metadata": {}, "source": [ "### 7. Change the file names in your runscript if desired\n", "If you have changed the name of a static input file either from those used in the reference yamls provided, or have changed the name of an individual file for an ensemble or other experiment, you can change it with this function by providing the target runscript (yaml or pfidb) and the new file name(s) as an arguments. Only those arguments with a specified file name will be updated" ] }, { "cell_type": "code", "execution_count": 11, "id": "e0288e27", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Initial pressure filename changed to conus1_baseline_mod_2005.10.01:00.00.00_UTC0_press.pfb\n", "Updated runscript written to /home/ga6/subsettools_tutorial/outputs/spinup_test_conus1_2006WY_spinup\n" ] } ], "source": [ "init_press = os.path.basename(press_init_filepath)\n", "runscript_path = st.change_filename_values(\n", " runscript_path=runscript_path,\n", " init_press=init_press,\n", ")" ] }, { "cell_type": "markdown", "id": "b14cf608", "metadata": {}, "source": [ "### 8. Change processor topolgoy if desired and then distribute your inputs and forcings to match the new topology" ] }, { "cell_type": "code", "execution_count": 12, "id": "24c155b6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "No forcing directory provided, only distributing static inputs\n", "Distributed mask.pfb with NZ 1\n", "Distributed slope_x.pfb with NZ 1\n", "Distributed slope_y.pfb with NZ 1\n", "Distributed pf_indicator.pfb with NZ 5\n", "Distributed pme.pfb with NZ 5\n", "Distributed ss_pressure_head.pfb with NZ 5\n", "Distributed conus1_baseline_mod_2005.10.01:00.00.00_UTC0_press.pfb with NZ 5\n" ] } ], "source": [ "runscript_path = st.dist_run(\n", " topo_p=P,\n", " topo_q=Q,\n", " runscript_path=runscript_path,\n", " dist_clim_forcing=False,\n", " )" ] }, { "cell_type": "markdown", "id": "9eb7c01e", "metadata": {}, "source": [ "### 9. Do a baseline run.\n", "Load in the yaml run file you've created which is in the same folder as your static forcings. This assumes you do not want to make any changes from the parent model you used (Ex. conus1 baseline) and will run your subset at baseline conditions. Outputs should be almost identical to the parent model at your subset location for the same time period if you make no additional changes." ] }, { "cell_type": "code", "execution_count": 13, "id": "abbc70fa", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loaded run with runname: spinup_test\n", "\n", "# ==============================================================================\n", "# ParFlow directory\n", "# - /home/SHARED/software/parflow/3.10.0\n", "# ParFlow version\n", "# - 3.10.0\n", "# Working directory\n", "# - /home/ga6/subsettools_tutorial/outputs/spinup_test_conus1_2006WY_spinup\n", "# ParFlow database\n", "# - spinup_test.pfidb\n", "# ==============================================================================\n", "\n", "\n", "# ==============================================================================\n", "# ParFlow ran successfully 💦 💦 💦 \n", "# ==============================================================================\n", "\n" ] } ], "source": [ "set_working_directory(pf_out_dir)\n", "\n", "# load the specified run script\n", "run = Run.from_definition(runscript_path)\n", "print(f\"Loaded run with runname: {run.get_name()}\")\n", "\n", "# The following line shows timing info that is likely important for interacting with your spin-up run\n", "\n", "# run.TimingInfo.BaseUnit = 1.0\n", "# run.TimingInfo.StartCount = 0\n", "# run.TimingInfo.StartTime = 0\n", "# normally set to something like 1000000.0 but shorter for demo purposes\n", "run.TimingInfo.StopTime = 48.0\n", "run.TimingInfo.DumpInterval = 24.0\n", "\n", "# run.TimeStep.Type = 'Growth'\n", "# run.TimeStep.InitialStep = .01\n", "# run.TimeStep.GrowthFactor = 1.1\n", "# run.TimeStep.MaxStep = 250.0\n", "# run.TimeStep.MinStep = 0.0001\n", "\n", "run.run(working_directory=pf_out_dir)" ] }, { "cell_type": "code", "execution_count": null, "id": "8651ed8d-0153-41da-a214-ff9e1f8cbc8c", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.17" } }, "nbformat": 4, "nbformat_minor": 5 }