Sea Surface Height
Contents
Sea Surface Height#
%load_ext autoreload
%autoreload 2
import os
import numpy as np
import xarray as xr
import pandas as pd
from glob import glob
import intake
import dask
from dask.distributed import Client
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import cartopy
import cartopy.crs as ccrs
import data_access
Reading in parameters#
### this cell will get parametrized
path_to_cat = "placeholder"
subset_kwargs = {}
### this is here so it can be run alone
#path_to_cat = "temp_data/ssh_cat_subset.json"
# Parameters
casename = "gcp-cases"
path_to_cat = "/glade/u/home/eromashkova/codes/my-cesm-experiment-extended/notebooks/temp_data/gcp-cases_subset.json"
cluster_scheduler_address = "tcp://10.12.206.47:39726"
subset_kwargs = {"frequency": "month_1", "component": "ocn", "variable": "SSH"}
Connecting to cluster#
if cluster_scheduler_address is None:
cluster, client = util.get_ClusterClient()
cluster.scale(12)
else:
client = Client(cluster_scheduler_address)
client
Client
Client-b0ee0f23-a1a9-11ed-9374-3cecef1aca46
| Connection method: Direct | |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/8787/status |
Scheduler Info
Scheduler
Scheduler-7cfddfff-723a-4eb5-8b97-437cfe055d0a
| Comm: tcp://10.12.206.47:39726 | Workers: 14 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/8787/status | Total threads: 14 |
| Started: Just now | Total memory: 325.92 GiB |
Workers
Worker: PBSCluster-0
| Comm: tcp://10.12.205.23:42696 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/42447/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.205.23:45370 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-ldwiof8h | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 163.02 MiB | Spilled bytes: 0 B |
| Read bytes: 6.90 kiB | Write bytes: 1.29 kiB |
Worker: PBSCluster-10
| Comm: tcp://10.12.206.54:46661 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/38540/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.54:36281 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-kdng5ht7 | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 173.57 MiB | Spilled bytes: 0 B |
| Read bytes: 0.96 GiB | Write bytes: 2.20 MiB |
Worker: PBSCluster-11
| Comm: tcp://10.12.206.56:32952 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/43547/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.56:39886 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-s_h6jgot | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 152.51 MiB | Spilled bytes: 0 B |
| Read bytes: 0.96 GiB | Write bytes: 516.65 kiB |
Worker: PBSCluster-12
| Comm: tcp://10.12.206.56:35153 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/33514/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.56:44855 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-wxbnwdzh | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 0.0% | Last seen: Just now |
| Memory usage: 168.13 MiB | Spilled bytes: 0 B |
| Read bytes: 0.96 GiB | Write bytes: 518.61 kiB |
Worker: PBSCluster-14
| Comm: tcp://10.12.205.23:35738 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/37595/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.205.23:32970 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-hd8il_3s | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 0.0% | Last seen: Just now |
| Memory usage: 163.62 MiB | Spilled bytes: 0 B |
| Read bytes: 6.89 kiB | Write bytes: 3.93 kiB |
Worker: PBSCluster-15
| Comm: tcp://10.12.206.54:41090 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/34349/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.54:37296 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-6m7n2cvt | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 151.96 MiB | Spilled bytes: 0 B |
| Read bytes: 0.96 GiB | Write bytes: 2.20 MiB |
Worker: PBSCluster-16
| Comm: tcp://10.12.206.60:42728 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/38728/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.60:35710 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-vunonffe | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 4.0% | Last seen: Just now |
| Memory usage: 151.09 MiB | Spilled bytes: 0 B |
| Read bytes: 81.20 kiB | Write bytes: 12.62 kiB |
Worker: PBSCluster-23
| Comm: tcp://10.12.205.23:39804 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/42286/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.205.23:41617 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-tg14tvbg | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 167.61 MiB | Spilled bytes: 0 B |
| Read bytes: 6.90 kiB | Write bytes: 5.26 kiB |
Worker: PBSCluster-24
| Comm: tcp://10.12.205.23:46599 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/43644/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.205.23:36001 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-ay6s71lw | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 166.08 MiB | Spilled bytes: 0 B |
| Read bytes: 6.90 kiB | Write bytes: 1.29 kiB |
Worker: PBSCluster-25
| Comm: tcp://10.12.206.51:44308 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/41280/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.51:44650 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-r68y8gm5 | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 4.0% | Last seen: Just now |
| Memory usage: 170.41 MiB | Spilled bytes: 0 B |
| Read bytes: 168.81 MiB | Write bytes: 474.89 kiB |
Worker: PBSCluster-29
| Comm: tcp://10.12.206.51:39955 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/41499/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.51:33751 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-adtwuntq | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 167.04 MiB | Spilled bytes: 0 B |
| Read bytes: 168.74 MiB | Write bytes: 476.01 kiB |
Worker: PBSCluster-3
| Comm: tcp://10.12.206.60:44687 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/39923/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.60:33900 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-7z9f9t6k | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 151.05 MiB | Spilled bytes: 0 B |
| Read bytes: 80.91 kiB | Write bytes: 9.83 kiB |
Worker: PBSCluster-31
| Comm: tcp://10.12.206.60:35661 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/45124/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.60:39472 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-fbi8k1sf | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 151.82 MiB | Spilled bytes: 0 B |
| Read bytes: 80.81 kiB | Write bytes: 9.82 kiB |
Worker: PBSCluster-9
| Comm: tcp://10.12.206.54:42439 | Total threads: 1 |
| Dashboard: https://jupyterhub.hpc.ucar.edu/stable/user/eromashkova/proxy/39444/status | Memory: 23.28 GiB |
| Nanny: tcp://10.12.206.54:38421 | |
| Local directory: /glade/scratch/eromashkova/dask-workers/dask-worker-space/worker-wug0p2lu | |
| Tasks executing: | Tasks in memory: |
| Tasks ready: | Tasks in flight: |
| CPU usage: 2.0% | Last seen: Just now |
| Memory usage: 149.64 MiB | Spilled bytes: 0 B |
| Read bytes: 0.96 GiB | Write bytes: 2.20 MiB |
Reading in data with subsetting#
dset_dict = (intake.open_esm_datastore(path_to_cat).search(**subset_kwargs)).to_dataset_dict()
--> The keys in the returned dictionary of datasets are constructed as follows:
'component.stream.case'
100.00% [1/1 00:05<00:00]
Plotting data#
for i in dset_dict.keys():
ds = dset_dict[i].isel(time=0)
fig = plt.figure()
ds.SSH.plot()
