saving multiple files with longer time axis for really big file (about 12GB for each)

For saving massive file, “set memory/size=nnn” doesn’t work. Because it’s too big (ref).

Creating axis and saving files orderly also doesn’t work. Because it’s too big (ref).

So, file should be saved in every each single time step.

Then, any massive files can be saved to one super massive big file.

 

Let’s start! (It’s based on real story … )

 

Example, (1993 ~ 1994, 6-hourly, lon x lat x lev, leap year)

 

set mem/size=500

let var = “geo”

define axis/t0=01-jan-1993/t=”01-jan-1993“:”31-dec-1994“:1/npoints=2920/unit=hours/edges/modulo tannual

use wrf_GP_199301.nc  ! CALL 1st DATA
let new_`var`=`var`[d=1,gt=tannual]
save/clo/file=all.nc new_`var`[l=1]
repeat/name=nn/range=2:1460 (save/app/file=all.nc new_`var`[l=`nn`])
cancel data 1

use wrf_GP_199401.nc ! CALL 2nd DATA
let new_`var`=`var`[d=1,gt=tannual]
repeat/name=mm/range=1461:`1461*2` (save/file=all.nc/append new_`var`[l=`mm`])

 

Tips!!: It’s not a necessary to use “calendar option” for defining axis if the data has leap year format. Just using “npoints” will be better.

 

Little, Simple 🙂

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s