saving multiple files with longer time axis for really big file (about 12GB for each)

For saving massive file, “set memory/size=nnn” doesn’t work. Because it’s too big (ref).

Creating axis and saving files orderly also doesn’t work. Because it’s too big (ref).

So, file should be saved in every each single time step.

Then, any massive files can be saved to one super massive big file.


Let’s start! (It’s based on real story … )


Example, (1993 ~ 1994, 6-hourly, lon x lat x lev, leap year)


set mem/size=500

let var = “geo”

define axis/t0=01-jan-1993/t=”01-jan-1993“:”31-dec-1994“:1/npoints=2920/unit=hours/edges/modulo tannual

use  ! CALL 1st DATA
let new_`var`=`var`[d=1,gt=tannual]
save/clo/ new_`var`[l=1]
repeat/name=nn/range=2:1460 (save/app/ new_`var`[l=`nn`])
cancel data 1

use ! CALL 2nd DATA
let new_`var`=`var`[d=1,gt=tannual]
repeat/name=mm/range=1461:`1461*2` (save/ new_`var`[l=`mm`])


Tips!!: It’s not a necessary to use “calendar option” for defining axis if the data has leap year format. Just using “npoints” will be better.


Little, Simple 🙂


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.