All, I am pleased to announce the release of "divgdr", a tool used to read in Pierre's fantastic Level 2 and 3 Gridded Data Records (GDRs). Like "divdata", "divgdr" takes a list of arguments on the command line and delivers a stream of data to standard output in binary "pipes" format, suitable for piping into other commands. To see the full manual for divgdr, simply type "divgdr" with no arguments. Level 2 Fields - Dated, monthly cycles: divgdr supports the monthly-cycled Level 2 fields. These are things like vb1 (Visual Brightness Channel 1), vb2, tb3, ..., tbol (Bolometric Temperature, semantically Level 3, but compatible with the other Level 2 fields), ltim (Local Time), and jd (Julian Date). All have the suffix of _avg (average), _cnt (count), or _err (error). Examples include vb1_avg, tb3_err, tb3_cnt, jd_avg, tbol_avg. One specifies a range of date=BEGIN[,END] arguments similarly to the use of the divdata program. Level 3 Fields - Hourly increments: Level 3 fields include ra (Rock Abundance), st (Soil Temperature), rms (Root Mean Square fitting errors), std_cf (Standard Christiansen Feature), and nen_cf (Normalized to Equatorial Noon CF). These fields have the suffix of _clc (calculated), e.g. st_clc, nen_cf_clc. All but the *_cf_* fields use the hour=BEGIN[,END] argument to constrain time. To see a list of supported fields, type "divgdr fields". To see a list of dates for which we have data, type "divgdr dates". To see a list of hours for which we have data, type "divgdr hours". Let's have some examples. * Extract the average TB3 and local time for a given mapping cycle, in 128 pixels per degree resolution. Return the ranges of each field using pgetranges: divgdr debug=2 res=128 date=20091230 dn=D extract=tb3_avg,ltim_avg | \ pgetranges > ranges.txt (Note: The \ character is a line-continuation character, used to split long commands into multiple lines. Not needed if you construct your pipes on one big line) * Extract the same fields at 4 PPD and print the records out as text using pprint: divgdr debug=2 res=4 date=20091230 dn=D extract=tb3_avg,ltim_avg | \ pprint > output.txt It is not recommended that you print records to text at higher resolutions, very much. At 128 PPD this produced a 90 GB file. If you do this, please recycle your disk space regularly. * In the above examples, you can view a range of dates, using either months or specific days, e.g.: date=201001,201012 date=20090705,20091011 The dn=D indicates you want Daytime cycles (as opposed to Nighttime). If you don't specify, you get both. * If you want to constrain certain fields, do so on the command line: divgdr debug=2 res=128 date=20091230 dn=D extract=tb3_avg,ltim_avg \ tb3_avg=0,200 ltim_avg=9,15 | \ pgetranges > ranges.txt * Cells with no data for a given field have the value -9999. You can constrain these out using, e.g.: tb3_avg=-9998,999999 Bear in mind that some cells have data for one field but not for another. Constraining both of them this way may eliminate useful data, as constraining uses -AND- logic. * You can also run "divgdr" on your own locally created IMGs, like so: divgdr debug=2 res=1 extract=jd_avg img=DGDR_JD_AVG_CYL_20100717D_001_JP2.IMG | pgetranges > ranges An IMG must have an associate PDS label, which can be specified with the lbl= command, or the program will attempt to find it in the same directory. IMGs must be of the _JP2.IMG variety, which use longitudes -180,180. Ask Pierre or Mark for more details. That's enough examples. I've copied the current manual page below this line, but it is subject to change, so always use "divgdr" with no arguments to see the current manual. I've updated the Diviner UCLA Account FAQ and fixed a bunch of information: http://luna1.diviner.ucla.edu/~marks/account_faq.html Have fun! -Mark Sullivan ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ Quick info: divgdr (No arguments, prints this usage statement) divgdr fields (to just print out selectable fields) divgdr dates (to see which dates we have data for) ---- How to pipe data into other pipes commands: divgdr [type=datatype] [noindex] [res=N] ([date=BEGIN,END] *OR* [hour=BEGIN,END]) ([dn=D] *OR* [dn=N]) [FIELD=MIN,MAX FIELD=MIN,MAX ...] [img=IMGFILE1,IMGFILE2,...] ---> Only if you want to specify your own local files. [lbl=LBLFILE1,LBLFILE2,...] ---> Only if you want to specify your own local files. extract=field1,field2,... | PIPES_COMMANDS ... Description of switches: * Specify either date= (with possibly dn=), or hour= date=BEGIN,END - For dated records. BEGIN and END for date can be the following format: YYYYMM - A month, gets you all the days in that month. YYYYMMDD - A date. If BEGIN=END, e.g. 200907, you can just use: date=200907 Multiple date=BEGIN,END arguments specify disjoint times. dn=? - For dated records, specify D (day) or N (night). Default is both. hour=BEGIN,END - For hourly records such as RA, ST, etc. In this case use the format HHMM. Hourly records tend to run from local time 1930 to 0430. Other switches: extract=field1,field2,... - The fields you want to extract from the data. FIELD=MIN,MAX - Constrain any FIELD in the dataset to only include values between MIN and MAX, inclusive. When specifying a MIN,MAX, if MIN<=MAX, you get all the data between MIN and MAX, inclusively. If MIN>MAX, you get all the data OUTSIDE of the inclusive MIN,MAX range. type=DATATYPE - Output data format. Default is 'div38'. res=N - Which resolution to use, in pixels per degree. 128, 64, 32, 16, 4, or 1. img=IMGFILE1,IMGFILE2,... - Use your own local IMG files instead of the database. Note: You still must specify res=N, and extract=field1,field2,... in order to name the fields that are coming down the pipe. Each IMGFILE must have a corresponding .LBL file. lbl=LBLFILE1,LBLFILE2,... - If you specify img=, you may also specify the corresponding label files using lbl=. If not, the program will try to find the label in the same directory. Use debug=1 or debug=2 to get more info about labels selected. noindex - Do not use indexing to match data constraints. Significantly SLOWS DOWN your data access. Use only for debugging, a sanity check to make sure you are getting all your data. Using this option *should* not alter your results except in terms of speed. Let us know otherwise. nodel - Do not delete the catfile this program creates. debug=N - Debug level where N is one of: 0 - Normal, only high level messages. 1 - Detailed 2 - Extra detailed. All debugging messages are printed to standard error.