Notes on getting RETREAT data from the IRIS DMC (6/13/07)
FIRST:
obtain JWeed from the IRIS website. Install on your computer.
SECOND: obtain ccrypt from sourceforge on the web. There are versions that have
executables for many processors and operating systems, but it might be safest
to get the source code and compile it in your own environment. First "./configure" then
"make" then "sudo make install" on a Mac.
Running JWeed
Three items to remember: 1) the RETREAT network code is YI. There are some other stations in Antarctica that are
included in network YI. You might
be able to avoid these in JWeed by restricting the stations to a box around
Italy. 2) the definition of YI is
time-variable. If you ask for the
station info when JWeed is set for a time interval in 2007, you wont get the
RETREAT stations. 3) the RETREAT and GAIA stations record on the BH? channels,
and the INGV network stations record on the HH? channels. You must request both channel types to
get data from both sets of stations.
Dont try to move back and forth in JWeed while you are
retrieving something from the internet.
In particular, the download of stations from the database is tedious,
and there is temptation to wander in JWeed. However, I have found that JWeed will halt when you move to
as different tab in the GUI. It is
smarter to use the Network DataCenter once for each time interval of interest,
then use the JWeed-written station file for later requests.
To get the INGV HH? stations, Ive wasted a day trying. HH? channel is not a channel normally found by JWeed. Also the IRIS database cant always find
the INGV stations in JWeed. It is
a pisser. The key is that JWeed
wont find a station that does not have channel that is being requested. So if you ask for BH? channels, only
the PASSCAL and GAIA stations appear in YI, not the INGV stations. I find that the request works if you
ask for BOTH B* and HH* channels.
The other JWeed "feature" is that you cant
create multiple breqfast files without exiting the software. Either the YI network decreases down to
a single station on the Antarctic ice cap, the breqfast file hangs, or
something else surprises you. Give
up! Exit JWeed and start
over!
If you have event files and station files, it is
possible to run JWeed without accessing the outside DHI servers. However, I had to turn off the internet
access for the computer to avoid problems. Also, you cant run JWeed multiple times to create multiple
files. Ive had to close JWeed and
start over every time.
Here is what seems to work:
1) set IRIS netDC for Jweed in first tab.
2) Set the whole world in the second tab -- it is the
default box. You can also specify
an epicentral range around a map point.
Use the crosshairs icon to the left
3) in third tab, specify magnitude range, depth range,
MB MS ML MW as the magnitudes to use, load the catalogs if they are not there
already, choose WHDF (maybe) and specify the time period for gathering events
(VERY important!)
4) Save events if you like
5) Map the events, an operation that pops open the
second tab again -- in the map specify a small rectangle over Italy for
choosing stations
6) in the 4th tab, access the DHI to bring down a list
of networks and stations from the server.
This can take a long time.
When "finished" open the YI folder to verify that the stations
desired are there. If only the Antarctic station is there, search the DHI
again. If only BH? stations are
there and not the INGV stations, start over if you want the INGV stations.
7) Highlight the YI folder and map the stations. save them to a file if you wish.
8) Go to the generate-request tab. click the "listing" button,
rather than the "map" button.
wait until the summary file appears in the text window. The program prompts for a file name and
write out the summary file automatically.
9) Go to the mail-request tab. Fill in the info, choose the Breqfast-IRIS tab to specify
the IRIS DMC. Do NOT email the
request directly, save it to a file.
This allows you to fix any syntax issues easily.
Running CCRYPT
the routine ccrypt has better encryption protection than
the old Unix version of crypt.
This is great, but IRIS encodes with the old version. Therefore you must use the
"u" option:
ccrypt
-u < file.seed.crypted > file.seed
and the routine prompts you for the crypting key. I put alias crypt "ccrypt -u"
into my .cshrc file
Using Breq_fast
JWeed will generate a breq_fast request. It is smart to save the breqfast file
on your home computer in order to fix syntax problems. Some syntax issues crop
up intermittently. For instance,
JWeed will let you specify multiple channels in the request
e.g. BHZ
HHZ, but in the last few days the two items in the breqfast file are not parsed
by the DMC software, only the first one is. Parsing the multiple-channel request by changing the
breqfast file to "BHZ HHZ" with quote marks WONT WORK. For the YI case you can try to specify
?H? as the channel and the request will obtain both HH? and BH?. BUT, watch the last number before the
channel identifier. It should be
the number 1, else there will be a syntax error. However, the ?H? trick works in some cases and not others. Therefore I have requested BH? and HH?
separately. One can use the same
breqfast file twice, since no station has both BH and HH channels.
A number of tests failed to work. I hypothesize that Breqfast wildcarding
wont allow '?', only '*' In any
case, the last time I sent a breakfast request with the '*' wiildcard and two
channel items, the DMC responded as documented. AAAAAUUUUUGGGGHHH!!!!!
Running rdseed on the SEED files
This seems to be OK. The use of summary files is important, because these will
synchronize start times for the three components of motion, if possible. RDSEED is not transferring event informationj
from the summary file, so there are no epicentral distances and azimuth
values (or correct ones!) in the
SAC header. It is possible to run
rdseed in commandline mode:
rdseed
-d -f filename.seed -x filename.summary
Some Comments on Using JWeedv2.9.7 (10/12/05)
The Java-based JWeed works reasonably well,
but there are maddening quirks.
You must download WEED-format station files from someplace. Downloading
them from within JWeed takes a long time, so it is wise to download them once
and store station files from your favorite networks locally. Downloading
station files (and event files) via FTP from the IRIS website has appeal as
well. However, not all data centers have all stations of a network in
their definition files. MEDNET stations are missing from some databases,
for instance. JWeed offers a button to look for local station
files. The same holds true for the event files. It sometimes is
more convenient to create local event files to speed up the data request
process. NEIC is a good bulletin to use. Some of the other
bulletins miss some M.ge.6 events. Note that sometimes events can be
doubled in the files, leading to redundant data requests if not pruned.
With JWeed you must define events first, then
stations, then make a summary file. The stations and events must be
respecified before creating each summary file (at least this seemed to be
necessary in my experience). It is very easy to read an event file that
doesnt overlap the time window that you specifiy for your request. That
is, mismatched time windows are a common mistake when trying to create a
summary file.
From the Summary file JWEED creates data
requests. These come in two flavors, breq_fast and netdc. breq_fast
requests can be addressed to only one data archive e.g. the IRIS DMC.
Netdc requests are addressed to a meta-archive of data providers. The
netdc server parses the data request and distributes it to differnt data
centers according to an unknown protocol. This protocol has bugs.
For instance my Netdc request for MEDNET data went to the ingv archive and
returned an incompleted dataset. A full MEDNET data set was available
instead from the IRIS DMC, which I interrogated with a separate Breq_fast
request.
The choice of whether JWeed writes a netdc or
breq_fast file is not obvious. It involves the toggle above the email
choice button. You can write either request format, but you probably cant
send the file automatically. It requires cutting and pasting the file
contents into a mail window and sending.
If the DMS sends you a crypted file, it can
be decrypted only on a Sun Solaris computer. I have tried Linux and MacOS
X without success, because they do not have a Unix line command crypt. If
the decryption is to be performed on rock.geology.yale.edu, be forewarned that
rock can be entered via ssh or sftp only from computers within the geology.yale.edu
domain. This can lead to a lot of ftp transfers. The bright side is
that the Sun is no longer needed to unpack SEED files according to data windows
defined in summary files. Rdseed 4.6 seems to do this OK, but my first
usage of the software seems to have time-shifted VLC data by 15 sec.
IF THE SUN IS USED
request data with JWEED and transfer crypted SEED files to the Sun (rock) in
directory /home/park/Data
decrypt SEEDfiles with secret RETREAT password according to DMS instructions:
crypt
password < file.crypted > file.seed
ftp the decrypted file to your home computer,
and use rdseedv4.6 to extract data.
once you have the SAC files, we can rotate
and process in MacOS X
FOR SMALL TIME TEARS THAT FRAGMENT THE DATA FILES
process the binary SAC files on the
DELL with the following merge procedure. (we will see about the Mac)
ls
*SAC > infiles_raw
files_merge
cp outfile sac_merge
cp outfile2 infiles1
in another window:
merge_stupid
outpute files of files_merge are
outfile -- a SAC macro for assessing time
tears for individual events, copy to sac_merge
outfile2 -- a list of files that the macro thinks will be the final set
of merged files
you can copy outfile2 to infiles1 for later processing, but events that occur
within 1200 s of each other will lead to extra files that must be added my
hand.
when you run sac_merge within SAC, the macro
pauses at plots of data files to me merged.
SAC is finicky about merging files, so I wrote a FORTRAN program that
merges files
during the SAC pause, run merge_stupid in another window. This program will automatically
merge files
whose start and end differ by 0.01s or less, that is,
the time tear does not lead to a gap ot an overlap in BH-channel data.
For files with an overlap or a gap, the program asks if it is all right to
merge the files. Gaps are filled with zeroes. Overlaps are written
over by the later file.