Category: Note

Astronomy · Note

Notes on PdBI/IRAM data reduction – I. Get Data Ready

Posted on by starwave

This is a personal note about PdBI/IRAM spectral data reduction. The steps will start from transferring the raw data to the local hard-drive, organising raw data, calibration with \tt{CLIC/GILDAS} to the further data reduction with \tt{MAPPING/GILDAS} procedure. The final stage will be mapping and acquire the spectra. If you have any questions or comments, please feeling free to contact me.

PART I: Get Data Ready (Get proj, etc.)

Now, let’s take a high-z observation project for example. Assuming we have observed a molecular line of a high-z galaxy in 2012. So our project name (starting from 2014, a new proposal system called PMS took over the old one, the project naming system has also been changed) would be:

W0B4

W is 2012, V is for 2011 and X is for 2013, etc. I guess there’s a loop of the letters starts from a certain date; 0 is for the target index in this proposal, it could be WAB4, WBB4 and WCB4, …; B4 is the project index, 16th, hexadecimal.
And also we have an observation project name containing the data information, and it could be:

XC12

Just as the proposed project code X represents the year “2013”, C stands for December, the hexadecimal form of the month; 12: the date 12th.
Thus the raw data file of one observation will be like this:

XC12WAB4.IPB, XC12WAB4.OBS, XC12WAB4.RDI

for the source, WAB4 observed on Dec 12th, 2013. IPB file contains the data scan by scan, and OBS & RDI files store the observation parameters.

Now we need to download the data from IRAM data centre to the computers that we perform data reduction (usually at IRAM-HQ) by using:

getproj -p w0b4 -s wab4

The data (subproject wab4 from project w0b4) will be downloaded to the “~/tmp/DATA” directory (actually, it’s a link to the “/scratch/DATA” directory).
You will need to create several directories for data calibration. Usually, they will be generated automatically after getting the project, the calibration directory is under ~/calib, such as

$HOME/calib/w0b4/wab4
$HOME/calib/w0b4/wbb4
...

And also there is a ~/reports directory containing some very useful notes from DoA for performing better calibration, and a ~/map for storing the results from your further data analysis.
After all, these get ready, you can now move on to the data calibration.

Astronomy · computer · Linux · Note

利用BASH脚本运行IDL程序、创建子目录或删除文件

Posted on by starwave

有时候你想在一个目录的所有子目录下批量建立文件夹，并且批量的运行这些子目录内新建立好的文件夹内的 IDL 程序，以下是一种可能的解决途径，如果你有更好的方法，欢迎告诉我

Here are the things I want to do:

  1.  I have a data folder containing many sources, like Arp220, NGC981, etc.. Each of them possesses a folder under the whole data folder (./source/Arp220, ./source/NGC891, …). I want to create a subfolder named “plot” under each sources folder, e.g., ./source/Arp220/plot.
  2. And then I want to copy the IDL routine to each “plot” folder I just created and run all the PRO routines at one time.

Here is a bash script being able to do such things:

#!/bin/bash
for i in $(ls -d */); do mkdir -p $i/'plot'; done
for i in $(ls -d */); do cp ./r_name.pro $i/'plot'/; done
for i in $(ls -d */); do cd $PWD/$i/'plot';idl -e ".r ./r_name.pro"; cd ../.. ; done

The first line means creating a “plot” folder under all the subfolders under you present folder. And the next line is to copy the IDL routine (r_name.pro) to each “plot” folder you created. The last line runs all the IDL routines.

顺便再提一个操作，即“批量删除某个目录下所有子目录中包含”X”关键词的文件”。这个可以用一句话可以解决：

find ./* -type f|grep X|while read file; do rm "$file"; done

比如你要删除“./you suck/arXiv.pdf“这种文件。上面那句话中 -type f|grep X|while read file 是为了保证有空格的目录也可以被写入到一个变量里面。

在一些大神的提示下，我学到了更简单的做法，比如：

find ./* -type f -name "*X*" -exec rm {} \;
Astronomy · computer · Internet · Linux · Note

Obfuscated ssh Client under Linux

Posted on by starwave

SSH is a powerful tool not only for astronomers. We often use it to download observation data from the observatory’s server or run some complicated programs on remote (overseas) servers. But for the reason as we all know, the SSH tunnels are always being obstructed. So we need some technique to get over such obstruction. Therefore, the obfuscated SSH turns out to be a sufficient method.

I will describe how to use obfuscated SSH_TUNNEL under Linux as below:
(Original Code: Here)

INSTALLATION:

wget http://aenes.googlecode.com/files/brl-obfuscated-openssh.zip
unzip brl-obfuscated-openssh.zip
cd ./brl-obfuscated*
./configure  --prefix=/usr/local/newssh --sysconfdir=/etc/newssh
make
make install

Then you need to configure the ssh server following the guide here.

USAGE:
Then use this to run the ssh client:

/usr/local/newssh/bin/ssh -N -v -Z ObfuscateKeyWord -p ***** username@hostname -D 127.0.0.1:7070

There are 4 parameters you need to specify: “ObfuscateKeyWord” is the obfuscated keyword you set in the configuration file (sshd_config) of the server, “*****” is the ssh server port number, “username” and “hostname” are as the name described.

Astronomy · Note

Plotting radial integrated profile from FITS images

Posted on by starwave

Sometimes, you need to know the total power in a particular radius of a FITS image, and the growth of this total power along the radius you choose for checking whether the object is extended or not, or contains some ring or disk structures. Therefore, this script could be very helpful for those kinds of quick checks.

Under the help of Zhangzheng Wen, who gave the idea of calculating the distance from a certain pixel to the image centre, I wrote this IDL radial_power.pro: (Download HERE)

;+
; NAME:
;   RADIAL_POWER
;
; PURPOSE:
;   Calculating the radial_power, returning radius and normalized
;   integrated power within the radius. 
;
; CALLING SEQUENCE:
;   PRO radial_power, image, radius, integrated
;
; INPUTS:
;   image:      read image from a FITS file and put it into a variable.
;
; KEYWORD INPUTS:
;   ...
;
; KEYWORDS SWITCHES:
;   ...
; OUTPUTS:
;   radius:     the radius from center to each circle.
;   integrated: the normalized integrated power within the radius.
;
; EXAMPLES:
;    IDL> image =...
;   ;input the image into the variable after read the image from FITS file or other file format.
;    IDL> radial_power, image, radius, integrated 
;   ;this will return the radius, and integrated.
;
; SIDEEFFECT:
;   ...
;
; RESTRICTIONS:
;   This pro is only useful for one peak structure, namely if
;   you want to study two objects with comparable peak flux, 
;   this pro is not available, but you can edit it to make it
;   function well.
;   NEED "wherenan.pro"
;
; MODIFICATION HISTORY:
;	Write, Chentao YANG (yangcht@gmail.com), 2013-02-04 
;
;-
 
PRO radial_power, image, radius, integrated
 
; make NaN values = 0
IF (SIZE(WHERE(FINITE(image,/NAN)), /N_ELEMENTS) GT 1) THEN image[WHERE(FINITE(image,/NAN))]=0
 
; get the index of the maximum point
s           = SIZE(image,/dimensions)
m           = (WHERE(image EQ MAX(image)))[0]
y_max       = m/s[0]
x_max       = m MOD s[0] 
 
image_size  = SIZE(image)
image_xdist = FINDGEN(image_size[1])#TRANSPOSE(FINDGEN(image_size[2])*0+1)
image_ydist = TRANSPOSE(FINDGEN(image_size[2])#TRANSPOSE(FINDGEN(image_size[1])*0+1))
image_xdist = (image_xdist-x_max)^2
image_ydist = (image_ydist-y_max)^2
image_dist  = SQRT(image_xdist+image_ydist)
 
radius      = FINDGEN(SQRT(MIN([MAX(image_xdist), MAX(image_ydist)]))*0.7)
integrated  = radius
FOR i=0, (SIZE(radius, /N_ELEMENTS)-1) DO BEGIN
  integrated[i]  = TOTAL(image[WHERE(image_dist LE radius[i])])
ENDFOR
integrated  = integrated/MAX(integrated)
 
END