#readme & help-file
Date : 20 June 2019 [G.C. Dewangan and Sunil Chandra for SXT-POC, TIFR Mumbai]
#----------------------------------------------------------------------
Content of the distribution directory :
1) readme.pdf [this file; readme/help file - pdf version]
2) readme.html [this file; readme/help file - html version]
3) install_sxtpyjuliamerger.csh [shell script for installation for CSHELL users]
4) install_sxtpyjuliamerger.sh [shell script for installation for BASH SHELL users]
5) sxt_gti_corr_evt_merger_Julia_v01.py [python script called for sxtevtmergerjl tool, i.e., python version of Julia based merger tool]
6) sxtjuliamodule [a subdirectory conating various julia codes which are called by python wrapper module. These codes are originally written by G. C. Dewangan]
#---------------------------------------------------------------------
Description :
This module combines the Julia based SXT merger tool, developed and managed by Prof. G. C. Dewangan, IUCAA, Pune on 28 October 2018, with a python wrapper which enable users to provide the input options from command line, similar to the inputs for older versions of SXT events merger tools.
Because of the basic architechure of codes by Prof. G. C. Dewangan, this tools handles events files frame by frame and retain only non-corrupt and unique entries in resulting merged events file.
Installations :
-------------------------------------------------------------------------------------
It is only available for Linux/Mac users...
Download the SXTPYJULIAMERGER_v01.tar.gz (or recent version; see TIFR-POC website (www.tifr.res.in/~astrosat_sxt) for updates of this module)
untar it in your preferred directory
>tar xzvf SXTPYJULIAMERGER_v01.tar.gz
> cd SXTPYJULIAMERGER_v01
> sh install_sxtpyjuliamerger.sh [for Bourne Shell Users]
[one can also install it in two steps as following
> chmod u+x install_sxtpyjuliamerger.sh
> ./install_sxtpyjuliamerger.sh
]
or
$ csh install_sxtpyjuliamerger.csh [for C SHELL Users]
[one can also install it in two steps as following
$ chmod u+x install_sxtpyjuliamerger.csh
$ ./install_sxtpyjuliamerger.csh
]
Above will add 'sxtevtmergerjl' command in your path [for new terminals ]
If you plan to use this tool in same terminal run :
> source $HOME/.bashrc
or
$ source $HOME/.cshrc
or
> source $HOME/.profile
[whichever is applicable for your case]
Now your terminal is ready for this tool...
Note that after installing this tool you may simply delete directory SXTPYJULIAMERGER_v01 to avoid confusion in future...
--------------------------------------------------------------------------------------
Requisites to use this tool :
-------------------------------------------------------------------------------------
a) working Julia code [version 1.0.3 or later; use latest version for bug free runs]
Download it from https://julialang.org/
copy tar file in any of your directory [$HOME or $HOME/Softwares or your_preferred_dir_at_preferred_location]
untar it using...
> tar xzvf julia-1.1.1-linux-x86_64.tar.gz [for latest version download]
Now add following line to your $HOME/.bashrc or $HOME/.cshrc or $HOME/.profile files
export PATH=/full/path/for/julia-1.1.1/bin/:${PATH} [for Bourne Shell Users]
setenv PATH /full/path/for/julia-1.1.1/bin/:${PATH} [for C Shell Users]
Now you should be able to use 'julia' command to enter in Julia Env ...
Use following for the first use of julia [only once, no need of repetting it]
open a new terminal
> julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.1.1 (2019-05-16)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
Now Get FITSIO and DataFrame packages which will be used in SXT specific modules...
You can load Julia Package Manager [Pkg] by typing following in Julia Env.
> ]
(v1.1) pkg> update
(v1.1) pkg> add FITSIO
(v1.1) pkg> add DataFrames
(Note: on typing "]", julia prompt changes to "(v1.0) pkg>") and (v1.1) pkg> confirms that you have successfully loaded the 'Julia Package Manager' v1.1 ...
Now you can come out of Julia Env. by key 'Cntrl+D'
You can close terminal ...
If you followed above criteria then is set to be used from command prompt ...
b) Install Pre-requisites...pip to use it for installations.
> sudo pip install astropy
> sudo pip install numpy
c) Install python support for using the Julia based SXT events merger tool from terminal command [using the same methodology as earlier versions of merger tool]
> python -m pip install pip --upgrade
> python -m pip install --user julia
or
> python3 -m pip install pip --upgrade
> python3 -m pip install --user julia
At this stage you are ready to use the command line option for sxt events merger tool just by running command : > sxtevtmergerjl -h [see available input options]
-------------------------------------------------------------------------------------
How to use this tool...??
The tool can be used from any terminal, if you have already set the requisites as per instructions given above. The input options are very simple. one can see details of available input options by running following [in you terminal window]
> sxtevtmergerjl -h
=========================================================================
Running SXT GTI Coorector and Events Merger Script
Source Module: sxt_gti_corr_evt_merger_Julia_v01.py
Version: Julia_v01, Release Date: 2019-06-20
Using The Julia based SXT merger Tool, Developed By :
Prof. Gulab C. Dewangan, IUCAA, Pune on 29 October 2018
Recently added with a python wrapper module on 2019-06-19 by :
Sunil Chandra (CSR-NWU Potchefstroom / TIFR-Mumbai)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Description: This tool merges the events files using the Julia based module which considers individual SXT frames and removes overlapping and cprrupt ones.
Please go through the 'readme' file distributed with the 'archived distribution' for a clearer understanding of the Tools.
Contacts :
Sunil Chandra (for python/julia related general issues; sunil.chandra355@gmail.com)
Prof. G. C. Dewangan (for general julia related and SXT merger related issues; gulabd@iucaa.in)
Ms. Nilima Kamble (For any related issues; nilima@tifr.res.in)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
=========================================================================
Usage: sxt_gti_corr_evt_merger_Julia_v01.py [options]
Options:
-h, --help show this help message and exit
-l LISTFILE, --listfile=LISTFILE
Input File List with full path of level2 cleaned
events files
-o OUTEVTF, --outevtf=OUTEVTF
The Name of output events file Default =
OutPut_Evts.fits
--sxtjuliadir=SXTJULIADIR
The directory where you have kept all SXT related
julia code...Default is 'None' which is equivalent to
'$HOME/auxpyscrpt/SXTMERGER_JULIA/sxt' You may use
any of [None, 'default', 'Default', 'DEFAULT', 0] for
default option
There are two primary input options : a) a text file with every line representing the full path of events file for individual orbits to be merged, and b) output merged events file name
There is a third optional input parameter which should be used if user have not followed the recommended way of installing this tool. This means the SXT Merger specific julia scripts are kept somewhere at non-trivial location... User can provide the full path of those scripts [as per case] and use this module in usual way..
Example :
One can make list of events files to be merged for a particular SXT obsID.
Generally ObsID directory contains a number of archived directories corresponding individual orbits..One can use following tricks to make list in one go...
> cd ObsID_DIR # It is C02_011T01_9000000850 in my case
Untar all the directories in one go...
Trick No. 1 :
> ls *tar.gz > tarlist.sh
> gedit tarlist.sh [or any of your favorite editor]
find and replace first few common characters of each line so that in addition to that character you have written tar xzvf; e.g.,
I have 2016 common string for the start of all the rows in tarlist.sh file...
so i replaced '2016' with 'tar xzvf 2016'
save it and close tarlist.sh file
> sh tarlist.sh
Trick No. 2 :
> for a in `ls -1 *.tar.gz`; do tar -xvzf $a; done
or
> for i in *.tar.gz; do echo working on $i; tar xvzf $i ; done
[for bash users.. one can easily convert this into csh format]
Trick No. 3 :
> find -name "*tar.gz" -exec tar xvzf '{}' \;
You can use any of three simple tricks to un-archive all subdirectories in ObsID_Dir
> ls `pwd`/common_string_from_subdir_names/sxt/*/sxt*/*PC*cl.evt > evtlist [PC Mode data]
or
> ls `pwd`/common_string_from_subdir_names/sxt/*/sxt*/*FW*cl.evt > evtlist [FW Mode data]
e.g.,
> ls `pwd`/2016*/sxt/*/sxt*/*PC*evt > evtlist [in my example case; You may need to change some parts to match with your data directory ]
(Note that `pwd` is not with single quotations)
(To know the current working directory)
This way you can make a list file with all the cleaned events file for a particular mode of observations, here named as "evtlist"
Now I will use this list with the merger tool by using following command...
> sxtevtmergerjl -l evtlist -o TrailData_PC_merged_julia_cl.evt
or
> sxtevtmergerjl --listfile=evtlist --outevtf=TrailData_PC_merged_julia_cl.evt
or
> sxtevtmergerjl --listfile=evtlist --outevtf=TrailData_PC_merged_julia_cl.evt --sxtjuliadir=Your_External_DIR_OF_SXT_Julia_Tools [Dont's use $HOME for here]
Please contact us in case you find issues and bugs in this module/codes. We appreciate your input and it will help us improve it for general SXT user community.
Best wishes !!!
Sunil Chandra
Gulab Dewangan
Nilima Kamble