From 856fbfd92064f393f3e794814a1d5187591ac893 Mon Sep 17 00:00:00 2001 From: chris-simpson Date: Fri, 3 Nov 2023 15:43:21 -1000 Subject: [PATCH] rewrite of tutorial (needs figures for retrieving BPMs from the archive) --- .../GHOST-DRTutorial/05_tips_and_tricks.rst | 92 ++++ .../GHOST-DRTutorial/_graphics/sliftlux.png | Bin 0 -> 29874 bytes .../GHOST-DRTutorial/cal_service.rst | 36 +- .../ex1_ghost_stdonetarget_cmdline.rst | 475 ++++++++++-------- .../ex1_ghost_stdonetarget_dataset.rst | 14 +- .../doc/tutorials/GHOST-DRTutorial/index.rst | 1 - 6 files changed, 393 insertions(+), 225 deletions(-) create mode 100644 geminidr/doc/tutorials/GHOST-DRTutorial/05_tips_and_tricks.rst create mode 100644 geminidr/doc/tutorials/GHOST-DRTutorial/_graphics/sliftlux.png diff --git a/geminidr/doc/tutorials/GHOST-DRTutorial/05_tips_and_tricks.rst b/geminidr/doc/tutorials/GHOST-DRTutorial/05_tips_and_tricks.rst new file mode 100644 index 000000000..63a31d435 --- /dev/null +++ b/geminidr/doc/tutorials/GHOST-DRTutorial/05_tips_and_tricks.rst @@ -0,0 +1,92 @@ +.. 05_tips_and_tricks.rst + +.. role:: raw-html(raw) + :format: html + +.. |verticalpadding| replace:: :raw-html:`
` + +.. _tips_and_tricks: + +*************** +Tips and Tricks +*************** + +.. _getBPM: + +Getting Bad Pixel Masks from the archive +======================================== +Starting with DRAGONS v3.1, the static bad pixel masks (BPMs) are now handled as +calibrations. They are downloadable from the archive instead of being packaged +with the software. There are various ways to get the BPMs. + + +.. _manualBPM: + +Manual search +------------- +Ideally, the BPMs will show up in the list of associated calibrations, the +"Load Associated Calibration" tab on the archive search form (next section). +This will happen of all new data. For old data, until we fix an issue +recently discovered, they will not show up as associated calibration. But +they are there and can easily be found. + +On the archive search form, set the "Instrument" to match your data and set the +"Obs.Type" to "BPM". Hit +"Search" and the list of BPMs will show up as illustrated in the figure below. +**FIGURE NEEDS CREATING** + +The date in the BPM file name is a "Valid-from" date. It is valid for data +taken **on or after** that date. Find the one most recent BPM that is valid +for your date and download (click on "D") it. Then follow the instructions +found in the tutorial examples. + +.. image:: _graphics/bpmsearch.png + :scale: 100% + :align: center + +|verticalpadding| + +Associated calibrations +----------------------- +The BPMs are now handled like other calibrations. This means that they are +also downloaded from the archive. From the archive search form, once you +have identified your science data, select the "Load Associated Calibrations" +(which turns to "View Calibrations" once the table is loaded). The BPM will +show up with the green background. **FIGURE** + +.. image:: _graphics/bpmassociated.png + :scale: 100% + :align: center + +|verticalpadding| + + +Calibration service +------------------- +The calibration service in DRAGONS 3.1 adds several new features. One of them +is the ability to search multiple databases in a serial way, including online +database, like the Gemini archive. + +The system will look first in your local database for processed calibration +and BPMs. If it does not find anything that matches, it will look in the +next database. To activate this feature, in ``~/.dragons/``, create or edit +the configuration file ``dragonsrc`` as follows: + +.. code-block:: none + + [calibs] + databases = ${path_to_my_data}/ghost_tutorial/playground/cal_manager.db get store + https://archive.gemini.edu get + +If you know that you will be connected to the internet when you reduce the data, +you do not need to pre-download the BPM, DRAGONS will find it for you in the +archive. + +If you want to pre-download the BPM without having to search for it, like in the +previous two sections, you can let DRAGONS find it and download it for you: + +.. code-block:: none + + $ reduce -r getBPM + $ caldb add calibrations/processed_bpm/ + diff --git a/geminidr/doc/tutorials/GHOST-DRTutorial/_graphics/sliftlux.png b/geminidr/doc/tutorials/GHOST-DRTutorial/_graphics/sliftlux.png new file mode 100644 index 0000000000000000000000000000000000000000..56c5332241b90fc4a17ece7e065f86a0a898c523 GIT binary patch literal 29874 zcmeFaXIPe5wk`UprIwK{6G4PhK@k)r2T5j0l0lFlA|N?~fCS4_Du{l9NJb?`$r)4> zBqKSBihyLvlJ8hlRd-eI-skMIZ|`&eT%WE-TfXqV>s@QkF~=BldPm{x=}qey)>9~y zO;Tr0C{ZXY=kVW&AJ^b-e%&=NjNgRqPM)(xVX&!{sK-*8zZhtj_g`|$U3Vt8nzS)jUM@LMYKeeF@<8s zEOp|j%9Ws@4ks<2?!~3i>9E6l${GK*qNGE*{xI($za4x{Thc_M4tk|k^!o%v9&#v4 zcw3hCwm(fHV^?3>fePhQ(b+u#Wr{XZGH+M2Y(J{7;?wNHd+#~E73(+tct5GP%UXh| z%GJFm#FW$Go7lA0w0)Z=_bFTrMfAS9y&(C6WW&#ctMEJ1-4`_Y?F!p#as2l2>5o_j zipT9e|NjsFn~Na1Da~O>v*YS`rAcl0ok54(*RNmKN6656?G!vqRZ&m~^XD~+wd^b~ zsgHix-*c-4TYB z0m0U~9O{YnUV$bj@|puZ|`ZFObxc|4Xjk_ug$d7jdSQM zaLt+N32bLBbOz9`9XxF(Z=Y`28``L*;-l^j!WkWJ!cj4vgN} zx}VQ!?9w6aoV3?Tk9>V2j6z-2b{%n0xo{y2>m0yu8ksv%aDJztMOuGPg@0GAV7h6O z+VwS??ApvTeXqnz*Sar`GrxWFhV9@%S#fdk^_w@Vc{7Pk^hJd}{>~fMrUw~vQwo(z zWzSEbl)a=xD;6xyns#+}8zdGkI`cXXYo*%tDUsiKE!u_ZUg#ay_v9||W)@trVnxpf zml6I@_r+|DluLK~x%F?YTfd&iyfuw`|Jfj`mR{a!QGGLkK>x#fzmOL=jlQMz^7i(o z=e~GDrbAU-+hBHj#PWmtk}LU!aqOT#?4+IU?(UC*f(Di5Uaq2fLaTG-RYj;o=Ru8> zH+iGQJ0o!y`wkwgU=+XF95EH*GR*Ce+mZME^F6~-Dr*luLJ$isIyabVG`LnmLSic& z-R0TI0d+H%>JTxxOP9j4yY3&UcuT8m79e1**OFriFt)gpIDfY*KT>iEKRXP{%J zYm=-$&)W!#ny{nAZEbC$WAFC%1X>nw-`jDBXC!a@KE04tSoTQ49D99K)Y)L3hYue{ zDn38e^GIHt`_7#^hA%xonqr zsTFIPuU=3~(lLL0P~&2VQxARDz<}wo*kdt>8%cPzx6Sj$B9e*@D}fcR3e#M(x4R?moPPvu1#O zXCPqt!OWA}u&db1uqjbntpn?x5K8M|>-~bJ{&17FV{rD@XZK{?+}x7QSFc`OOs9V~ zR9yR@!cP5)-cWaZwztG$v!>(sZ+gmz0JbCf(>e9xc#y@0so6hd9KXGJdt2|dRck#n zGadbpefeee22U1Vmnkz-Z5x}E}xuGvLQ@_S_<6@N5{u`-IwM%tDC2E-4`0p@NC_G_Mr6~vu={8>nv7; zPRbDRrZH!J{%T#M+)mpDRrTOxxnjCH9A@QA3++e#{!Wdp=PmOa>Jzowf<)~5&YfN!!uI2RFFP= zlUC<2^{(p+4r)MviCX+Q#bJRZ4Lv=*WSW+fXM=?F)^FI*;pQa~yH0P3?D~R&0;7bz zgR2_u>4{2iviCcBDx3E(;6G*ZLUc zoPoq#Et8R)0Zw{-KFbfHrrMV;Uru%v2|mkjQvc2%G$KZ~uz)2y&Gkc`6Hl?v_u2jO zAu7j?9ZO#DMEKvtz*qD2{c{=Dgl7dpfr!!axFhGmSx>6MMd_N{7vV)5kq6wGF_3}v{S$b5lS=d@>7zM$|@`EokA)N>)t)wiz~H^u^(tm zX&ajTRz#Dq`xK6P&fIXWYN4AevP6^fB7%7&r*_V}>S}o$&i=Y6ZvpdGm3VE_rbJa) zS*6?;nxa{FHrQpu(T`^iMzigl{qF$VnHU z{ra4DYLvNi$z>!z>d}6A%_Iqv3Ykx)<42EK7G?*Ma)vV6 zzSxN*?7k=FJaOS2jl!L~QR^rTv3L&7I8%Ji6PHQaRoAkti}2GN zpgwxUDuwW5A7fq({ThoBwXjx^JIJRgf@)poP{Q=4~?LcuE6Z z-V%*SL(lQW9Q*g5E-fu>yd9cy8Mn7(%a(ei@lDL)?IxD**LtK%$;$eKD|siUwC1~H zvcwpaKc$91-T*3sG2D* zbU3@0X_uI@3Nm>uu$8f^{Yieev9+z9)pMTgZ*ZRN zi%P##x@BOb>LjhUO8oF$j>H<0B+?^v3*D&Ur@R=Xs1|vnr?zj~_7$0ue4`zrGEPmb zbmB=0=edU2{FNRTH#3WyC1&?9>@7XR^WnpXNNjS$)->ZD9NP$S%caEylBVqjo8KF} zE5-78c35zY3S-0Y<-1IC+J^!$MV`;J*mwAFO?Y_twzd$2y>w-2UwzCYVXKcGGIB{e z`LZWYoVYVLJvwSlzup!PxDJ?+7bo0Q+qme;;#b+CD}Z~l!6Npul9C=ySdjS9Nd(58 z{0$G znYNvsFedP(4k$^fb2`nqE{U% zeQDaAbKu^smDsbI11$8ni$2-7zGQB5=nZzviI;s)*$cz(*uM#CL9zLkl|!XkfC z&9noL>=I|r*FCmlUrLqQG+Pgcq!gSzdi0vLNN^7xq}9x>=2Ro^(WxoUy2Bw)kKbmH zkSM%35GY`7a%=N06ERM6y6R6wx4pv0#{$-Uy!!euQ`oH-^`sCaoToUd12es$O&%*& zO^jDdn7+QT&ZI3P0nb@$P?#J7KxsC8Pm+@_8g*CMsimfw7t`VVXmO~;*O0?r?6s3N zZr!?d?~!5CZMvoN5gkRGC!%v4M-uDn>(}Qm;y7|zT3X5l2_@IhjpExO=o_F1~m1(we~+kL=2#f#?yALgQT@r_0N@ zEa4)3!?(=oQVpv^4q=~kIQ0fc6l8w;W@SIvtcbwTh>e+qEz^&z#AR*fV65XZ#4iSL z9Uy2KLbln*j~7U$ia4vMu|0s#cu${ukyTmj<*cUZPWL2iM|mK!x}nTYolgr)!Zs&9 zefsoP+~el0Td@Q>x!sS@?o_PJcsB4OsScf7TyhVFx=#5|?p_1{e>733$Jwpg*mS^| z%biI_y)`HHLTXFemC=ineRZd~3cf!}%=)~(AxT&F`DyP)Y}ab3ogN!j=rIe?Tyb$R ztqqr=YL#ReUAh!)R2@Qc%nnYi%U-LNoJA5JO+0&EISj^()bp}Lzn7NrEc50Rk_DCEAH#{ zW@auk&U?t;ad`UuAM$fQI1e1^u&ox8K69p~aB0C(s?hzyW~6>~T~~vJyy|eBd}k56 zuL{Yn6BBm9;;x#zckkvLC?EHddvCk6Fyd~yFgyA7F`}RJY^VL^x#R^2c#b6rgx#wi+t@+VsXY@T!dD1L?b)d)u9VEKoYdI1DeC)F zo27Om*SQyxwIj!mvh*PplvP!6S~I&3&y7v=At}WVwPl{)Ei=3DtVdpa<^;}Y%G3;k z)x<+Ut7@SeQ7kf#4?bk^TjcV7@!~KbsVsJ1t^-cMXF=MC+74vU9d<28Y(zA zI3&~TI>}o<0X!cxGUyJ{;yD58027Kh)G}J(om`v+K&umrb0eXnf$_Dox$#L`fBNa^ znoSIshugC`nJqxj#Hq%qseX5T9F^eSh%!eEu?1nQuA^mwZlMk$cO$T$er3gODRR~ZTT<<(I~?NxT{-8{+J+j}-mMH2G#zYCo^9(~ zjNEZZ^Idtl49?hllpXN^LB^DuMy0Po_RcGhsRwM;JDU?m=WUgGf za;Io5V)t{rDZ4#Eb#@`pBjJ3e^z4G{7jQZ8yg69@7F0&2DYS4tta47f*2( zB5&OIfh1k>{(5BpEurevv%w;u$Ta=59sCg=(r_+Z=SNHWYa_}vqH^Y+dw5W=g;aE1 zCw-kKzCK0#L?P+ZcQ!uVq(SNG)ja3v;RMt)Uq{@RauHxTL8=UO94arJ(R&pX`$`l5v-2n+i$L`ij!#Jpv4$aV-m$?CkZmG-}`meEt1ny373rf*n7+ zNBw4pTsu&8WTbwlW!_U`(X;p+KsUe`&tkSjV(}>{ z*K>+L);v9~nQ0L`x#=DYS+~1)?kM(F1@j{$@Eg~jIB?(q35A3mLo~^hNh+9)1q@NXpLuRDZ&D;#`wm-y9ldr#l5Ka|=2OolW21n9U~oRoB@aYM1i?_Jx6OepSZ+Ujdo zv9+KuG+aZ&WX75OWlPt{h$Sj7IqW)FGqd=X9Q!!#;wK zI8c9tdjEZL0d&mF%yl@JHypXe5G0dvuEF2&-0?Y9zmP#|}f2rbP4kg@sB&{ULFuZ?Pg|KP7ilfg0+DLVw;eI zXZU1pq_C6ka?MLZU#Tb(6YbKc@7PUAiPC`$uhzLeC@Y?A{Y zRN1*WC762o{T{y@P%VKwEwWw-F>b~WD+x*mJ{wRX)k>G0h!>cRhnGl_7;bVxQm

nnt6zz>SeJWe>N)hP2cGi6yNK=t1q`k@%{f0V+^}u z_T9XBlR(v#D_6!|71)TM%3tt&4`kGBC*ofk=lSDY!x!++Chb|)_JUAyKIWxbc8Y_B zip~kN*7HV&UcP*3VzCK|Po*C?bL>|2KG};Qw_fPV-MuyZ!klq}*tQ~}tJMnnw?+@@7HDyEbK8gnr?i+6%v(M+IB14iNQnSM zfNX@P1Oodw`LnaL8Td_nz~X%A?TuPom@`F9&dHB>eb>v&x9HP@qE8BlKgb$AN{^`2 zXW$r6>l|QVIf0a_rlS)gQwc(C+xG3OfIOgG6u^BLZYEHLCc#o}N(8;#j0`9gp#d@CES! zRl!HfKUrnfgM%pRv(x0qaaOdjNE9$@zR+;u(GC zPNz5N)PZ{;=&J!mo{o;ra!G;9IYUJO@ujQlIqF@%UYyedhYnT7uG_TfP{v42je=hQ zkf+}epyU|Xml)&5c#Y(y1FelHzAq*{3=E8GZ`T^IR#Q`R`ar%;!J@a)f1ydoMGgeW z`}vvibiJZAgngnB{pNuHZdcq(uMZ3zU*bORXwdj%$=%vQs@gXkflq^W^%2I+{1)xs zJ%@;J-EOaQV*voK|IYr98q~oAR+V$+%92bQcJ%rL9=Q}1ye;9%JUuu1^+rZToj2+z z0+ai-=IO<`B@}mBCU>ho(w3qoc5_>#H?6I!V~uIfTzQf}l!^*j(-!c4PPrsNu=$Pe z8SL%t<$%Vhqmx*f0D%m^nN@0+h~0zC%;gfzTwYF26(U29!w|a^aML{=US6 zvRu9o`LGOqdQjy6EU=HZdqpnmKkc4kI5Dgl&7iKKB-{1wc6{ zWh2B$=mdnosJHiMO_7CiplH=+n4g~^-0GA+%plEI6P`xs8# z-Cuuwzye6@;=%v zhMyQny0I5B@Gf7uWVeWA50V)r>j8*YpO~uZ>ZH5pe;Hp#vcgBvj|kL}BBw5QdD9gW zJHZvUD&ianTAPxZ8kf6Hh-NumRIT!Sd-~+zJjG=MaZlC?)YuG+qZ z63u|b@N#u|XJ?_Teen}^i**#0O@Cxy-Z_fL1~n9T_JS)YvE;+8UR^vWPLW4>j0br; z>o8;QmgSpg{QTM~NJcA+?8-mx`?wa}ecL-o>XO2~_7&-GvP`#?a9{iD-%uLvy`Xuy z_i*lN3gs=<=gJR;nJ=^7JTay8Zh#9T?1ZNbSI5mMw-rH!NO7VojWju!yJ}Z_oW~V> z{7I;8Jw?(QHz0ZEd&UoxD)&GAQKz8;*FvGF{thdmk!$?8bQ6)D5cgZm1W?&k8{}3R za#;I|yXL4qLNJ2%O7a$#roXeWaB8G;BqbEF;cano*(a-a*cAc-YN)_$V~}GMA;bVm zZQHWt85B?sF0SebL*Spgckhz)$sk}BjUtX`C+_QCpVqMm1Uo{AvqcFOD$z#eH@GFn;jlmmp^DEp6S8i>UIPS8?O-sc>^* z6tO$gd3D_D77epSLfZ?y>)=IMczrfB{N1ftG!5h>MS}K7t4i1}-Q?uU@@cbxwIAE4rr|-{9-(+ns`s zqp0@f1sOp(>TG(j`?v}cYic9If0C7q%X|(PnR8Jn(KC2dWm@H8916izxx90P>IP)8 zgLIP>PeVha4xq%t!=u$0_182sznq2^uR4hD-FqCk$|zn+UfwVKqS5?4*(ZmOLX~Mu zfrBQczyRr>==(h$Y3u_9&_mq^XxLZc<~wV2@- z+?A_VY40El%---sPqd|_1s~xzc?DIGw6wI6mDP)7=5+c1m&DS%Ez-@o($6Mn+;vxIM_lOT#O9MzV;{nyJ66|d~;oD`MY zK|w)0O|vsIs(Lcmc2^M+Y5CmUmw1|NXpT`{&fNM(3E{Cp^Y_^9!G64Xt<_O|Msdo^ zMonZqvqIl`O#!1?&r5g|7Z>TmDpivI028@mW1 z`sZ4Sl#^B;I0hI8JO|cTJ>8AeSmk06tRFnnutMwdpEG8aOGWZ#3M)(rc!lTNwH{RN z{|~a|AE}go-yqjN6-zqF0bmGY+?N)CqSpWTqXMWRUn=#Yd39xF14JU^ggv0QCPzAT z!8<#)oIZ8x7J_!Sl@K2Eww*f(!<}7^UB$L<-zg-U^-#LNodSvWDy15f?*Ow$SRxR% z^hlL9IG64 zUzOLeg6?fe2?rH!RNy*C=wBkFtl7*+u0rZBMbfQ57fDP1TqFtW%f0V<79dexS*fa~_PBgySQa_a zhk!5#P1GG%l|2J7OeLzk*RCUKRJ)mOKT~*%oS_G!##ZgO_2NcJ-`FLwn9E~Yu~aCO z8Qv>h3lSPV10SWfbsrsJDKUuU)%qI^<)ub2t2gc3Iaa^f6thn7irVk&6mwGhTkKfS)$IE8crR2m2ApBfI57@wky3rRJB zq8JCz;1_^|6%iS!$q%ax94-AdPp_+{UY)V=>l50#bt_mPxIk04bYAbUP=`y3tQn&H z?`u|ZI0V!NO006{JeO$;R#w&nu#CtB@V(#bE&2@mwGTE2W#la12M@xQ=6fYby~KhV z;PUrxpWp!rB6b_l*NIym;KAeLDd7Ud+N|L|XY^JD(PR2wxMt{U}!Q5?bSmIxNa zrNYxHr&M$@jipowIyIyirO}KK}A{ zQ0JQ+G7_mH7{x_*k=mi2&LIE7;|H64HsSwy*xvBhtxAlit*}Wxs31UITJ@8m+X|~h z2#d^qP*t;%Ql)Ega{;O=hp4C)HSNNy zl^rmOn6#uskZ&c0rxL@D6#n-!cDEkW?2F#a=qOqFQLUL~!-hI&o(*9~SAo9zim2to z5-enW3=uunXU$4VyREqE;cLAgf4?M1G!fvNRpFj{P-&L6(K303qJ@)_`TMhW|hD^&& zX;T*R{bO+o&4qoDfh&>d_JH5_bnL{wDVix)KpE)Y!GTS#qB5zN{?41nUS!g3w(=&bFu3CV$O;{k)z8sxgvD$fC!IQA8y*M%Mmp zy$eL0YH7lYP9Rh$I$0^LD_nRKIUnZ}BTu{&jte<7wpta>zYjo^H;ad4Cyko>9O ziWRRj(a|Y%SNlGG&Xu!9?1;tmM)~5yOcY5~0K(Ao@aci|*BGLFJo@7|9GZ=IfFO9v zOi+kZJfwTnnsp1V&d_J-Lc|*$1dyb7>}+2@2Vl>UG3?znO`l3y8&sGx4ZFZ+S!saF zBJqwG{}5As8&2V;l}rKW{W8qYRhg5M&m1 z2k_{WFwb_8)Ix*~zmj4)MCg9dk?JrOx0}$B51hsVb3y58G0V7^nL7f3n-sm73?S%Y zX$9Y{s&dC|5Xnk(YHUmiiak-(GZ|J>gxr7no7nBbLU(`DrK_K5md}bMYDt6iRg{mr zfBcoAV2h7tHHP)m8J0t$MxvA^-ZpR1v0oGOr&`M{Beq2U>{w_}3YNU0hZ4O8-$itC ztdJi`%qZK->7L{4kgXWNYjhtxvX;is50s$&KmN*pJGV3CaK(C|r54Yfoln8*YN8ZL zB4SL68)|81=)~#x*36w>WB!idW1uheTWqGhT(vNoIAv7yfIWx?)NOC0=v?9AY~00* z7w;yFbQZcN-*c=cPDaR#59|7T;0pyGs+B;+ja^PhN_YLD2%02=%iIG4? z?*A2*oH>p-rwBf(Jdg5C7V80cXFB@%?b}8!oxF2I4{j0pm@?WFPE(#FJ=f5!7&ayp~%tP>*keE8)$X6XqQ`s&IKc!oy)9^mEazO?9c& z;&IUIT;VeU58;D7uovI=?%kOM24)$v==h1P!OGtjtD(5+(}Wv^)(V=gm-zAS?lfq&vs={bSyoPq%j3 zl^@EC2A589U5Io40d4n%$yQJ@=qUM*EWaGp2R@^knwpDqn>TH;k6E{2gKZ1bwrwm3 z)qVje61KYpKQl)a%r(UTdbtiFJ^NG-!kx!@D~`b=2rp0zIE~6!je*TbLy3sP;8(S- zMR;nDHWu#PySE`(|1^P$gdM_pJ=Z_($<0}0BF%77nV$b7ZSP}&8%A*ZIe-6jJE25F zL!-MOamCAck+BH1#2GfIj+RLc{>wFzKnx8~LRh8X+A1qA zSH_W&(MG3^eWMtXHyqyV8A+XPbCnU1UpKlv1OeFH+Z&o3dHKxy(8BF(7c<+C_P(Yd zy;wbE1-XPU3+q(Fc6BX9Gx&Li(oxMysRX1PN5TbXo%Ozb`ykxx z*!f{qP&FZx^01r%NL7}ZZ8D?Z=L#NUgE~ekJZWhv%g~4XF zK(PepA|#<&y{OIg(MpK#>US&BnRSnrs}V{R@m@p&@HP&L!gitdJKRmsNIgZg8En*T z|ALpngVxNulKclRW7PF8)qp2=)-pxK)84RA1Q){ZV_gJ6Mpi||2Mo-UbWYIt(4zd$ zTVzLQ&1wEaCJ?s%e50=;HfY={=dOkc zaXPP|1j%gQJclSvVlep<{87~~kn&RBw>RVmt?HE&EA^WtgH5rjG9);n_l`YT7#~7m zzX#OOnqeLX{G*C-kvxDtAPyYfbfZ-Kl6jL4B~JitTYLe zAWIvN5<4sy@e+@_wj6^a&B@2nbEKU-3DK?HjOZKDhDe;mx`qS%wlPU7TZ!0YjhgTy zO8XbJ=0bcN+-Ni;uAvks#s%`6N&N^JQ_>*)C%jN{br%0*59GL5Y%Tkd589xSYq5j& zBd5UFU<$j$KS0?oyL&+>3J_&f;gAIk+6zLj>~i4guFY2XtnTAbuQIC!x!~A^OC=rt z=Q!jtHMKx;Nh&LGr@BkC>fJp(u{bYKD3s7gNDNc9%PeE`iuN2yLD0*fkW``V6UQ(q zUPwTu_BRpHUsNjbQ|3h3%pjt@rrI9-&zuLDKEvkL%d30I)qPa{iaR6e(HV4 z7tIBBI@X9?AMZA?DG*uX(O=Se@uOp}j9aW!nX6&pix)56xEJw`hOVKgNG$GH70m|B zujed>PYdh8m1LqPmOP6OZgb}7`$$V^CASl{1$f$he)P0`U+u|i(Q!}k{G^o?Mqw=+ zAKwNd&r`OO?fv;K2cPS#T|{{JSJaK^7Kz-CUE<)*X}2T|C{GO?Z2EoL}(D2JPR zDg*t!dN*bT;7EmCT}H$H3ww|z>%e71!+@lvuHzH|C2fKj;)@fP_t}HCJcD;C22zx7J+Z-1ws$xBFQB(kQ(fEM*A3SWJC{76RoEFP{#oivs&6H( zZGk{J(rj2h3*P~pF48a!LvK&jwdj&xY_ooQdeft3iLyaLNa| z=RQ6@HbbOw2r=Qvz_(D`2ni2hRNrQh1}MnnckkV+gHgpdI5^?!*UNsUYyW+=md68P zWOJ*QWxXQ&DkS&jSv;GaTwF=(N7RWH$d%!SHHd_0zfV}W1nxj~pgIco z7bef1JwtV?ZGw}1u3!76?Vq*1m}9hG#pk~jQCR{jBOxEfqWP~e1C>mIc5X&<4U=R3 zx7QnD@#jZW{sJBJz@eEL98$&F~wA|;tKC|c4GMq16dxa0{=GMcWdt7}L%RhLrw3;k<3wpW=x?j-VPy8)} zLxhL;fE4u7bkgHPY`)NEAG35IKis{4KVzy6$~E%mDU`nfa24D~J1EKKZ+`u$acv-! zd{gMb@CG~yx$=44PD=3QHt5{29wA^PRS!5eI|?zXh1m-+`ZhiwU4vjpGMQ+- zESYg?QFAIsx{{MgHXv!_3Kn%zf(gF{x2l3p-MAjiC1^mgoS@-?28HR9V)9UDU17$9Ldcg_~heHbg8;>bhgNL3gRGLjvBkb z%^HKebw(;^xA;4wgqp&vLr(C}AvyFd!KpN`-1r6N&$m{Re9`tUwPyLVXB%M;W|f*P zT$+;rQ=Sf$Wqs)R3OmBMN%VGqtFka($}he!@* zpo(Sw4$-s8dn1f3veZ@%X_PN(W?#4XF7na?IE26s~rZE)!bUEu+88Tj>y#v}0 zxJsZZ@>nD*|Fsv@cWz||p_wbvAvkc33@_(Vs*QYp3%I3}p=QInq{2nbr>C@kd?JEJ zw4937aZq3xOT6nqz9IS&aq2?3^=T7qAY@Zdb!ZaIR_NHuHVXv7k(*0?tZ)>o0>@Vz zC@GT+y8-E4~HNmk>EngUHH0|=^qr5fS|Q-yDR-2N!?sP zp=rF#)I!}Hfc7vpJ>3NHbMo~DE*0rj%QxeBUau1!(FDx1fH%^0n|a&nfQHg(&1Ix? z$il0IFpOYf){-NL0CR>=cEwRsp$4&O9JR8}}TbehtdXy9pAgRngx4scIRqWP$lDMD#>3 z-F7|{RwBfotW;Bc48Oz6<)r4ptdsyYq2!^d_P zr&me(!lHa&9LZYrH0tm+vy3vc+kYZqSuHG~BI&p<*psq`Ncji~^~f`bGL~xCA^N1N z6%1nnDp7cFkcZUJ$g9eG<=X`!cES=(ykjurAOol8qY}f(P==3yEC$}@iXKcNbE=Xt zG6-s=eRd{z1vckn9p^sQ${^uXrMVxX4-(l}2T!0ky5=ww>|w%;_fyv_5|4MhqCg;= z+4gmEg1(x{6Z}=rPTs43ewqd}Y%KcuNqr95?PsZ4B)|r;5zy22I}b~mHwmSL;6P;| ze*ho$9vabh@UW!E7Md!0HbPO1upEFR0X(cSh&uR=Cac{SmACGbg)(}3S>QsUl1H2g z3v<(ceb@h{d;F6Yd-ZMGf~PAf^(sW1tXZ8yCB62z?lLnOe?+xD^bS3QH5z&a%d(>L zyEmW#20@~qAqW%o2}4f~-7eBw?J7k&^dNk}k`@6&Qi4vtCMLG1)%b<~16(RVNKS`O z4kDw*(ZXm$5R@>YdBbAww%K_FTkdZzB%J3pAZ5s)A;@YRsfUavVZ$59XtSsTAppALJj{1!0ZT52Vte^=w2T}8XZaKvO7#7ERy`kRx#odw<{SqGdr{U33N|?h z`TDAcWNyxS3#00CvW-1@vv^Q_>dHb2>9B*v=YjqAz>&ss4aCgNCu{r9L&mT$hI}AL?am>73rP(Goe~;i;qi4 z9R|g^+@-m&h~3J`f70mNJrv5n83#@>i!9D3NdCn)@F9~VXCIlHuifrXCf zVHnr4(ZA|d*T;tzvPd`^B9WpE5#Ik5J^Fv_YO92wRm+47=YhPYN$j2g%l-(Kq+MZr zu9CDX$Ql|N&JcL@K1)qDN`n;y2Fc;yAy+$b+^-v0d+|FiSm zVqu=9L?46(W{@~JJ|6e_F&*@a63fAHhfPQ$<(w2TtsOce%i zWIC+(0V%@$ywEFx^p^;Uak)AE#=5PfDHqf4q-x2q0?=NhCIUV*a`lGV1?wStM##Lk zK(RHBe?1UYH~w%}BLzb+?IR;`Sd0-#%BrhLk8uJ6o-#l?${lHH8q$?8<7N(&8=!k( z+4#>yMUv3|a8|448>sw5HZZ)+l-@_Hc%(3QCUrYOPoqO1RiR3=y$P=!lbaB0Cc~k&K1-0GTqiai_l4tp*>y zH<3IhcTXMng#d`)3Eyv-M%z7Gu0Kw>0&7(q?;KQfG8HNvo-g8iHi#$1J!aVXz_mLT z^jB9~;eXN97Ls|N9bIthJz#7;UbXpfW8qm?Av|0}StcAp%ZER?+H(FquC_L4d%p>e zSZURm;)>!&s8_%WNl^iflE?@}=vg4^A7eWco}VZl#IjP}G7fqJ*x(Fx5na{;nD5p| zC=v9+gOeBl1}54rP!6lqviOS~{0}o5(MZXNK&e9cd?y@34$KK86GdQ}03Uyt%*G%H z5v~23@BjL14bV%{AHHjBWHPhJ?|?}4jkr=Abb`WH{3TblST?H}O>yyE<-wC&7A=AW^_RWYJ~YIX_V4fU{KHR7EN$qnC6?*` z@xIc3`p$;L;Wd=rxW5?0DblB!*RsDu`%gtW+JDT}Ay=3F$&J0;CpcKUgNO%0dtZwy z9Q7;$2f)AU_gwX_sffScn~K-EIW)1UqhAIv9%CG#2=t~bfR9r_pd?;tR9?_aFV4OD zm-gux(tn521;88sgO3^`T)6!LKun;{BqP!QlNq36;Am@^WSF%)L)#msbjE_q4iK?t zca*{tmh@8kYpSUqB0^aR+^9;^-2XR>(l`xGdvimgvm^} zp{}7Ja~Nde?V%(eMt>!l`UK-Md;wl8sP`D+3=v;WQCkXJbJ1e6dczLVp<@gADb5fR zG>R=U1EC{gpeC8}U@gSh1MWnv60ToQpg{=H2F~8dSf{Az^0u02AB{6{Nu{Jme$ zM?&Va6$lcN+vicqjRg$%K}~l6?g0bWznD3u+Lz57J=I3NygWSUk##R7re-O2hvLF_ zNV90YFg^r9D(+=2QNqfq@YR2^YYZqUEBiw7g!f3Anq$}Z3=0jip(<7bv;aJi+9qJE zeKB>cfd%Fai_AEbDr%MB6|^I}WHAa3ZA^H8(q^H5n3M4xDiN&>9UgoBb;ryz)YR;( ztS5oMd{|($J6=0t)PlZLWad9C8rTcUFz*tZ1}#W@~G)m>>al2qUgT3s$p+x?&;~6I zMtMsnoSCJqB`{HtrWj&2CKEML(eGnrecNq+pHcWCaV4XvpBOYqGbeE-#O*TA zc`>>^$wX7mIx`0S3dD^{G$NbAjkJ!2Mi7!P=XhFRaZpjr^ORJtnT5iOkH?Kms}ftg ze(5Ua_3Ib05Ynltj>uW6qHKRzAJ}b#;S2pUPptLI%gU%I`%$YjZ0;*9eK@}mXR#E` z(9%K;)t$^2FR7e~$+^Gtz8~&$DR>3@$Dr>0G)ici54&5RFgM3+7o-b?eX*%GOL$48 z$+~FN#w~WZke5=m?sY*_YKxHV^|7w3qfS|;t0CTBP#m5dP+-d?shhg*9EtaSRWgkP8wFj29$pod^>h$=c+~CaB@jaY&NU^eTiyZ&YU0bY<1D^z>j- zh9bI<9)^a7LN-bQhZb7osF)g@;C*}3PPM@-q?lBhO|DuwFbDRPaU`kWz#^Cv#?P3v zb?`HmY*&B0*iGjE8g53OjI(U?N+H1#Bwk}{)u_wj-a(fvLb4MNq{QMV4WT=TUO|ix zuo9>^)>X;C{1Jm_n*hhip~eg{g9<%H^}wx(Fo_alJ&LwkVAnX@%lW@_LK*bc{U_z& zFSe5cCTXdY$t>Wxn!2-eUR=0V?l7BWhwrGvWaDVU%D|>2f{COgN}^nhyVaHTxvaKu zOUa8P7=00rvBEkMi{GCUivh7jG913(vHh#Co0!M7)48Y#E6@r$l0sbgWN>s+IH zlUdTJWvevWs%hPa_4PeTpFI8H3z9L_Prb|ci-~od;lT~?7F#i@XGlyZaWfXF?uH1R z2a_^mA=59sb4&-J;u4!W%02By&{@Rc9vi6^-03jfjzms2(K6u(S%_VVI6k3WYt6QT z7Gr0>TaXx*GSTNIMxidaUN1e|%lA_Q=)mFVNJyZO4wD6me7(aJ4kl$BwI` zM4otmnz@h6K*WD>AU#QcDjbx_vyTcAMe%t5&VxqRGX%B3?;9pBv7pk45rf23L}pGB z{|NXBmf0*(Pj2E0B_ny+0BN?ASR1g%xU=5>RQ3XD(9GOixkcsxMl7kr%!~rEXmnsV zi8Bh5K&R#U`m)LPLb+rL&#WUjkp@W7+IH_xE)VOy_v-6FnfgQVh|Y2{)cZg2sebMH zeCj@93_NqDuqwaJux89nN+YL37d{rSu9#-Zy-d5~p<*cw&3nI#6Ytn6)UmsW=n9FdCjfm23l~LwuP}4^_?6HD0Oj31!cJXALp1V&=8GY-x^S? z7&{0086{>yq?*%xSY@Ej$&v2?U$mWxX}Gq@$;k=!E_zq(qj)rE48s)KwupVuWMEyL za9Oj!)6fuw~5cHCs5p<-XoDy}nOiVEO34 zf!I&aE(9G1Q$vQOV?tS*Sw~I-7y{6heaJon#t?yy)l+GC2nQ7sYh$_S z3n|N^=@Dd$OG-+#prvvuKYlt0ui+&IzRMR7APBh&X_1dib5GAlJButZ7Obi}(Y2MB zJ1m0s(Z?oc)|HtU%)2+HnFj+B^<`UTC!lf?AVu7+-)Cn701GjyM`_bYq@jp1bEuH@ z>xtoxw)!O9PSUYSrguSB*)br2Fv&WZz%u@7E%P}N;~g`|F7AY2XKru<)L9^tnDe=# zo#g5-0A1NmixD^uK9(EpUjOZXIwQ6omunBwLEVv83%bH-FqLpKH+1-n6S)@CO!xgJ z)~Q}*2D@Dc35-2myP;2uNg4Z+?C?o4HY{BLejn23hcFhKr%f^0_gdUzeJThvE&NR+ z@nxX(p`k;C{CgB+?h35*ak&Q~Od8|;b)33=rQ1S}yqfn0z>Y-A9x-~57m<85z0)y- z9xe_tiPX=&ckBrPOIV(uCS6hzfdO`}deLxmIXXY^DV&n1`?(?Mg# zfcDrK%Msi#N(-_kxVq@d z-Kx9d=s5&3s?%K@qm}dLITFYy>)`RZ7XJA-C+wc=cr--gg>#H_*kRaB|JbXORN^IoCp8wZ zg|s`vk)L3Whvkg0-+)=kq{Rij7_g-G(a%|8Ch_n8U)t1U9Ac@DLz!h!k6`Xj3P%BAQ?#2SH5;gMx|#NLm!vv`sPwsfMVF)EEyC zSp5&B;A2R>)>;_b@>~3%x{(rD)bEv1;}frip+KP99WKk{|(6?f(=ehP2IJbzP>Mnl1Ve>|3mkY*V^)C1|_r! zI>Gb$tQJIuSwpYH#ACA0HwT*a!lvhB`$yOMDQ8 z+h4f5vmm23RE#yMX`-{Jq!dbs4^$T=&1=i@X~iz=K4J6D`MfF(R2_6#V_CjAPZgodkXU z9O0fRl@1G?-*R1?J7;AxK0h<2(65@PTx5x$k-8lKqo9$@ZHVXnz}hwI`RD&e6d55p zolsBPdFf`B&Ym^vQX3-rnOi(xy5}A7^>_L7Llbtp135QP8`D+I!S!OIk^SL^(@vy; zm^5ZK6igY8g_qzXtUOx=$pzm*W$&Q2!B>S2eHMYlzl)Au5Tns(>vC~&is5+f2^r^Q@`QQ$ zhv`c#=Sg0^a>b+6jw*?h@VR0-^QHloC_vC&-%ev-qvRd`Xk8J~-}ops;75kA_H9bc4qy>5T#+Ksx8LRuf-OgP9cf^7$? zLiC^e4e9{uUV56$uc-R6E0w1ueOJB!sX62_HS=~wY(@}1b)2_g5opPbB|MP>BruDaH< ziN|;Q$|P-F#M9yF7gz>mJrK!PvojD_((IkOqv&g(;sec>g^!CuutA zkXc4WkVyo^Zz2U0qC?3A!1KNCvI3Hxgy4vc92w=L^=N80_NKRPDGSz(L=(h)+zBC| zDILE{`6BaMh_D6lMl&O$X!<~6%)nn=_x!SZ=yuWFEYHi-#N>1}rKq`&-hg9X12us7 zPuq5wW8Z3YnCk^=rQ=Zl;NW||-&A~kS!8Hf*c^kP$J8dJi=WmS$_*T}J%dbTiM|Q~ zr{lnVJXKaXq6bBbe?NLweyfs zNgFWE5Zi~Ju9)K&eQDAK)TYt~3k}CDEjcEhxg_&wbeQ*3(aX)Z|6QHryTvf-Yd3oB z2p@tqaD;FwjwcH$+w%8aQ~fDLfLXjba*!5YfB?hQ{gm*kZwC%7CA_fE3c}QFSMSju zjuf$FTGOC?w9?+lZST)FkRyQrUwT>*=1T+Y9}lfLuxNf`;<`-Sv8bY|Vs7BRJIa+} zqp`{yt@LCTL1odo*L-Hs8?dxHIULS_K@eB&S#(x(pf_kkipp)Oy0vxRNI9`=JWPfW zuJni0H}0~r&0BxYTYDzt#FWhp6eDY;JG-_ZEjIgT6>t_xB-c2bzQns3$E&k?Z+~AO z#Ox1(#x$~H{Y3{^l)%J(uGg4%?SOgg=FOW`gF~?tK3-?Zk(CsrXgUGxU<*-+--Q+R zJIMX+ynDI_H{{QzOzC*~YWme8dtaX*uW9Qg`9*zfR#}Afp@~!fA%%yNytE)OmKh8$ zHnsEQ7sZdtDWOseB& zGyn8+|9%fwNFLiy#b3A-d&LI@siL(Li^q#WN)Irg3 z4>`tc10B13miRaWEWVq*qtCKxWjEYaDJnGgjsVxW?A)r2s`yBYO(8e;@GauZqKeVp z1sY-_FVPQ!-&Bqtf_#hW`c)yEu_f5o`DZIs@nFxKVswcO#%!{=tpDl){l2Ia>zg_> z$$sWY%)i>co!kFBP+A7)#8a+YlMRqapm#$9Pqt7Q9F?PSBLKL20YcD14~MrpvwM); z%G7DadKy`J!)PCy&X_ln_#Ey?r7o!*A2>$FNfAb993N>kynp7Uc-MWQ!LW42Z;O7j{ty2Js?_4F literal 0 HcmV?d00001 diff --git a/geminidr/doc/tutorials/GHOST-DRTutorial/cal_service.rst b/geminidr/doc/tutorials/GHOST-DRTutorial/cal_service.rst index 00b48fe15..fd757edf8 100644 --- a/geminidr/doc/tutorials/GHOST-DRTutorial/cal_service.rst +++ b/geminidr/doc/tutorials/GHOST-DRTutorial/cal_service.rst @@ -23,32 +23,32 @@ System User Manual. The Configuration File ====================== -The database is configured in the DRAGONS configuration file under the +The database is configure in the DRAGONS configuration file under the ``[calibs]`` section. -In ``~/.geminidr/``, create or edit the configuration file ``rsys.cfg`` as +In ``~/.dragons/``, create or edit the configuration file ``dragonsrc`` as follow: .. code-block:: none [calibs] - standalone = True - database_dir = /ghost_tutorial/playground + databases = ${path_to_my_data}/ghost_tutorial/playground/cal_manager.db get store -The ``[calibs]`` section tells the system where to put the calibration database. -The database file name is ``cal_manager.db``. In -DRAGONS 3.0, you can only set the path to the database not the name. As for -the ``standalone`` setting, just make sure it is set to True. This setting -is used at Gemini by internal systems. +The ``[calibs]`` section tells the system where to put the calibration database +and how to name it. Here we use ``cal_manager.db`` to match what was used in +the pre-v3.1 version of DRAGONS, but you can now set the name of the database +to what suits your needs and preferences. -The database will keep track of the processed calibrations that we are going to -send to it. The database will be used -by DRAGONS to automatically *get* matching calibrations. You will have to -``caldb add`` (command line) or ``caldb.add_cal()`` (API) -your calibration products yourself however. +That database will keep track of the processed calibrations that we are going to +send to it. With the "get" and "store" options, the database will be used +by DRAGONS to automatically *get* matching calibrations and to automatically +*store* master calibrations that you produce. If you remove the "store" option +you will have to ``caldb add`` (command line) or ``caldb.add_cal()`` (API) +your calibration product yourself (like what needed to be done in DRAGONS +v3.0). .. note:: The tilde (``~``) in the path above refers to your home directory. - Also, mind the dot in ``.geminidr``. + Also, mind the dot in ``.dragons``. .. _cal_service_cmdline: @@ -80,11 +80,11 @@ From the API, the calibration database is initialized as follows: .. code-block:: python - caldb = cal_service.CalibrationService() - caldb.config() + from recipe_system import cal_service + + caldb = cal_service.set_local_database() caldb.init() - cal_service.set_calservice() The calibration service is now ready to use. diff --git a/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_cmdline.rst b/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_cmdline.rst index cfbd30a35..2aeda5b54 100644 --- a/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_cmdline.rst +++ b/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_cmdline.rst @@ -49,20 +49,9 @@ Here is a copy of the table for quick reference. +-----------------+ | | Std arc biases || | +-----------------+-------------------------------------------------+ - -Special Step for Engineering Data -================================= -Because the data in this tutorial was obtained during commissioning, they -are identified as "engineering" data. DRAGONS refuses to use such data, as -a safeguard. To use the data anyway, we need to modify the program ID and -make the data look non-engineering. We run the following script to do that. - -It is unclear at this time if this will be applicable to the SV data. - -:: - - cd /ghost_tutorial - python fixprogid.py playdata/example1/*.fits ++ BPMs || bpm_20220601_ghost_blue_11_full_4amp.fits | +| || bpm_20220601_ghost_red_11_full_4amp.fits | ++-----------------+-------------------------------------------------+ Set up the Local Calibration Manager @@ -74,18 +63,6 @@ Set up the Local Calibration Manager :ref:`cal_service`, specifically the these sections: :ref:`cal_service_config` and :ref:`cal_service_cmdline`. -Save some typing -================ -Because the GHOST data reduction package is not yet fully integrated into -DRAGONS, the tools need to be told to use it. This can be done with options -to the tools. To save some typing, we can create aliases. - -:: - - alias gdataselect="dataselect --adpkg=ghost_instruments" - alias gshowd="showd --adpkg=ghost_instruments" - alias greduce="reduce --adpkg=ghost_instruments --drpkg=ghostdr" - alias gshowpars="showpars --adpkg=ghost_instruments --drpkg=ghostdr" The Files ========= @@ -106,29 +83,50 @@ large.) :: cd /ghost_tutorial/playground - gshowd ../playdata/example1/*.fits -d object,detector_x_bin,detector_y_bin,read_mode + showd ../playdata/example1/*.fits -d object,detector_x_bin,detector_y_bin,read_mode :: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- filename object detector_x_bin detector_y_bin read_mode --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- - ../playdata/example1/S20230416S0047.fits GCALflat {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230416S0049.fits ThAr {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230416S0050.fits ThAr {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230416S0051.fits ThAr {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230416S0073.fits CD -32 9927 {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230416S0079.fits XX Oph {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0011.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0012.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0013.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0014.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0015.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0036.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0037.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0038.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0039.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} - ../playdata/example1/S20230417S0040.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230416S0047.fits GCALflat {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230416S0049.fits ThAr {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230416S0050.fits ThAr {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230416S0051.fits ThAr {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230416S0073.fits CD -32 9927 {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230416S0079.fits XX Oph {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0011.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0012.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0013.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0014.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0015.fits Bias {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 2, 'red': 2, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0036.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0037.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0038.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0039.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/S20230417S0040.fits Bias {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 1, 'red': 1, 'slitv': 2} {'blue': 'slow', 'red': 'medium', 'slitv': 'standard'} + ../playdata/example1/bpm_20220601_ghost_blue_11_full_4amp.fits BPM 1 1 slow + ../playdata/example1/bpm_20220601_ghost_red_11_full_4amp.fits BPM 1 1 slow + + +Bad Pixel Mask +============== +Starting with DRAGONS v3.1, the bad pixel masks (BPMs) are now handled as +calibrations. They are downloadable from the archive instead of being +packaged with the software. They are automatically associated like any other +calibrations. This means that the user now must download the BPMs along with +the other calibrations and add the BPMs to the local calibration manager. + +See :ref:`getBPM` in :ref:`tips_and_tricks` to learn about the various ways +to get the BPMs from the archive. + +To add the static BPM included in the data package to the local calibration +database: + +:: + + caldb add ../playdata/example1/bpm*.fits Master Biases ============= @@ -154,22 +152,9 @@ Debundle the biases :: - gdataselect ../playdata/example1/*.fits --tags BIAS -o biasbundles.lis - - greduce @biasbundles.lis - -.. note:: The GHOST data reduction software currently depends on `pysynphot`. - That package issues an annoying but completely harmless `UserWarning`. - What it complains about is not used in the GHOST software. - Just ignore it until we have time to clean it up. - - :: - - /Users/klabrie/condaenvs/ghost3.0.4/lib/python3.7/site-packages/pysynphot/locations.py:46: UserWarning: PYSYN_CDBS is undefined; functionality will be SEVERELY crippled. - warnings.warn("PYSYN_CDBS is undefined; functionality will be SEVERELY " - /Users/klabrie/condaenvs/ghost3.0.4/lib/python3.7/site-packages/pysynphot/locations.py:345: UserWarning: Extinction files not found in extinction - warnings.warn('Extinction files not found in %s' % (extdir, )) + dataselect ../playdata/example1/*.fits --tags BIAS -o biasbundles.lis + reduce @biasbundles.lis Reduce the slit biases ---------------------- @@ -178,28 +163,26 @@ channels, are identical. Then can all be stacked together to reduce noise. :: - gdataselect *.fits --tags BIAS,SLIT -o biasslit.lis + dataselect *.fits --tags BIAS,SLIT -o biasslit.lis - greduce @biasslit.lis + reduce @biasslit.lis Reduce the science biases ------------------------- :: - gdataselect *.fits --tags BIAS,RED \ - --expr="detector_x_bin==2 and detector_y_bin==2" -o biasredsci.lis - gdataselect *.fits --tags BIAS,BLUE \ - --expr="detector_x_bin==2 and detector_y_bin==2" -o biasbluesci.lis + dataselect *.fits --tags BIAS,RED --expr="binning=='2x2'" -o biasredsci.lis + dataselect *.fits --tags BIAS,BLUE --expr="binning=='2x2'" -o biasbluesci.lis - greduce @biasredsci.lis - greduce @biasbluesci.lis + reduce @biasredsci.lis + reduce @biasbluesci.lis All the data was obtained with the same read modes. If this is not the case for your data and you need to select on read mode, use an expression like this one:: - --expr="detector_x_bin==2 and detector_y_bin==2 and read_mode=='slow'" + --expr="binning=='2x2' and read_mode=='slow'" Reduce the flat/arc biases @@ -211,13 +194,11 @@ set for the arcs. Fortunately, not the case here, one set for both. :: - gdataselect *.fits --tags BIAS,RED \ - --expr="detector_x_bin==1 and detector_y_bin==1" -o biasredflatarc.lis - gdataselect *.fits --tags BIAS,BLUE \ - --expr="detector_x_bin==1 and detector_y_bin==1" -o biasblueflatarc.lis + dataselect *.fits --tags BIAS,RED --expr="binning=='1x1'" -o biasredflatarc.lis + dataselect *.fits --tags BIAS,BLUE --expr="binning=='1x1'" -o biasblueflatarc.lis - greduce @biasredflatarc.lis - greduce @biasblueflatarc.lis + reduce @biasredflatarc.lis + reduce @biasblueflatarc.lis Master biases to Calibration Database @@ -230,17 +211,18 @@ subdirectory representing the type of calibrations. For the biases, This is a safe copy of the calibrations that will be needed later allowing us the freedom to clean the work directory between steps, which is -particularly helpful in the case of GHOST. +particularly helpful in the case of GHOST. Because the database +was given the "store" option in the ``dragonsrc`` file, the processed biases +will be automatically added to the database at the end of the recipe and no +additional commands are required. -Since we will indeed clean up the work directory, we will add the safe files in -``calibrations`` to the calibration manager database instead of the files in the -work directory. A reminder that the files are not added to the database, only -the information about them and their location on disk; if you delete the file -on disk it is gone even if information about it remains in the database. -:: +.. note:: If you wish to inspect the processed calibrations before adding them + to the calibration database, remove the "store" option attached to the + database in the ``dragonsrc`` configuration file. You will then have to + add the calibrations manually following your inspection, eg. - caldb add calibrations/processed_bias/*.fits + ``caldb add calibrations/processed_bias/*.fits`` Clean up -------- @@ -262,43 +244,44 @@ Debundle Flats :: - gdataselect ../playdata/example1/*.fits --tags FLAT -o flatbundles.lis + dataselect ../playdata/example1/*.fits --tags FLAT -o flatbundles.lis - greduce @flatbundles.lis + reduce @flatbundles.lis Reduce the Slit-flat -------------------- The slit-flat is required to reduce the red and blue channel flats, so it is -important to reduce it first and add it to the calibration database. +important to reduce it first. Again, you will need to manually add it to the +calibration database if your ``dragonsrc`` file is not set up to auto-store +the calibrations as they are created. :: - gdataselect *.fits --tags SLITFLAT -o slitflat.lis + dataselect *.fits --tags SLITFLAT -o slitflat.lis - greduce @slitflat.lis - caldb add calibrations/processed_slitflat/*.fits + reduce @slitflat.lis .. note:: You will see this message in the logs:: ERROR - Inputs have different numbers of SCI extensions. You can safely ignore it. It is expected and the wording is misleading. - This is not an real error. + This is not a real error. Reduce the Flats ---------------- The flats have a 1x1 binning and must match the read mode of the science data. If the science data is binned, the software will bin the 1x1 flats -to match. +to match. Reducing the flats takes a little time because of the step to +trace each of the echelle orders. :: - gdataselect *.fits --tags FLAT,RED -o flatred.lis - gdataselect *.fits --tags FLAT,BLUE -o flatblue.lis + dataselect *.fits --tags FLAT,RED -o flatred.lis + dataselect *.fits --tags FLAT,BLUE -o flatblue.lis - greduce @flatred.lis - greduce @flatblue.lis - caldb add calibrations/processed_flat/*.fits + reduce @flatred.lis + reduce @flatblue.lis Clean up -------- @@ -312,45 +295,51 @@ the work directory Arcs ==== The arcs have a 1x1 binning, the read mode does not matter. It does save -processing if they are of the same read mode as the flats as different biases -for the specific read mode are in such case not needed. If the science data is binned, -the software will bin the 1x1 arcs to match. +processing if they are of the same read mode as the flats as otherwise they will need +their own flats with a matching read mode as well as their own biases. If the science +data is binned, the software will bin the 1x1 arcs to match. + +A minimum of three arc exposures are required in each arm to eliminate cosmic rays +(which can look very similar to arc lines). At the time these data were taken, +each arc bundle only contained a single exposure in each of the red and blue arms, +so three separate files are needed. Now each GHOST arc observation (and hence each +raw bundle) contains three exposures in each arm so only one file should be needed. Debundle the Arcs ----------------- :: - gdataselect ../playdata/example1/*.fits --tags ARC -o arcbundles.lis + dataselect ../playdata/example1/*.fits --tags ARC -o arcbundles.lis - greduce @arcbundles.lis + reduce @arcbundles.lis -Reduce the slit-view data -------------------------- -We have 3 slit images for the arc but we really just need one. We grab -the first one. +Reduce the slit-viewer data +--------------------------- +We have 3 slit images for the arc because there are 3 arc bundles but we really +just need one because the illumination of the slit by the arc lamp is stable. +We grab the first one. :: - gdataselect *.fits --tags ARC,SLIT | head -n 1 > arcslit.lis + dataselect *.fits --tags ARC,SLIT | head -n 1 > arcslit.lis - greduce @arcslit.lis - caldb add calibrations/processed_slit/*.fits + reduce @arcslit.lis Reduce the arcs --------------- :: - gdataselect *.fits --tags ARC,RED -o arcred.lis - gdataselect *.fits --tags ARC,BLUE -o arcblue.lis + dataselect *.fits --tags ARC,RED -o arcred.lis + dataselect *.fits --tags ARC,BLUE -o arcblue.lis - greduce @arcred.lis - greduce @arcblue.lis + reduce @arcred.lis + reduce @arcblue.lis caldb add calibrations/processed_arc/*.fits .. note:: If you want to save a plot of the wavelength fits, - add ``-p fitWavelength:plot1d=True`` to the ``greduce`` call. + add ``-p fitWavelength:plot1d=True`` to the ``reduce`` call. A PDF will be created. Clean up @@ -373,22 +362,45 @@ Debundle the Standard :: - gdataselect ../playdata/example1/*.fits --expr="object=='CD -32 9927'" -o stdbundles.lis + dataselect ../playdata/example1/*.fits --expr="object=='CD -32 9927'" -o stdbundles.lis - greduce @stdbundles.lis + reduce @stdbundles.lis -Reduce the slit-view data -------------------------- +Reduce the slit-viewer data +--------------------------- Since we have cleaned up all the intermediate files as we went along, we are able to just select on the tag SLIT. If we had not cleaned up, we would need to use the object name like we did above. :: - gdataselect *.fits --tags SLIT -o stdslit.lis - - greduce @stdslit.lis - caldb add calibrations/processed_slit/S20230416S0073_slit_*.fits + dataselect *.fits --tags SLIT -o stdslit.lis + + reduce @stdslit.lis + +Note that four separate files are produced by this reduction and stored in +the database. This is because the bundle contains four separate exposures in +the red and blue spectrographs that cover different periods of time, and each +needs its own slit-viewer image produced from the slit-viewer camera data taken +at the same time. In general, as many calibration files will be produced as +there are spectrograph exposures, unless the red and blue exposure times are +the same, in which case the first exposures in each cover exactly the same +period of time and so a single reduced slit-viewer image can be used for both. + +During the reduction of on-sky slit-viewer images, a plot is produced of the +total flux (summed from both the red and blue slit images) as a function of +time, with the durations of each of the spectrograph exposures also plotted. +By default, this is written to the working directory as a pdf file with a +like ``S20230416S0073_slit_slitflux.pdf``. You can change the format with +the ``-p plotSlitFlux:format=png`` or ``-p plotSlitFlux:format=screen`` +where the latter option will display a plot on the screen without saving it +to disk. A scatter of ~10% appears to be fairly typical, and the lower count +rate in the early exposures can be traced to poorer seeing conditions, which +are reported during the reduction of the spectrograph images. + +.. image:: _graphics/slitflux.png + :width: 325 + :alt: plot of the slit-viewer flux variations Reduce the standard star ------------------------ @@ -396,132 +408,152 @@ Since we have cleaned up all the intermediate files as we went along, we are able to just select on the tag RED and BLUE. If we had not cleaned up, we would need to use the object name like we did above for the bundle. -This step takes a while. +This step takes a while, with the extraction of each spectrum needing +about 3 minutes. As each image is processed, the estimated seeing is +reported, together with the fraction of light collected by the IFU. Here +the seeing improves significantly from the first to the second red exposure, +which explains the increase in the counts from the slit-viewer camera seen +in the previous plot. :: - gdataselect *.fits --tags RED -o stdred.lis - gdataselect *.fits --tags BLUE -o stdblue.lis + dataselect *.fits --tags RED -o stdred.lis + dataselect *.fits --tags BLUE -o stdblue.lis - greduce -r reduceStandard @stdred.lis - greduce -r reduceStandard @stdblue.lis + reduce -r reduceStandard @stdred.lis -p scaleCountsToReference:tolerance=1 + reduce -r reduceStandard @stdblue.lis -p scaleCountsToReference:tolerance=1 The reduced spectrophotometric standard observations are the ``_standard`` files. -The ``_flatBPMApplied`` files are the last 2D images of the spectra -before it gets extracted to 1D. They are saved just in case you want to +The ``_arraysTiled`` files are the last 2D images of the spectra +before they gets extracted to 1D. They are saved just in case you want to inspect them. They are not used for further reduction. For the wavelength calibration, the pipeline will try to find an arc taken before the observation and one taken after. If it finds two, it will use them -both, however, one is enough. This is what happens here: the software finds +both and interpolate between them to obtain a wavelength solution, but one is +enough provided it is taken close enough in time (the stability of the +instrument has not yet been sufficiently well quantified to say what "close +enough" means, but here the arcs are taken on the same night so that is +definitely OK). This is what happens here: the software finds a "before" arc, but no "after" arc. So, do not be alarmed by the messages saying that it failed to find an arc, it's okay, it got one, it's enough. -This standard observation has three red arm exposures. They are not stacked. -They possibly should but the software was delivered without a final stacking -capability. So for now, no stacking. In this case, one exposure is bright -enough, and there's no problem at all using only one of the red exposures. -Pick one, anyone if the conditions were stable, they should all look the same. -If you suspect that the conditions were highly variable, you can inspect the -``_flatBPMApplied`` files and see which one is the brightest and use that one. -To display such a file, launch ``ds9`` and type:: - - greduce -r display -p zscale=False S20230416S0073_red001_flatBPMApplied.fits - -Since here it doesn't matter which file we use, we pick ``red001``. +This standard observation has three red arm exposures whose counts can be +scaled to match the level of the first frame and then stacked. By default, +the ``scaleCountsToReference`` primitive only scales by the exposure time +(which is the same for all these exposures), so no scaling will occur. +This choice of default is to prevent erroneous scaling factors being calculated +when the signal-to-noise ratio in the data is low, but that is not the case +here so we can trust the ratios calculated by the software. Setting a tolerance +of 1 indicates that the calculated ratios should be used whatever they are, +whereas a value of, for example, 0.1 means that the ratio should only be used +if it is within 10% of that expected from the relative exposure times. If the +calculated ratio is outside this range, then the relative exposure times will +be used to scale the data. In this case the second and third exposures are +brighter by about 15%, which is consistent with the improvement in the image +quality reported during the reduction. Since the first exposure is the +reference to which the others are scaled, the flux scale produced from this +red-arm calibration will be based on the poorer image quality of that +exposure. + +There is only a single blue exposure so there aren't multiple frames to stack +and the ``tolerance`` parameter is irrelevant. It is included in this tutorial +to make clear that it can be specified for both arms. Clean up -------- -Because the processed standard files are not recognized as such they are **NOT** -copied to the ``calibrations`` directory. So you have to be very careful here -with your clean up. - -You want to keep the `_standard` files for sure, those are the -calibration files that will be used to reduce the standard. You might want to -keep the `_flatBPMApplied` files for visualization purposes. - -The selective clean-up looks like this. +With the calibrations safely in the ``calibrations`` directory, we can clean +the work directory :: - rm *_slit.fits - rm *_red???.fits - rm *_blue???.fits + rm *.fits Science Frames ============== -As explained above, unlike for GMOS, the standards are not automatically -recognized as such. They are just like any other science observation. -Therefore to select the science, we will need to use the object's name. Debundle the Science Frames --------------------------- :: - gdataselect ../playdata/example1/*.fits --expr="object=='XX Oph'" -o scibundles.lis + dataselect ../playdata/example1/*.fits --expr="object=='XX Oph'" -o scibundles.lis - greduce @scibundles.lis + reduce @scibundles.lis -Reduce the slit-view data -------------------------- +Reduce the slit-viewer data +--------------------------- :: - gdataselect *.fits --tags SLIT -o scislit.lis + dataselect *.fits --tags SLIT -o scislit.lis + + reduce @scislit.lis - greduce @scislit.lis - caldb add calibrations/processed_slit/S20230416S0079*.fits +Again, there will be four "processed slit" calibration files produced, one for +each spectrograph exposure in the bundle. Reduce the Science Frames ------------------------- -The processed standards are not associated automatically for GHOST. -They need to be specified on the command line with the ``-p`` flag. - -The standard we are using is "CD -32 9927". This is one of the baseline -standards Gemini uses. The flux profile of that star is available in -DRAGONS and the name will be recognized and the file automatically retrieved. - -If you were to use a spectrophotometric standard not on the Gemini list, you -would need to provide that flux standard file with the -``-p specphot_file=path/name_of_file``. The accepted format are the "IRAF -format" and the HST calspec format. +The spectrophotometric standard used in this tutorial is in the Gemini list and +so the file containging the table of spectrophotometric data will be found +automatically. If you were to use a spectrophotometric standard not on the +Gemini list, you would need to provide that flux standard file with the +``-p filename=path/name_of_file``. The accepted format are the "IRAF +format" and any FITS table which properly describes its columns (files in the HST +calspec and ESO X-Shooter libraries fulfil this criterion). .. note:: Possible customizations. * The sky subtraction can be turned off with ``-p extractProfile:sky_subtract=False`` - if it is found to add noise. + if it is found to add noise (of course, the sky emission lines will still be + present in your data). * If you expected IFU-2 to be on-sky but there's an accidental source, tell the software that there is a source and it isn't sky with ``-p extractProfile:ifu2=object``. * If you do not want the barycentric correction, turn is off with - ``-p barycentricCorrect:correction_factor=1``. + ``-p barycentricCorrect:velocity=0``. + * If you don't have a spectrophotometric standard and are happy to have your output + spectra in units of electrons, make sure to add + ``-p responseCorrect:do_cal=skip`` + +:: -.. warning:: The reduction of the red channel is very slow. Launch - and go get a coffee or something. Make sure that you got the name for the - standard star file correct or it will crash after having done most of the - work and you will have to start again. Not fun. The blue channel in this - example reduces rather quickly. + dataselect *.fits --tags RED --expr="object=='XX Oph'" -o scired.lis + dataselect *.fits --tags BLUE --expr="object=='XX Oph'" -o sciblue.lis + + reduce @scired.lis + reduce @sciblue.lis + +Note that during the ``extractProfile`` step (which takes a few minutes for +each expoure), a warning appears that "There are saturated pixels that have +not been flagged as cosmic rays" with pixel coordinates. The pixel coordinates +are different for each exposure and in all cases only the first instance is +reported so the data could be severely affected. You can investigate further +by displaying the DQ plane of the ``_arraysTiled`` frame, e.g., :: - gdataselect *.fits --tags RED --expr="object=='XX Oph'" -o scired.lis - gdataselect *.fits --tags BLUE --expr="object=='XX Oph'" -o sciblue.lis + reduce -r display S20230416S0079_red001_arraysTiled.fits -p extname=DQ - greduce @scired.lis -p standard=S20230416S0073_red001_standard.fits - greduce @sciblue.lis -p standard=S20230416S0073_blue001_standard.fits +(make sure you have ``ds9`` running first) and saturated pixels will appear +as white (they have the value 4, which is the maximum value in the DQ plane +at this point in the reduction). + +:: dgsplot S20230416S0079_red001_dragons.fits 1 --bokeh -The final products are the ``_dragons`` files. In those files, all the orders -have been stitched together with the wavelength on a log-linear scale, +The final product from each arm are the ``_dragons`` files. In those files, +all the orders have been stitched together with the wavelength on a log-linear scale, calibrated to in-air wavelengths and corrected for barycentric motion (unless -that correction is turned off.) +that correction is turned off.) Individual exposures will also be stacked after +being scaled. -The first extension (the "1" in the call to ``dgsplot`` above) is the spectrum. -The second extension is the spectrum of the sky. This is for an observation +The first "aperture" (the "1" in the call to ``dgsplot`` above) is the spectrum. +The second aperture is the spectrum of the sky. This is for an observation with one object and sky subtraction turned on (default). Here's the list of possible configurations: @@ -537,34 +569,71 @@ possible configurations: It is possible to write the spectra to a text file with ``write1DSpectrum``, for example:: - greduce -r write1DSpectra S20230416S0079_red001_dragons.fits + reduce -r write1DSpectra S20230416S0079_red001_dragons.fits The primitive outputs in various formats offered by ``astropy.Table``. To see the list, use |showpars|. :: - gshowpars S20230416S0079_red001_dragons.fits write1DSpectra + showpars S20230416S0079_red001_dragons.fits write1DSpectra The ``_dragons`` files are probably what most people will want to use for making their measurements. -The files ``_calibrated`` are the reduced spectra *before* the stitching -the orders and the format of the file is more complex and somewhat less -accessible. But if you need it, you have it. The flux pixels are in a -3D array with the first axis of size 2, one for target, one for sky, then a -second axis being the wavelength direction, and finally a third axis of -30-something orders. The ``WAVL`` extension contains the wavelength at each -of the pixels in the wavelength-order array. +The files ``_calibrated`` are the reduced spectra *before* stitching +the orders and stacking the files and the format of the file is more complex +and somewhat less accessible. But if you need it, you have it. These files +can also be plotted with the ``dgsplot`` command, and each order will appear +in a different color. The flux pixels are in a +2D array for each aperture with the first axis being the wavelength direction +and the second axis going through each of the 30-something orders. The +wavelengths of each pixel are stored as an image in the ``AWAV`` extension +(following the FITS naming convention for in-air wavelengths). + +The software automatically stacks the three red exposures during the +``combineOrders`` step after scaling them (there is no ``tolerance`` +parameter here) so there is only one ``_dragons`` file for each arm, +but there are 3 ``_calibrated`` files for the red arm and 1 for the blue arm, +one for each exposure. + +Alternative data products +------------------------- -While we have three red exposures, the software does not stack them. The -software was delivered without stacking. If the sky conditions (cloud, seeing) -were stable and the individual spectra do not have large numbers of cosmic -rays, it is possible to use DRAGONS' ``stackFrames`` with no rejection, no -scaling to create a stacked spectrum. Please use your best judgement. +If you don't wish to stack the individual order-combined spectra, you can +adjust the parameters to ``combineOrders`` to get the result you desire. +You can turn off stacking with ``-p combineOrders:stacking_mode=none`` +nd you will obtain one ``_dragons`` +file for each exposure. These can be combined later if you wish by running +the following command: :: - greduce -r stackFrames -p reject_method=none *red???_dragons.fits + reduce -r combineOrders S20230416S0079_red00?_calibrated.fits + +It is also possible to stack the individual spectra without scaling them with +``-p combineOrders:stacking_mode=unscaled``. + +The ``combineOrders`` primitive will also combine spectra from *both* arms +so you can obtain a single spectrum covering the full wavelength range of +GHOST. Simply send the ``_calibrated`` files from both arms to the primitive. +By default, the output will have the ``_dragons`` suffix and so may overwrite +an existing file (depending on the first file in the input list) and so you +may wish to specify an alternative output suffix. + +:: + + reduce -r combineOrders S20230416S0079*calibrated.fits -p suffix=_full + +If you wish to turn your reduced spectrum into a ``specutils.Spectrum1D`` +object, you can do this within a python session as follows: + +:: + import astrodata, gemini_instruments + from gempy.library.spectral import Spek1D + ad = astrodata.open("S20230416S0079_blue001_full.fits") + spectrum1d = Spek1D(ad[0]).asSpectrum1D() +Note that you need to specify ``ad[0]`` to obtain the first aperture +(the target). \ No newline at end of file diff --git a/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_dataset.rst b/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_dataset.rst index 72ec8b716..239b5663e 100644 --- a/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_dataset.rst +++ b/geminidr/doc/tutorials/GHOST-DRTutorial/ex1_ghost_stdonetarget_dataset.rst @@ -25,8 +25,14 @@ The calibrations we use for this example are: * Flats. The flats must match the read mode of the science. The binning must be 1x1. They will be binned in software to match the science. The matches must cover both the red and blue arms. -* Arcs. The binning for the arcs must be 1x1. The read mode does not matter. +* Arcs. The binning for the arcs must be 1x1. The read mode does not matter, + but they require their own biases and flats and it is therefore simpler if + they have the same read mode as the science data. * Spectrophotometric standard. The binning must match the science. +* BPMs. The bad pixel masks are found in the Gemini Science Archive + instead of being packaged with the software. They are associated like the + other calibrations. Separate BPMs are required for the blue and red arms. + There is no BPM for the slit-viewer camera. Here is the breakdown of the files. All the files are included in the tutorial data package. They can also be downloaded from the Gemini Observatory Archive (GOA). @@ -38,7 +44,7 @@ package. They can also be downloaded from the Gemini Observatory Archive (GOA). +-----------------+-------------------------------------------------+ | Science Flats || S20230416S0047 (1x1; blue:slow; red:medium) | +-----------------+-------------------------------------------------+ -| Science Arcs || S20230416S0049-51 (1x1) | +| Science Arcs || S20230416S0049-51 (1x1; blue:slow; red:medium) | +-----------------+-------------------------------------------------+ | Flats Biases || S20230417S0036-40 (1x1; blue:slow; red:medium) | +-----------------+ | @@ -57,7 +63,9 @@ package. They can also be downloaded from the Gemini Observatory Archive (GOA). +-----------------+ | | Std arc biases || | +-----------------+-------------------------------------------------+ - ++ BPMs || bpm_20220601_ghost_blue_11_full_4amp.fits | +| || bpm_20220601_ghost_red_11_full_4amp.fits | ++-----------------+-------------------------------------------------+ diff --git a/geminidr/doc/tutorials/GHOST-DRTutorial/index.rst b/geminidr/doc/tutorials/GHOST-DRTutorial/index.rst index 22b305735..83edd5546 100644 --- a/geminidr/doc/tutorials/GHOST-DRTutorial/index.rst +++ b/geminidr/doc/tutorials/GHOST-DRTutorial/index.rst @@ -17,7 +17,6 @@ Tutorial Series - GHOST Data Reduction with DRAGONS :caption: Contents: overview - sv_installation datasets cal_service ex1_ghost_stdonetarget