The compression factor of on board EIS data may vary depending on a few factors such as the observing target (QS, AR, CH, etc), slit/slot selection, exposures, etc.

The purpose of this study is trying to investigate how different compression schemes effect EIS data volume on board (MDP) and work out a better estimation of compression factor for compression scheme (eg. DPCM, JPEG98, JPEG95, etc.)

The approach:#

1. to get actual data volume from MDP status information: the inclined curve means data packets from EIS on MDP increasing, the vertical curve means data packets dumped to ground station. So in general, knowing a raster's start and end time one can fitstly calculate actual data volume in that duration, and then compare it with the designed data volume in raster's definition, to get data compression factor.

2. to get related information from planning database/eis catalogue/fits header, for example: raster ID, compression scheme, designed data volume, SCI_OBJ, TARGET, slit/slot, exposures, etc.

3. prepare plots based on various factor combinations: compression factor vs. slit/slot, factor vs. target, factor vs. exposures


Some preliminary results (plots):#


The investigation here is for date sets obtained mainly between 2007-Sep-15 and 2007-Dec-15, as EIS operations were most efficient and stable over this duration. The other data sets inluded here are: 2008-Mar (for JPEG85), 2006-Dec (for JPEG75), 2007-Jan to 2007-Apr, and 2008-Feb.




The average values of compression factor for each scheme are listed in following Sortable Table (although only a single number can't show the compression factor varation over a large dataset).
Scheme Total QS AR CH 1" 2" 40" 266" 10s 15s 20s 30s 60S
DPCM 2.522.802.802.703.02.963.632.493.083.012.52.62.42
DPCM QS3.142.942.912.522.32
DPCM AR2.963.02.352.52.14
DPCM CH 3.052.522.882.47
JPEG982.562.672.463.442.802.362.56 3.212.54 2.4
JPEG98 QS3.322.60 2.43
JPEG98 AR3.172.42 2.49
JPEG98 CH3.6
JPEG956.256.126.38 6.236.3 5.86 4.60
JPEG95 AR5.82 4.08
JPEG908.117.398.68.44.85 5.048.34 5.04
JPEG90 QS 3.87
JPEG90 AR 3.81
JPEG855.565.584.926.21 5.56
JPEG7511.39 11.3911.39

However, the number shown above has difference from previous work done by others, eg, Hara'san result:

> 40" SLOT
> DPCM      2.36
> JPEG98    2.70
> JPEG95    3.47
> JPEG92    4.22
> JPEG90    4.63
> JPEG85    5.74
> JPEG75    7.63
> JPEG65    9.43
> JPEG50    12.0
> For 10s exposure time. 


This work is just a start, the method and results shown here need to check again, and compare with other numbers. !As I mentioned above, these are only preliminary results on EIS compression factor. Still there are more need to be done, so any contributions, comments and suggestions are very helpful!


As for target selection, in this case, I uses value of SCI_OBJ keyword instead of TARGET keyword as the later is only meaningful at (study/raster) design stage, the actual target is chosen during the EIS planning.

However, sometimes there is no value for SCI_OBJ in EIS fits header, and sometimes there are two many values (see SCI_OBJ_Example). Lots of EIS dataset are thrwon away as hardly to decide to which catagroy the SCI_OBJ belongs to, such as QS,AR or CH.

Compensation has been applied for those paused (and aborted) rasters. This is implemented by comparing NEXP and RAST_REQ and timing the ratio of two values.
There is a structure array to store all related information for the EIS data investigated here. The array has element with the following format:
compFactor={compression_factor, $
          study_ACR     :'', $  ;string
          study_id      :'', $  ;string
          rast_ACR      :'', $  ;string
          rast_id       :'', $  ;string
          ll_ACR        :'',$   ;string
          ll_id         :'',$   ;string
          start_time    :'', $  ;string
          end_time      :'', $  ;string
          fitsname      :'',$   ;string
          target        :'',$   ;string
          sci_obj       :'',$   ;string
          slit          :'',$   ;string
          def_volume    :0LL,$  ;long64 int, unit: bits
          mdp_volume    :0.0,$  ;float, unit: kbits
          comp_scheme   :0,$    ;int
          nexp          :0,$    ;int
          rast_req      :0,$    ;int
          exposures     :fltarr(8) $    ;float, unit: sec
        }

I attached an IDL sav file here. You may download and play it, for example, I use:

if (str1[i].SCI_OBJ eq 'QS') && (str1[i].COMP_SCHEME eq 1) && (str1[i].MDP_VOLUME gt 0.) then ind[i]=1

to extract records associated with 'QS' SCI_OBJ and using DPCM compression scheme.


Add new attachment

Only authorized users are allowed to upload new attachments.
« This page (revision-43) was last changed on 07-Dec-2016 14:14 by JianSun