Features
ALMA User-Defined Imaging (AUDI)
User-defined data cubes can be produced for proprietary ALMA data (by the PI or delegee) and non-proprietary data. Users can specify the spectral width and channel width in frequency or velocity units to produce a data cube that may better fit their science needs. Continuum imaging is also produced as part of this process in case it is needed for the interpretation of the spectral line data. The cube imaging process makes use of the ability to restore a calibrated measurement set and the ALMA imaging pipeline. This feature has the same limitations as the calibrated measurement set download in that it can only operate on pipeline-reduced Cycle 5 data and beyond.
The imaging pipeline will operate using the current version of the ALMA imaging pipeline to take advantage of the most recent improvements to continuum finding, automasking, and self-calibration. Thus, imaging products may be different and possibly superior to those delivered with an ALMA project by default.
The cube imaging process functions as follows:
- The selected dataset is restored to a calibrated state using the same process for download of a calibrated measurement set
- Imported into the ALMA imaging pipeline
- Custom cube(s) and continuum images are produced using a special imaging pipeline recipe which includes automated self-calibration
How to run ALMA User-Defined Imaging (AUDI):
- Start at https://data.nrao.edu
- Search for some data (e.g., project code, target name, position/radius)
- Click the ‘+’ to see more detail about the project
- Click the blue ‘Download Restored MS’ button for the desired data
- Fill out the dialog box for a frequency-based or velocity-based cube
- A valid setting for the frequency-based cube is filled by default for the selected spectral window
See the tutorial video for AUDI for more detail.
Important caveats: See Known Issues below
ALMA Calibrated Measurement Set Download
PIs and their delegees are able to download calibrated measurement sets for their proprietary ALMA data once ingested into the ALMA archive and mirrored in the NRAO archive. Archive users can also download calibrated measurement sets for any non-proprietary data. Calibrated measurement sets can currently only be downloaded for datasets that were fully calibrated by the ALMA pipeline; manual reductions are not supported. Only Cycle 5 and later data from the 12m array and ACA can be restored using this process due to the way the data were archived in earlier cycles.
The calibrated measurement sets are generated on-the-fly when requested. As such, there will be a time lag between when a request is submitted and when the data are ready to be downloaded. The time for the calibrated data to be ready for download depends strongly on the size of the dataset. The restoration process reapplies the flagging done by the pipeline and applies the calibration tables produced and archived by the ALMA pipeline. The manual reductions follow a different workflow and are not compatible with being restored to a calibrated state in the same manner. The delivered measurement sets will include the calibrators and science targets and will not have self-calibration or continuum subtraction applied.
Time until dataset ready for different raw data sizes:
10GB – ~1 hr
100 GB – ~10 hr
1 TB – ~60 hr
Known Issues
Archive
ALMA Cycle 0 data are not present in the NRAO archive interface, due to the way it was archived. If you require Cycle 0 data, please visit the ALMA Archive.
For ALMA data products other than the ASDM files and calibrated measurement sets, users must still visit the ALMA archive at the present time. We intend to make image products available in the NRAO archive in the future.
Some ALMA projects may not be properly displayed in the NRAO archive due to problems with incorporating them into the index. If a user spots an inconsistency in the NRAO archive (e.g., missing execution blocks [EBs], missing MOUSes/SBNames, or any other issue), they are encouraged to submit a ticket to the NRAO helpdesk under the ‘Data Products’ topic.
ALMA Calibrated Measurement Sets
The ALMA Restores are only available for 12m and ACA data that were pipeline calibrated in Cycle 5 and beyond (select Cycle 4 datasets processed in Cycle 5 may also be available). Data that were manually calibrated cannot be restored by this process and require a full download and execution of the calibration script. Single-dish (Total Power) datasets are also not supported for restoration at the present time.
Certain pipeline-processed projects that were sent for manual imaging can be restored via this process, but some additional flags may have been added during manual imaging and these will not be applied to the restored dataset. These flags may not be of significant consequence, but users should check if flagdata commands are present in the *scriptForImagingPrep.py included with the auxiliary package from the ALMA data archive.
What if my ALMA Cycle 5 restore is failing?
When attempting to restore ALMA datasets from Cycle 5 (typically those with a project ID like 2017.1.XXXX.S), the restores may fail when using a CASA version different from the one it was originally calibrated with. This is because the data are imported to a measurement set differently in later CASA version versus the CASA version 5.1.1-5 that was used for processing. If a failure is encountered, the restore should be resubmitted, but with CASA version 5.1.1-5 selected from the drop down box. This issue should not be frequently encountered because as of late 2024, the restore tool automatically selects the version of CASA the data were originally processed with (if possible), but failures could still be seen if the CASA version was changed manually for the restore.
ALMA User-Defined Imaging
ALMA Cube Imaging runs a modified recipe of the ALMA imaging pipeline to produce cubes with custom channel width, overall spectral width, and velocity or frequency-defined channels. Cleaning uses the same automasking procedure as the ALMA Imaging Pipeline, and low signal-to-noise emission may not be fully encompassed within a clean mask. Imaging results may differ from the archived products because ALMA Cube Imaging uses the most recent imaging pipeline and incorporates the recent improvements to functions such as continuum finding, automasking, and self-calibration.
There are limitations to restorations (see Known Issues for ALMA Calibrated Measurement Sets above). ALMA Cycle 5 datasets that cannot be restored in the latest CASA version also cannot be imaged. This is because imaging requires the latest CASA version and it is not currently possible to restore with one version of CASA and image with another automatically. However, please submit your request and we will be able to workaround this issue.
At present, the archive does not allow ingestion of more than one cube with the same combination of MOUS and source. These jobs will fail, even if a different spectral window is requested for imaging compared to the initial request.
Why Don’t I Get the Angular Resolution I Requested?
The modified ALMA imaging pipeline recipe uses a task called hifa_imageprecheck to set the appropriate tclean settings (robust and uvtaper) to get an angular resolution as close as possible to the requested resolution. Based on the data being imaged, there will be a maximum (smallest beam) and minimum (largest beam) angular resolution allowed by the pipeline. So requests for a beam smaller or larger than the limits will simply be set to the appropriate limit for a large or small beam.