.. _installation: Downloading and installing **LINC** =================================== You can choose between the manual installation of all software required by **LINC** or the use of pre-compiled Docker images. The latter option is highly recommended. Docker installation ------------------- To make use of the included container capabilities of **LINC** you need to install Docker on your system. Instructions on how you can install Docker on your system can be found here: * `CentOS`_ * `Debian`_ * `Fedora`_ * `Ubuntu`_ If installing Docker on your system is not possible you can also use `Singularity`_ (version 3.1 or later) or `udocker`_. For running **LINC** using Singularity or uDocker follow the instructions on the page :doc:`how to run the pipeline`. .. note:: Due to a bug Debian/Ubuntu packages of Singularity version 3.9.9 will not run with **LINC**. Getting **LINC** workflow description ------------------------------------- Next you will need to download the **LINC** workflow description files written in the `Common Workflow Language`_ (CWL). They can be retrieved with:: $ git clone https://git.astron.nl/RD/LINC.git To start the pipeline you also need to install an interpreter for the CWL description files. The most common ones are `cwltool`_ and `toil`_:: $ python3 -m pip install cwltool cwl-runner toil[cwl] It is recommended to install `cwltool`_ or `toil`_ in a `conda`_ or `virtual environment`_. You do **not** need to run **LINC** inside a container environment. `cwltool`_ or `toil`_ will call the container for you if necessary. Please make sure that your `cwltool`_ version is at least 3.1.20220406080846 or later. Once you have installed `cwltool`_ or `toil`_ locally on your system, LINC will automatically pull the right Docker/Singularity image for you once you start the pipeline for the first time. .. note:: The **LINC** Docker/Singularity image also provides an installation of `cwltool`_ and `toil`_ for your convenience if you want to avoid installing `cwltool`_ or `toil`_ on your system. You will find more details how to run **LINC** within an image on the page :doc:`how to run the pipeline`. Manual installation ------------------- .. note:: Skip this section if you want to run **LINC** using software containers. For installing **LINC** a Debian-based operating system is recommended (e.g. Ubuntu 22.04). In order to compile the required packages for running **LINC** you may need to install the following packages:: $ apt-get update && \ $ apt-get install -y bison build-essential casacore-data casacore-dev cmake flex gfortran git \ libblas-dev libboost-date-time-dev libboost-filesystem-dev libboost-numpy-dev \ libboost-program-options-dev libboost-python-dev libboost-system-dev libboost-test-dev \ libcfitsio-dev libfftw3-dev libgsl-dev libgtkmm-3.0-dev libhdf5-serial-dev liblapack-dev \ liblua5.3-dev libncurses5-dev libpng-dev libpython3-dev pkg-config python3 python3-casacore \ python3-numpy python3-pip wcslib-dev wget In order to run **LINC** you need to get the following packages onto your system:: $ apt-get install -y casacore-tools gdb libatkmm-1.6-1v5 libblas3 libboost-filesystem1.74.0 libboost-program-options1.74.0 libboost-python1.74.0 libcairomm-1.0-1v5 \ libcasa-casa6 libcasa-fits6 libcasa-measures6 libcasa-ms6 libcasa-python3-6 libcasa-scimath6 libcasa-tables6 libcfitsio9 \ libfftw3-double3 libfftw3-single3 libgfortran5 libglib2.0-0 libglibmm-2.4-1v5 libgomp1 libgsl27 \ libgtkmm-3.0-1v5 libhdf5-103-1 libhdf5-cpp-103-1 liblapack3 liblua5.3-0 libpangomm-1.4-1v5 libpng16-16 libpython3.10 \ libsigc++-2.0-0v5 libstdc++6 nodejs python3 python3-casacore python3-distutils rsync wget Then you need to install the following software: * `LofarStMan`_ * `Dysco`_ (v1.3 or newer) * `IDG`_ (v1.2.0 or newer) * `aoflagger`_ (v3.4 or newer) * `LOFARBeam`_ (v4.1.1 or newer) * `EveryBeam`_ (v0.5.3 or newer) * `Sagecal`_ (v0.8.1 or newer) * `DP3`_ (v6.0 or newer) * `WSClean`_ (v3.4 or newer) * `RMextract`_ (v0.4.4 or newer) * `LoSoTo`_ (v2.4 or newer) * `LSMTool`_ (v1.5.1 or newer) Additionally, extra casacore tables are needed:: $ wget -q -O /WSRT_Measures.ztar ftp://ftp.astron.nl/outgoing/Measures/WSRT_Measures.ztar $ cd /var/lib/casacore/data $ tar xfz /WSRT_Measures.ztar $ rm /WSRT_Measures.ztar To install the **LINC** software package call:: $ pip3 install --upgrade pip && \ $ git clone https://git.astron.nl/RD/LINC.git && \ $ cd && \ $ git checkout && \ $ git pull && \ $ pip3 install --upgrade $PWD && \ $ cd .. && \ $ rm -rfv where ```` is the name of the temporary **LINC** source directory from where you build the package and ```` the `version number`_ you aim to install. You will find the workflow description files in ``/usr/local/share/linc/workflows`` if not specified elsewhere. Those are written in the `Common Workflow Language`_ (CWL). .. _toil: https://toil.readthedocs.io/en/latest/index.html .. _cwltool: https://github.com/common-workflow-language/cwltool .. _Common Workflow Language: https://www.commonwl.org/ .. _CentOS: https://docs.docker.com/engine/install/centos/ .. _Debian: https://docs.docker.com/engine/install/debian/ .. _Fedora: https://docs.docker.com/engine/install/fedora/ .. _Ubuntu: https://docs.docker.com/engine/install/ubuntu/ .. _version number: https://git.astron.nl/RD/LINC/-/tags .. _LSMTool: https://git.astron.nl/RD/LSMTool .. _RMextract: https://github.com/lofar-astron/RMextract.git .. _LoSoTo: https://github.com/revoltek/losoto.git .. _WSClean: https://gitlab.com/aroffringa/wsclean.git .. _DP3: https://git.astron.nl/RD/DP3.git .. _EveryBeam: https://git.astron.nl/RD/EveryBeam.git .. _LOFARBeam: https://github.com/lofar-astron/LOFARBeam.git .. _aoflagger: https://gitlab.com/aroffringa/aoflagger.git .. _IDG: https://git.astron.nl/RD/idg.git .. _Dysco: https://github.com/aroffringa/dysco.git .. _LofarStMan: https://github.com/lofar-astron/LofarStMan .. _ASTRON gitlab page: https://git.astron.nl/RD/LINC .. _Sagecal: https://github.com/nlesc-dirac/sagecal .. _Singularity: https://sylabs.io/singularity/ .. _virtual environment: https://docs.python.org/3/library/venv.html .. _conda: https://docs.conda.io/en/latest/miniconda.html .. _udocker: https://indigo-dc.gitbook.io/udocker/