Configuring Git LFS for downloading LSST data packages

LSST uses Git LFS to efficiently store large files in Git repositories. Typical Science Pipelines installations, like lsst_distrib, do not require Git LFS. However, some tutorials might require Git LFS to clone a specific Git repository that does use Git LFS. The testdata_ci_hsc package is one example. This page describes how to configure Git LFS to work with LSST’s servers.

Note

LSST staff and contributors should follow the instructions in the Developer Guide for configuring Git LFS with authenticated (push) access.

Getting Git LFS

Git LFS comes with your LSST Science Pipelines installation. To check that it’s available, run:

git-lfs version

You can also install Git LFS independently of the LSST Science Pipelines. Follow the instructions on the Git LFS homepage to install Git LFS onto your system. LSST requires Git LFS 2.3.4 or later.

Configuring Git LFS for LSST

Since LSST uses its own data servers, rather than GitHub’s, you’ll need to add some extra configurations beyond a regular installation.

Create or edit the ~/.gitconfig file to add these new lines:

# Cache anonymous access to LSST Git LFS S3 servers
[credential "https://lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com"]
    helper = store
[credential "https://s3.lsst.codes"]
    helper = store

Also create or edit the ~/.git-credentials file to add these new lines:

https://:@lsst-sqre-prod-git-lfs.s3-us-west-2.amazonaws.com
https://:@s3.lsst.codes

Try it out

Trying cloning the testdata_decam Git repository to test your configuration:

git clone https://github.com/lsst/testdata_decam.git

Note

LSST contributors need to follow some extra steps to authenticate commands that push to the upstream repository on GitHub. See the Developer Guide for details.