How to access Landsat 8 DataSet stored in AWS S3?

NASA provides free access to the Landsat data products. Also, NASA dumps the datasets into an S3 bucket so that the data can be accessed by anyone easily and can leverage the scalability of the cloud to process those datasets.

In this article, I will show the commands you need to copy the datasets to your local.

AWS will charge YOU for the S3 Operations like Data Transfer, API requests etc.

Prerequisites:

• You need an AWS account.
• You are aware of the costs.
• The user has permission to access S3.

Path to the DataSet:

In order to access specific datasets, you need more details about the tile ids, and the data products you are looking for. The data set path will look like,

Bucket/Collection/Processing Level/Projection/Sensor/Year Acquired/Path/Row/Product

The “Product” value will be generated using the logic mentioned in these pages,

Collection 1: https://www.usgs.gov/faqs/what-naming-convention-landsat-collections-level-1-scenes?qt-news_science_products=0#qt-news_science_products

Collection 2: https://www.usgs.gov/faqs/what-naming-convention-landsat-collection-2-level-1-and-level-2-scenes?qt-news_science_products=0#

LXSS_LLLL_PPPRRR_YYYYMMDD_yyyymmdd_CC_TX

Where:

L = Landsat
X = Sensor (“C”=OLI/TIRS combined, “O”=OLI-only, “T”=TIRS-only, “E”=ETM+, “T”=“TM, “M”=MSS)
SS = Satellite (”07”=Landsat 7, “08”=Landsat 8)
LLL = Processing correction level (L1TP/L1GT/L1GS)
PPP = WRS path
RRR = WRS row
YYYYMMDD = Acquisition year, month, day
yyyymmdd - Processing year, month, day
CC = Collection number (01, 02, …)
TX = Collection category (“RT”=Real-Time, “T1”=Tier 1, “T2”=Tier 2)

For this example, I am interested in the Landsat 8, Collection 2, Level 2 data products for path 144 and row 052. So my “Product” name will be,

LC08_L2SP_144052_20211111_20211117_02_T1

You can validate the Product name, Acquisition date, and Processing data from the Earth Explorer website. If you are programmatically fetching these records, you can do “s3 ls” to figure out the files for a specific tile.

The full path to the dataset folder is below,

usgs-landsat/collection02/level-2/standard/oli-tirs/2021/144/052/LC08_L2SP_144052_20211111_20211117_02_T1

AWS CLI Command:

List all files for a specific aquistion:

Use AWS S3 ls to list the files.

Dont forget the “–request-payer requester parameter.”

% aws s3 ls s3://usgs-landsat/collection02/level-2/standard/oli-tirs/2021/144/052/ --request-payer requester 

...

                           PRE LC08_L2SP_144052_20210825_20210901_02_T1/
                           PRE LC08_L2SP_144052_20210910_20210916_02_T1/
                           PRE LC08_L2SP_144052_20210926_20211001_02_T1/
                           PRE LC08_L2SP_144052_20211012_20211019_02_T1/
                           PRE LC08_L2SP_144052_20211028_20211104_02_T1/
2021-11-04 11:13:36      12043 catalog.json
% aws s3 ls s3://usgs-landsat/collection02/level-2/standard/oli-tirs/2021/144/052/LC08_L2SP_144052_20211028_20211104_02_T1/ --request-payer requester

2021-11-04 11:13:12     117391 LC08_L2SP_144052_20211028_20211104_02_T1_ANG.txt
2021-11-04 11:13:12      18718 LC08_L2SP_144052_20211028_20211104_02_T1_MTL.json
2021-11-04 11:13:12      15423 LC08_L2SP_144052_20211028_20211104_02_T1_MTL.txt
2021-11-04 11:13:12      22725 LC08_L2SP_144052_20211028_20211104_02_T1_MTL.xml
2021-11-04 11:13:12    3502697 LC08_L2SP_144052_20211028_20211104_02_T1_QA_PIXEL.TIF
2021-11-04 11:13:12     224279 LC08_L2SP_144052_20211028_20211104_02_T1_QA_RADSAT.TIF
2021-11-04 11:13:12   94674915 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B1.TIF
2021-11-04 11:13:14   94644623 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B2.TIF
2021-11-04 11:13:16   94615545 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B3.TIF
2021-11-04 11:13:17   95409785 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B4.TIF
2021-11-04 11:13:19   98350578 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B5.TIF
2021-11-04 11:13:21   96379448 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B6.TIF
2021-11-04 11:13:24   94331525 LC08_L2SP_144052_20211028_20211104_02_T1_SR_B7.TIF
2021-11-04 11:13:26    3995517 LC08_L2SP_144052_20211028_20211104_02_T1_SR_QA_AEROSOL.TIF
2021-11-04 11:13:26      19177 LC08_L2SP_144052_20211028_20211104_02_T1_SR_stac.json
2021-11-04 11:13:26   41521397 LC08_L2SP_144052_20211028_20211104_02_T1_ST_ATRAN.TIF
2021-11-04 11:13:27   85576611 LC08_L2SP_144052_20211028_20211104_02_T1_ST_B10.TIF
2021-11-04 11:13:29    8439616 LC08_L2SP_144052_20211028_20211104_02_T1_ST_CDIST.TIF
2021-11-04 11:13:29   28266885 LC08_L2SP_144052_20211028_20211104_02_T1_ST_DRAD.TIF
2021-11-04 11:13:30   36609690 LC08_L2SP_144052_20211028_20211104_02_T1_ST_EMIS.TIF
2021-11-04 11:13:31   18006951 LC08_L2SP_144052_20211028_20211104_02_T1_ST_EMSD.TIF
2021-11-04 11:13:31   49425021 LC08_L2SP_144052_20211028_20211104_02_T1_ST_QA.TIF
2021-11-04 11:13:32   74933780 LC08_L2SP_144052_20211028_20211104_02_T1_ST_TRAD.TIF
2021-11-04 11:13:33   40261148 LC08_L2SP_144052_20211028_20211104_02_T1_ST_URAD.TIF
2021-11-04 11:13:34      18717 LC08_L2SP_144052_20211028_20211104_02_T1_ST_stac.json
2021-11-04 11:13:34     113523 LC08_L2SP_144052_20211028_20211104_02_T1_thumb_large.jpeg
2021-11-04 11:13:34      14972 LC08_L2SP_144052_20211028_20211104_02_T1_thumb_small.jpeg
% 

Copy all files to local:

Below copy comman will download ~900 MB of data. Beware of cost implications.

% mkdir LC08_L2SP_144052_20211028_20211104_02_T1
% cd LC08_L2SP_144052_20211028_20211104_02_T1
% aws s3 cp s3://usgs-landsat/collection02/level-2/standard/oli-tirs/2021/144/052/LC08_L2SP_144052_20211028_20211104_02_T1/  . --request-payer requester --recursive --request-payer requester

download: s3://usgs-landsat/collection02/level-2/standard/oli-tirs/2021/144/052/LC08_L2SP_144052_20211028_20211104_02_T1/LC08_L2SP_144052_20211028_20211104_02_T1_MTL.txt to ./LC08_L2SP_144052_20211028_20211104_02_T1_MTL.txt
....
.....
.....

% du -sh 
1.0G	.

Open the large image file:

open LC08_L2SP_144052_20211028_20211104_02_T1_thumb_large.jpeg