[et_pb_section fb_built=”1″ custom_padding_last_edited=”on|desktop” _builder_version=”4.14.7″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||||false|false” custom_padding=”2%|20%|2%|20%|false|true” custom_padding_tablet=”4%|0%|4%|0%|true|true” custom_padding_phone=”6%|0%|6%|0%|true|true” border_color_top=”#e1e1e1″ locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ custom_margin=”||||false|false” custom_padding=”0px||0px||false|true” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.14.7″ _module_preset=”default” text_text_color=”#000000″ text_line_height=”1.8em” global_colors_info=”{}”]<\/p>\n
Organizations who process large amounts of data and large files, and who also use Amazon S3 storage, need a simple way to get data into S3, and then from S3 to Glacier for data archiving and other cold storage purposes.<\/p>\n
But this is easier said than done\u2014especially for IT teams managing multiple storage destinations, permissions, and other configurations for potentially hundreds of users<\/strong>:<\/p>\n All of the above can lead to a state of management and administration hell for IT teams. That said, here’s a brief tutorial on uploading data from S3 to Glacier using the AWS Management Console and CLI\u2014along with why MASV<\/a> is a better option to upload data to any version of S3.<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_font_size=”26px” width=”100%” width_tablet=”100%” width_phone=”100%” width_last_edited=”on|tablet” max_width=”100%” custom_margin=”|-54px|0px||false|false” custom_padding=”0px|||0px|false|false” global_colors_info=”{}”]<\/p>\n Table of Contents<\/strong><\/p>\n<\/div>\n<\/div>\n [\/et_pb_text][et_pb_text _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_cta title=”Ingest Large Files Into Cloud Cold Storage Fast” button_url=”https:\/\/app.massive.io\/en\/signup” button_text=”Start for Free” module_class=”starttrial” _builder_version=”4.14.7″ _module_preset=”default” header_level=”h5″ header_font=”||||||||” header_font_size=”26px” header_line_height=”1.3em” body_line_height=”1.8em” background_color=”#202332″ use_background_color_gradient=”on” background_color_gradient_start=”#072231″ background_color_gradient_end=”#031119″ custom_button=”on” button_text_size=”18px” button_text_color=”#FFFFFF” button_bg_color=”#0472ef” button_bg_color_gradient_start=”#0472ef” button_bg_color_gradient_end=”#005dc6″ button_bg_color_gradient_direction=”90deg” button_border_width=”0px” button_font=”Roboto|700|||||||” button_custom_padding=”10px|42px|10px|42px|true|true” custom_margin=”||20px||false|false” link_option_url=”https:\/\/app.massive.io\/en\/signup” border_radii=”on|10px|10px|10px|10px” border_color_top=”#3d72e7″ border_color_left=”#3d72e7″ box_shadow_style=”preset2″ box_shadow_horizontal=”-13px” box_shadow_style_button=”preset1″ locked=”off” global_colors_info=”{}” button_border_width__hover_enabled=”on|hover” button_custom_padding__hover_enabled=”on|hover” button_custom_padding__hover=”|2em|||false|false” button_border_width__hover=”0px” button_bg_color__hover=”#005dc6″ button_bg_color__hover_enabled=”on|desktop”]<\/p>\n MASV integrates with multiple cloud cold storage platforms and can automate storage workflows.<\/p>\n [\/et_pb_cta][et_pb_code _builder_version=”4.14.7″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_code][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ custom_padding_last_edited=”on|desktop” _builder_version=”4.14.7″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||||false|false” custom_padding=”2%|20%|2%|20%|false|true” custom_padding_tablet=”4%|0%|4%|0%|true|true” custom_padding_phone=”6%|0%|6%|0%|true|true” border_width_top=”1px” border_color_top=”#e1e1e1″ locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ custom_margin=”||||false|false” custom_padding=”0px||||false|true” locked=”off” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text module_id=”section-one” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Amazon Simple Storage Service<\/a> (S3) is an object storage service with powerful scalability, security, data availability, and performance. Because object storage is designed to house unstructured data, Amazon S3 storage is popular among M&E companies who work with tons of\u00a0 images and videos.<\/p>\n [\/et_pb_text][et_pb_text module_id=”physical” _builder_version=”4.14.7″ _module_preset=”default” text_text_color=”#000000″ text_line_height=”1.8em” header_4_text_color=”#444444″ background_color=”rgba(158,213,247,0.19)” custom_padding=”2%|3%|2%|3%|false|true” border_width_left=”5px” border_color_left=”#3d72e7″ global_colors_info=”{}”]<\/p>\n \ud83d\udca1 MASV’s no-code integrations<\/a> with several major cloud providers, including S3, allow hands-free content ingest into cloud storage and file transfer and storage automations.<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.14.7″ _module_preset=”default” text_text_color=”#000000″ text_line_height=”1.8em” header_3_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n S3 isn\u2019t a monolith, however: It contains several different storage classes<\/a> tailored to different use cases and data storage requirements. These storage classes include:<\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/massive.io\/wp-content\/uploads\/2024\/10\/black-woman-videographer-in-web-online-conference-2023-11-27-05-33-04-utc.jpg” alt=”Placeholder image” title_text=”Black woman videographer in web online conference with project manager” _builder_version=”4.14.7″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][et_pb_text _builder_version=”4.14.7″ _module_preset=”default” text_text_color=”#000000″ text_line_height=”1.8em” header_3_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Each S3 storage class has differing storage and data egress costs and performance, with infrequent access (IA) and cold (Glacier) storage options featuring cheaper storage but more expensive (and less performant) data egress. Some Glacier storage options take several hours or even days to retrieve data.<\/strong><\/p>\n For an analysis of various S3 storage classes, their cost, performance, and suitability for various use cases\u2014specifically around S3 Glacier storage classes\u2014see our post on setting up and managing a cloud archive<\/a>.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=”1_3,2_3″ _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ background_color=”#f5f5f5″ custom_margin=”||||false|false” custom_padding=”3%|3%|3%|3%|true|true” border_radii=”on|8px|8px|8px|8px” locked=”off” global_colors_info=”{}”][et_pb_column type=”1_3″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_image src=”https:\/\/massive.io\/wp-content\/uploads\/2025\/10\/How-to-Set-Up-and-Manage-a-Cloud-Archive-1.jpg” alt=”Cloud Storage Machines” title_text=”How-to-Set-Up-and-Manage-a-Cloud-Archive-1″ _builder_version=”4.14.7″ _module_preset=”default” border_radii=”on|5px|5px|5px|5px” global_colors_info=”{}”][\/et_pb_image][\/et_pb_column][et_pb_column type=”2_3″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n What are some of the top cloud archive best practices you should follow? We’ve got you covered.<\/p>\n Read more ><\/a><\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ custom_padding_last_edited=”on|desktop” _builder_version=”4.14.7″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||||false|false” custom_padding=”2%|20%|2%|20%|false|true” custom_padding_tablet=”4%|0%|4%|0%|true|true” custom_padding_phone=”6%|0%|6%|0%|true|true” border_width_top=”1px” border_color_top=”#e1e1e1″ locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ custom_margin=”||||false|false” custom_padding=”0px||||false|true” locked=”off” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Amazon S3 Glacier is a type of S3 storage\u2014specifically, cold storage used for less frequently accessed data, such as cloud archive data<\/a>.<\/p>\n Amazon Glacier and other cold storage options, such as Google Coldline<\/a> or Azure Blob Storage\u2019s Cold Tier<\/a>, offer several benefits when storing infrequent access (IA) data, such as:<\/p>\n But while there are plenty of advantages of cold storage, it\u2019s also important to remember to closely monitor who has access to AWS Glacier from your team.<\/strong><\/p>\n That\u2019s because while Glacier offers cheaper storage costs than other storage types, as mentioned, it can be much more expensive to egress data from cold storage (and it takes longer than retrieving data from hot storage).<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ custom_padding_last_edited=”on|desktop” _builder_version=”4.14.7″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||||false|false” custom_padding=”2%|20%|2%|20%|false|true” custom_padding_tablet=”4%|0%|4%|0%|true|true” custom_padding_phone=”6%|0%|6%|0%|true|true” border_width_top=”1px” border_color_top=”#e1e1e1″ locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ custom_margin=”||||false|false” custom_padding=”0px||||false|true” locked=”off” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text module_id=”section-three” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n First off: We always recommend using either the CLI or infrastructure-as-code (IaC) to configure and manage your Amazon storage, because it allows for greater control and better oversight over potentially devastating misconfigurations. It’s easy when using the Management Console to create a set of cascading problems through one or two misplaced clicks.<\/p>\n But we also recognize that non-technical and newer users may prefer the AWS Management Console.<\/p>\n So, here\u2019s how to migrate your data from S3 to Glacier<\/a> using the Management Console.<\/p>\n [\/et_pb_text][et_pb_text module_id=”console” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Part 1: Set up an S3 Bucket<\/strong><\/p>\n Part 2: Upload to S3 and select your storage class<\/strong><\/p>\n [\/et_pb_text][et_pb_text module_id=”physical” _builder_version=”4.14.7″ _module_preset=”default” text_text_color=”#000000″ text_line_height=”1.8em” header_4_text_color=”#444444″ background_color=”rgba(158,213,247,0.19)” custom_padding=”2%|3%|2%|3%|false|true” border_width_left=”5px” border_color_left=”#3d72e7″ global_colors_info=”{}”]<\/p>\n \ud83d\udca1 You can upload a single object up to 160GB to S3 using the console. To upload a file larger than 160GB you will need to use the AWS CLI, AWS SDK, or Amazon S3 REST API. Or you can use MASV to upload files of up to 5TB to S3<\/strong>.<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 2. Choose the Add files<\/strong> button, then navigate to the files you want to upload. The file will appear in the Files and folders<\/strong> section of S3.<\/p>\n 3. Under the bucket\u2019s Properties<\/strong> section, you can then select the S3 storage class (such as Glacier Deep Archive or Glacier Instant Retrieval) you want to upload your bucket to.<\/p>\n [\/et_pb_text][et_pb_text module_id=”physical” _builder_version=”4.14.7″ _module_preset=”default” text_text_color=”#000000″ text_line_height=”1.8em” header_4_text_color=”#444444″ background_color=”rgba(158,213,247,0.19)” custom_padding=”2%|3%|2%|3%|false|true” border_width_left=”5px” border_color_left=”#3d72e7″ global_colors_info=”{}”]<\/p>\n Note<\/strong>: Some storage classes have minimum durations for data uploaded; for example, Glacier Deep Archive bills for 180 days even if the file is deleted after just a few days.<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 4. You\u2019ll next see a file upload status banner. Once the upload is complete, you\u2019ll see an upload summary.<\/p>\n Congratulations! You have successfully uploaded a file from your computer to S3, and from S3 to Glacier, using the AWS Management Console.<\/p>\n [\/et_pb_text][et_pb_text module_id=”cli” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Now let’s try setting up the same process using the AWS CLI. In this example, we’ll create a bucket named masv-archive using the AWS command line tool<\/a>.<\/p>\n This bucket will follow a few best practices for security: Public access blocked, versioning on, at-rest encryption on, object lock on. To track this archive for project management and billing, we’ll add a tag, project-x. Then we’ll upload a small file and a large file from local storage to S3 Glacier.<\/p>\n While the AWS CLI is more flexible than Management Console in that it can support uploads larger than 160GB, it’s got a 5 GB single object PUT limit, necessitating multipart uploads.<\/strong><\/p>\n Before you begin, make sure your credentials and region are already configured for the AWS CLI<\/a>. We’ll use the masv profile in this example.<\/p>\n Let’s create and configure the bucket. You only need to do this once.<\/p>\n 1. Create the bucket:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Output:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 2. Block public access:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 3. Turn on versioning:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 4. Turn on server-side encryption:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 5. Add a tag named “project-x”:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Now the bucket is ready for storing our archived files. We’ll upload from local storage to our bucket with the Glacier storage class.<\/p>\n Uploading small files, less than 5 GB, is straightforward. It takes just one command. For example, if your local file is named my-small-file.mp4<\/strong>, then you would enter this:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Output:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n Uploading files larger than 5 GB using CLI requires multiple steps, because it doesn’t support uploading files greater than 5 GB directly.<\/p>\n Instead, you need to break it up into parts on your local computer and upload each part individually. AWS calls this multipart uploading. 1. Get an upload ID. We need this upload ID for each part that we upload in later steps.<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 2. Upload the first 5 GB part of the file.<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n We need to remember the ETag value in the output:<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 3. Upload the last part of the file.<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n 4. Finalize the multipart upload and clean up.<\/p>\n [\/et_pb_text][et_pb_text module_id=”file-uploading-isnt-easy” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ locked=”off” global_colors_info=”{}”]<\/p>\n [\/et_pb_text][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n At this point, the file is safely stored in S3 Glacier.<\/p>\n As you can guess, this example for a multipart upload isn’t the simplest way to transfer large files to S3. It isn’t the fastest way either, even for small files.<\/p>\n To improve performance, you can tune the size of each part and upload multiple parts at a time, but that’s beyond the scope of this example.<\/p>\n Multipart uploading does have other advantages, however, like faster recovery from errors and resuming paused or interrupted uploads.<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ custom_padding_last_edited=”on|desktop” _builder_version=”4.14.7″ _module_preset=”default” background_color=”#FFFFFF” custom_margin=”||||false|false” custom_padding=”2%|20%|2%|20%|false|true” custom_padding_tablet=”4%|0%|4%|0%|true|true” custom_padding_phone=”6%|0%|6%|0%|true|true” border_width_top=”1px” border_color_top=”#e1e1e1″ locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ custom_margin=”||||false|false” custom_padding=”0px||||false|true” locked=”off” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text module_id=”section-four” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n So, you’ve got your files uploaded to S3 Glacier. Great! Now what?<\/p>\n At MASV, we\u2019ve got plenty of experience managing a variety of S3 and other cloud instances. Here are some of our top tips for managing S3 Glacier in particular:<\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/massive.io\/wp-content\/uploads\/2024\/10\/S3-glacier.jpg” alt=”Placeholder image” title_text=”Group of Coworkers Discussing Issues In Evening” _builder_version=”4.14.7″ _module_preset=”default” global_colors_info=”{}”][\/et_pb_image][et_pb_text module_id=”section-two” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ global_colors_info=”{}”]<\/p>\n [\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ custom_padding_last_edited=”on|phone” _builder_version=”4.14.7″ background_color=”#f5f5f5″ custom_margin=”||||false|false” custom_padding=”2%|20%|2%|20%|true|true” custom_padding_tablet=”4%|0%|4%|0%|true|true” custom_padding_phone=”6%|0%|6%|0%|true|true” global_colors_info=”{}”][et_pb_row _builder_version=”4.14.7″ _module_preset=”7b1bf5ad-cc2a-4448-981c-4963d88bd6e8″ custom_margin=”||||false|false” custom_padding=”||||false|true” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.9.3″ _module_preset=”default” global_colors_info=”{}”][et_pb_text module_id=”section-five” _builder_version=”4.14.7″ _module_preset=”2514b1ee-af07-4bc3-a96b-c9aaa32f4a18″ text_text_color=”#000000″ text_line_height=”1.8em” header_2_text_color=”#000000″ hover_enabled=”0″ global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n Because post-production workflows<\/a> require data to be saved in multiple locations (such as on-prem device storage for video editing and the cloud for backup and archive), configuring S3 Glacier (and uploading from S3 to Glacier) is typically part of a larger content ingest workflow setup process that involves both IT and operations staff.<\/p>\n Developers can get around these limitations by spending months creating their own uploader for S3\u2014or they can use MASV Centralized Ingest<\/a> as a central, secure point of data ingest to cloud and networked on-prem storage and save teams hours of configuration and setup time.<\/strong><\/p>\n Teams can use MASV to facilitate stakeholder uploads to multiple destinations at once, including a range of S3 Storage Classes, without having to grant stakeholders privileged access. When you set a Storage Class for a MASV integration, you eliminate user error and reduce the risk of storage cost overruns.<\/p>\n\n
\n
\n
What is Amazon S3?<\/h2>\n
S3 storage classes<\/h3>\n
\n
Cloud Archive: Benefits, Challenges, and Best Practices<\/h4>\n
What is Amazon S3 Glacier?<\/h2>\n
\n
How to Migrate Data To S3, and From S3 to Glacier<\/h2>\n
Management Console<\/h3>\n
\n
\n
\n
Command Line Interface (CLI)<\/h3>\n
$ aws s3api create-bucket \\\\\n --bucket masv-archive \\\\\n --region $(aws configure get region --profile masv) \\\\\n --object-lock-enabled-for-bucket \\\\\n --profile masv \n<\/pre>\n
{\n \"Location\": \"\/masv-archive\"\n}\n<\/pre>\n$ aws s3api put-public-access-block \\\n --bucket masv-archive \\\n --public-access-block-configuration
BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true \\<\/pre>\n --profile masv\n<\/pre>\n
$ aws s3api put-bucket-versioning \\\\\n --bucket masv-archive \\\\\n --versioning-configuration Status=Enabled \\\\\n --profile masv\n<\/pre>\n
$ aws s3api put-bucket-encryption \\\n --bucket masv-archive \\\n --server-side-encryption-configuration '{\n \"Rules\": [{\n \"ApplyServerSideEncryptionByDefault\": {\n \"SSEAlgorithm\": \"AES256\"\n }\n }]\n }' \\\n --profile masv\n<\/pre>\n$ aws s3api put-bucket-tagging \\\n --bucket masv-archive \\\n --tagging 'TagSet=[{Key=project-x,Value=true}]' \\\n --profile masv\n<\/pre>\n$ aws s3api put-object \\\n --bucket masv-archive \\\n --key my-small-file.mp4 \\\n --body my-small-file.mp4 \\\n --storage-class GLACIER \\\n --profile masv\n<\/pre>\n
{\n \"ETag\": \"\\\"e5x2a5mbpdl4e2d4c3549862b2a5f2b\\\"\",\n \"ServerSideEncryption\": \"AES256\",\n \"VersionId\": \"eixDahmmp.lreyCa8cKkkHoV80r17S8k\"\n}\n<\/pre>\nMultipart uploads via CLI<\/h3>\n
In this example, we’ll keep it simple with a 10 GB file that only needs to be broken up into two parts.<\/p>\n$ uploadId=$(aws s3api create-multipart-upload \\\n --bucket masv-archive \\\n --key my-large-file.mp4 \\\n --storage-class GLACIER \\\n --query UploadId \\\n --output text \\\n --profile masv)\n<\/pre>\n
$ head --bytes 1073741824 my-large-file.mp4 > my-large-file.mp4.part1\n$ aws s3api upload-part \\\n --bucket masv-archive \\\n --key my-large-file.mp4 \\\n --part-number 1 \\\n --body my-large-file.mp4.part1 \\\n --upload-id \"$uploadId\" \\\n --profile masv\n<\/pre>\n
{\n \"ServerSideEncryption\": \"AES256\",\n \"ETag\": \"\\\"e5x5adm2pflee9ed4c3549862b2a5f2b\\\"\"\n}\n<\/pre>\n$ tail --bytes 1073741825 my-large-file.mp4 > my-large-file.mp4.part2\n$ aws s3api upload-part \\\n --bucket masv-archive \\\n --key my-large-file.mp4 \\\n --part-number 2 \\\n --body my-large-file.mp4.part2 \\\n --upload-id \"$uploadId\" \\\n --profile masv\n<\/pre>\n
$ aws s3api complete-multipart-upload \\\n --bucket my-archive \\\n --key my-large-file.mp4 \\\n --upload-id $upload_id \\\n --multipart-upload '{\"Parts\":[{\"ETag\":\"e5x5adm2pflee9ed4c3549862b2a5f2b\",\"Part
Number\":1}, {\"ETag\":\"5e5xda2mfpel9ede4c534986ba225b2f\",\"PartNumber\":2} ]}' \\\n --profile masv\n$ rm my-large-file.mp4.part1 my-large-file.mp4.part2\n<\/pre>\nManaging S3 Glacier: Tips and Best Practices<\/h2>\n
\n
\n
\n
\n
\n
MASV: The Easy Way to Upload Massive Files to S3 Glacier<\/h2>\n
\n