Setup separate AWS S3 bucket replica for use in local development and testing #87463

New Issue

Anna Sirota · 2021-04-13T16:56:28+02:00

Anna Sirota commented

2021-04-13 16:56:28 +02:00

Some features that'd be great to have are problematic if we continue using the same storage bucket for local dev servers.
One example would be syncing training section names with ContentDisposition of the relevant videos stored in S3: testing something like this locally while having access to production bucket can actually lead to unintended changes of download names for existing training sections.

Steps to make this happen:

- Setup replication from https:*s3.console.aws.amazon.com/s3/buckets/blender-studio?region=eu-central-1&tab=objects to https:*s3.console.aws.amazon.com/s3/buckets/blender-studio-test?region=eu-central-1;
Run an AWS sync to copy all the existing files are also in the test/dev bucket (this doubles the costs of storage, maybe copy files selectively, e.g. trainings, blog post static assets only);
Get IAM permissions to be able to create new IAM users (or ask @fsiddi to create them);
Create a new user for production and generate new credentials with access to the production bucket;
Replace production credentials;
Delete credentials previously used in production (making sure they aren't usable anymore);
Create a new user for Mike and generate new credentials for him with access to the test bucket only;
Replace Mike's credentials;
Generate new credentials for myself with access to the test bucket;
Replace my local credentials.

Some features that'd be great to have are problematic if we continue using the same storage bucket for local dev servers. One example would be syncing training section names with `ContentDisposition` of the relevant videos stored in S3: testing something like this locally while having access to production bucket can actually lead to unintended changes of download names for existing training sections. Steps to make this happen: - - [x] Setup replication from https:*s3.console.aws.amazon.com/s3/buckets/blender-studio?region=eu-central-1&tab=objects to https:*s3.console.aws.amazon.com/s3/buckets/blender-studio-test?region=eu-central-1; - Run an AWS sync to copy all the existing files are also in the test/dev bucket (**this doubles the costs of storage**, maybe copy files selectively, e.g. trainings, blog post static assets only); - Get IAM permissions to be able to create new IAM users (or ask @fsiddi to create them); - Create a new user for production and generate new credentials with access to the production bucket; - Replace production credentials; - Delete credentials previously used in production (making sure they aren't usable anymore); - Create a new user for Mike and generate new credentials for him with access to the test bucket only; - Replace Mike's credentials; - Generate new credentials for myself with access to the test bucket; - Replace my local credentials.

Anna Sirota self-assigned this 2021-04-13 16:56:28 +02:00

Anna Sirota commented

2021-04-13 16:56:28 +02:00

Changed status from 'Needs Triage' to: 'Confirmed'