Understanding Amazon S3 by Uploading One File

Amazon S3 (Simple Storage Service) stores files as objects inside buckets, and uploading an object doesn't make it public. You can download it through an authenticated AWS CLI request and still get AccessDenied when you open its URL. A presigned URL gives someone temporary access without making the bucket public.

The confusing part comes right after the upload. The file is there. The CLI can read it. You copy the object's URL into a browser and get a 403. It's easy to assume you missed a setting, or that S3 needs time to catch up.

The important detail is what the CLI sends that a plain browser request doesn't: an AWS signature.

This is the first post in AWS by Doing, a series about learning services by running small experiments. We're starting with one text file: upload it, read it three ways, and watch a temporary link expire. That's enough to make the access rules visible.

Quick answer

To understand S3 access with one file:

1. Create a private bucket and keep all four public-access blocks enabled.
2. Upload a text file, then download it through the authenticated CLI.
3. Request its HTTPS URL anonymously. The private object should return 403 AccessDenied.
4. Generate a presigned URL. Use it to download the same object without configuring credentials in the recipient's terminal.
5. Repeat the request after expiry. The temporary link should stop working, while the object stays in the bucket.

We ran this with AWS CLI 2.37.9 in US East (Ohio), us-east-2. The results below are from that run.

What are we actually creating?

S3 stores objects inside buckets. For this experiment, the object has three useful parts: a key, the file's bytes, and metadata describing those bytes.

The key is notes/hello.txt. It looks like a path, but the whole string is the object's name. notes/ is a prefix. You don't create a directory first, and there isn't a separate folder object in this example. S3's console displays prefixes as folders, which makes browsing easier but can hide the underlying model. AWS's object-key documentation

For now, think of the bucket and key as the address of your file. Permissions decide whether a request to that address succeeds.

Prerequisites

• AWS CLI 2.32.0 or newer for the aws login flow shown here. Installation instructions
• A Bash or Zsh terminal with curl, uuidgen, and diff. The commands use macOS/Linux shell syntax.
• An AWS identity allowed to create and delete the lab bucket, configure and inspect its public-access blocks, and upload, read, and delete its objects.

Check your account's billing plan and credits before starting. This is a tiny experiment, but its cost depends on your account and region. The cleanup commands are included below.

Step 1: Sign in and choose the region

For an account using console-based local sign-in, start with:

set +x
aws --version
aws configure set region us-east-2
aws login
aws sts get-caller-identity --no-cli-pager

aws login opens the browser and obtains temporary credentials. get-caller-identity tells you which account and identity the CLI is using. Check that before creating anything.

Keep the login flow out of screen recordings. The identity check prints an account ID, ARN, and user ID, which identify the caller but aren't credentials. Redact them before sharing output. set +x disables shell tracing so later commands don't print expanded variables. Leave AWS --debug and curl verbose/trace options off for this walkthrough.

For IAM users and roles, AWS documents the SignInLocalDevelopmentAccess managed policy as a prerequisite for this login method. Organizations using IAM Identity Center should use their configured SSO profile and aws sso login instead. AWS CLI sign-in documentation

One setup gotcha from this run: the account used AWS's new project experience, with Ohio assigned as its home region. There was no region selector where we'd normally expect one. Selecting other regions requires advanced features, so we kept the experiment in Ohio. Use your project's supported region, and keep it consistent in the commands below. AWS project regions

Step 2: Create a private bucket

Run the following blocks in the same terminal session. Later commands reuse these variables.

LAB_REGION=us-east-2
LAB_BUCKET="ravi-aws-s3-lab-$(uuidgen | tr '[:upper:]' '[:lower:]')"
LAB_KEY=notes/hello.txt
LAB_DIR=$(mktemp -d)
 
aws s3 mb "s3://$LAB_BUCKET" --region "$LAB_REGION"

The random suffix helps avoid a bucket-name collision. If creation fails, stop there and resolve the error before continuing.

Explicitly enable and inspect all four public-access blocks:

aws s3api put-public-access-block \
  --bucket "$LAB_BUCKET" --region "$LAB_REGION" \
  --public-access-block-configuration \
  'BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true'
 
aws s3api get-public-access-block \
  --bucket "$LAB_BUCKET" --region "$LAB_REGION" --no-cli-pager

All four values came back true. These settings guard against public grants through bucket policies and ACLs. Authorized reads still work, which is what we'll test next. Block Public Access documentation

Step 3: Upload one file

printf 'Hello from AWS by Doing.\n' > "$LAB_DIR/hello.txt"
 
aws s3api put-object \
  --bucket "$LAB_BUCKET" --key "$LAB_KEY" \
  --body "$LAB_DIR/hello.txt" --content-type text/plain \
  --server-side-encryption AES256 \
  --region "$LAB_REGION" --no-cli-pager

Inspect what S3 stored:

aws s3api head-object \
  --bucket "$LAB_BUCKET" --key "$LAB_KEY" \
  --region "$LAB_REGION" \
  --query '{Bytes:ContentLength,Type:ContentType,Encryption:ServerSideEncryption}' \
  --no-cli-pager

What came back:

{
  "Bytes": 25,
  "Type": "text/plain",
  "Encryption": "AES256"
}

Three fields, each telling us something different. Bytes is the length of the stored content, including the trailing newline. Type is the content type we supplied. Encryption: AES256 reports S3-managed encryption at rest, or SSE-S3. Encryption protects stored data; permissions control who can request it. SSE-S3 documentation

Download the object through the CLI and compare it:

aws s3api get-object \
  --bucket "$LAB_BUCKET" --key "$LAB_KEY" \
  --region "$LAB_REGION" --no-cli-pager "$LAB_DIR/downloaded.txt"
 
diff "$LAB_DIR/hello.txt" "$LAB_DIR/downloaded.txt"

The byte comparison passed. With diff, no output means the files match. The CLI signed its read request with the configured credentials, and S3 allowed it.

There is no consistency delay to wait out here. S3 provides strong read-after-write consistency for object writes, so once the upload succeeds, a subsequent authorized read sees the object. S3 consistency model

Step 4: Request the same file anonymously

LAB_PUBLIC_URL="https://${LAB_BUCKET}.s3.${LAB_REGION}.amazonaws.com/${LAB_KEY}"
curl --max-time 30 -sS -i "$LAB_PUBLIC_URL"

The response was HTTP/1.1 403 Forbidden. These were the relevant XML fields, with request identifiers omitted:

<Code>AccessDenied</Code>
<Message>Access Denied</Message>

The authenticated download had just worked. This request failed because curl sent no AWS signature, and we hadn't granted anonymous access.

That's the distinction the upload hides. Knowing the bucket and key lets you ask for the object. It doesn't give you permission to receive it.

For a document download in an application, this is useful behavior. The file can stay private until your application decides to grant a particular download. A presigned URL is one way to do that.

Step 5: Share a temporary download

set +x
LAB_SIGNED_URL=$(aws s3 presign "s3://$LAB_BUCKET/$LAB_KEY" \
  --region "$LAB_REGION" --expires-in 60)
 
date -u
printf 'url = "%s"\n' "$LAB_SIGNED_URL" |
  curl --config - --max-time 30 --silent \
    --output "$LAB_DIR/signed-response.txt" \
    --write-out 'HTTP %{http_code}\n'

The fresh URL returned 200 and exactly the original 25 bytes. We hadn't changed a bucket policy or disabled public-access blocking. The authorization was in the signed request.

The command above prints only the HTTP status and saves the response in the private temporary directory. Passing the URL through curl's stdin keeps it out of curl's process arguments. Don't echo LAB_SIGNED_URL or paste an expanded URL into a command: that can expose it in terminal output or shell history. Keep raw error responses private too; signature errors can contain signing details. curl's config and output options

The signature lets the request use the signer's permissions for this download. The URL is reusable during its valid lifetime, so anyone holding it can request the file. Keep it out of screenshots, source control, and published examples. Presigned URL behavior

Wait at least 75 seconds after generating it, then make a fresh request using the same variable:

date -u
printf 'url = "%s"\n' "$LAB_SIGNED_URL" |
  curl --config - --max-time 30 --silent \
    --output "$LAB_DIR/expired-response.xml" \
    --write-out 'HTTP %{http_code}\n'

Use the same URL. Generating another one would start a new expiration window.

These signed-request commands deliberately omit raw headers and bodies from terminal output. If you see HTTP 000, curl didn't receive an HTTP response; don't treat that as an S3 access decision.

Why does an S3 URL return 403 AccessDenied?

We repeated the request after 90 seconds. It returned 403, with these XML fields:

<Code>AccessDenied</Code>
<Message>Request has expired</Message>

Look at the two errors. Both returned HTTP 403. Both used the XML code AccessDenied. But one said Access Denied, and the other said Request has expired.

If you only log the status code, these failures look identical. They need different responses. The anonymous request needs authorization; the expired request needs a new valid link. Making the bucket public would solve the wrong problem.

The object was still in S3 when the link expired. Only the temporary authorization had stopped working.

All four public-access blocks were still enabled when checked after the experiment.

What can go wrong

A generated URL can still fail. Signing is not proof of read permission. The signing identity needs s3:GetObject, and applicable policies can still deny the request.

Temporary credentials can shorten the lifetime. A URL cannot outlive the credentials used to sign it, even if you request a longer expiration.

Use GET to test this URL. curl -I sends HEAD. The CLI's presign command creates a URL for a GET request, so changing the method changes what you are testing. CLI presign reference

The URL is for an HTTP request, not a download timer. S3 checks expiry when the request begins. A download started before expiry can continue afterward. Our test made a new request after the deadline, which is why it was rejected. Presigned URL expiration

Delete the lab resources

After recording the results, remove only the test object and the dedicated bucket:

aws s3api delete-object \
  --bucket "$LAB_BUCKET" --key "$LAB_KEY" \
  --region "$LAB_REGION" --no-cli-pager
 
aws s3 rb "s3://$LAB_BUCKET" --region "$LAB_REGION"
unset LAB_SIGNED_URL

If bucket deletion fails, inspect why before reaching for force deletion. Unexpected objects or versioning change the cleanup procedure.

Cleanup succeeded in our run: the test object was deleted and the CLI confirmed removal of the bucket.

This experiment used one tiny object and a handful of requests. Check your account's credits and the storage, request, and transfer prices for your region. We haven't measured the actual billed cost, so there's no cost figure to report yet. S3 pricing

What's next

One file was enough to separate three things that are easy to blur together: where an object lives, who can read it, and how long a particular authorization lasts.

The next post takes on IAM. We'll repeat this kind of experiment with an identity that can upload an object but can't download it. That's where AccessDenied starts becoming something you can reason about instead of something you work around.

Further reading

• S3 object keys: why folder-looking names are still keys.
• S3 Block Public Access: what the four settings control.
• Presigned URLs: permissions, expiry, and temporary credentials.
• AWS CLI presign reference: the GET-based command used in this experiment.

Hit an S3 access error that looks different from these? I'd like to compare notes. Share the error code and message, with credentials and signed URLs removed, on LinkedIn.