Skip to content

GH-50148: [C++] Add Content-Encoding support to S3 filesystem metadata#50167

Open
alytantawyy wants to merge 1 commit into
apache:mainfrom
alytantawyy:fix-s3-content-encoding
Open

GH-50148: [C++] Add Content-Encoding support to S3 filesystem metadata#50167
alytantawyy wants to merge 1 commit into
apache:mainfrom
alytantawyy:fix-s3-content-encoding

Conversation

@alytantawyy

@alytantawyy alytantawyy commented Jun 12, 2026

Copy link
Copy Markdown

Rationale for this change

The S3 filesystem metadata handling supported headers such as Content-Type,
Content-Language, Cache-Control, and Expires, but omitted
Content-Encoding.

As a result, Content-Encoding was not propagated when writing S3 object
metadata, and it was also not returned when reading metadata back.

What changes are included in this PR?

  • add Content-Encoding to S3 object metadata extraction
  • add Content-Encoding to the S3 metadata setter whitelist
  • add a dedicated setter for Content-Encoding
  • extend the S3 metadata round-trip test to cover explicit and default
    Content-Encoding metadata

Are these changes tested?

Yes.

./cpp/build-s3-system/debug/arrow-s3fs-test --gtest_filter=TestS3FS.OpenOutputStreamMetadata

* GitHub Issue: #50148

@pitrou

pitrou commented Jun 25, 2026

Copy link
Copy Markdown
Member

@raulcd @kou I have no idea why the CUDA Python jobs have run here, as I only see non-CUDA C++ changes?

@raulcd

raulcd commented Jun 25, 2026

Copy link
Copy Markdown
Member

I have no idea why the CUDA Python jobs have run here, as I only see non-CUDA C++ changes?

This is weird, all extra jobs are running not only CUDA but R Extra, Packaging Extra, C++ Extra and it does not seem like any of the required labels were ever present.

@raulcd

raulcd commented Jun 25, 2026

Copy link
Copy Markdown
Member

ok, the PR seems to be quite outdated with respect to main so the git diff when checking labels seems to show lots of files changed and forces the jobs to run as some of those files trigger the them, see: https://github.com/apache/arrow/actions/runs/27442216149/job/83430959300?pr=50167#step:3:39

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants