Describe the bug

Comet's native S3 object store does not map profile credential provider class names declared to Rust AWS profile credential providers. It currently handles static keys, IMDS, environment variables, web identity, assumed role, and anonymous, but does not recognize AWS's ProfileCredentialsProvider.

Steps to reproduce

Set spark.hadoop.fs.s3a.aws.credentials.provider to com.amazonaws.auth.profile.ProfileCredentialsProvider, it throws

Unsupported credential provider: com.amazonaws.auth.profile.ProfileCredentialsProvider

Expected behavior

Should map the configured Profile Credential Provider to corresponding AWS SDK Rust impl, and also profiles to be used

Additional context

No response