Adding ability to sign URL from Compute Engine Service account

[dhermes]

See #607 and #921

[dhermes]

This is the extent of what the metadata server can do

$ SVC_ACCOUNT_URI="http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/"
$ curl ${SVC_ACCOUNT_URI} -H "Metadata-Flavor: Google"
FOOBAR-compute@developer.gserviceaccount.com/
default/
$ DEFAULT="${SVC_ACCOUNT_URI}default/"
$ SPECIFIC="${SVC_ACCOUNT_URI}FOOBAR-compute@developer.gserviceaccount.com/"
$ curl ${DEFAULT} -H "Metadata-Flavor: Google"
aliases
email
scopes
token
$ curl ${SPECIFIC} -H "Metadata-Flavor: Google"
aliases
email
scopes
token
$ curl "${DEFAULT}aliases" -H "Metadata-Flavor: Google"
default
$ curl "${DEFAULT}email" -H "Metadata-Flavor: Google"
FOOBAR-compute@developer.gserviceaccount.com
$ curl "${DEFAULT}scopes" -H "Metadata-Flavor: Google"
https://www.googleapis.com/auth/devstorage.read_only
https://www.googleapis.com/auth/logging.write
https://www.googleapis.com/auth/userinfo.email
$ curl "${DEFAULT}token" -H "Metadata-Flavor: Google"
{"access_token":"ya29.REDACTED","expires_in":3599,"token_type":"Bearer"}

[dhermes]

@jgeewax Is there anyone other than @ludoch that might know if a GCE equivalent of app_identity.sign_blob() exists?

[jgeewax]

@ajessup ?

[ajessup]

What's the use-case?

[dhermes]

Creating a signed URL for a cloud storage blob. With a normal service account (and key), this is done with crypto from the key. On App Engine, the signing functionality is surface via app_identity.sign_blob() (as mentioned above).

[julienmarcbrown]

To use GCE and Cloud storage

1. Go to your Project credentials
2. If you don't already have a service account and json/p12 key, you can create a new one then generate .p12 or .json file.
3. Copy it to your compute engine
4. You can use classmethod from_service_account_p12 or classmethod from_service_account_json respectively and copy it to your compute engine directory. For the email parameter in the classmethod from_service_account_p12 you will use the service account email that you've generated the key from. You will include the link to the file in either of the classmethod parameters.
5. You then have to assign read-only or ownership to whatever file/directory you're trying to access in the cloud storage browser or through command line
6. You will then have to establish the bucket (if you're only working within one bucket) with gcloud.storage.bucket.Bucket(name='BucketName')
7. To initialize a blob object you will have to use

blob = gcloud.storage.blob.Blob(blobname, bucket=bucket, chunk_size=None)

I wrote this kind of fast, I'm not sure if i've missed anything.

[dhermes]

UPDATE: I should make it more clear that I really appreciate the attempt to help out 👍

---

@julienmarcbrown Thanks for the comment but it's missing the point of this issue. (Note that I'm the author of this issue and of the methods you suggest using.)

This question is about GCE service accounts. These are service accounts attached to a GCE instance and they provide methods (as I mention above) via the GCE metadata server:

http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/aliases
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/email
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/scopes
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

[JacobHayes]

This would be really helpful to have. To create/modify objects we can just auth with oauth2client.client.GoogleCredentials.get_application_default on any environment. But to get signed urls, we need to also have a .p12 which is incredibly inconvenient.

Has there been any progress on this?

[dhermes]

I don't think there are any plans for implementing this with Compute Engine service accounts.

As for .p12, is there a reason you aren't using a JSON service account key?

[JacobHayes]

Not particularly, but I was hoping to avoid busing around key files when we can already do so much with oauth2client.client.GoogleCredentials from get_application_default. It seems odd to be able to delete objects, make them public, etc but not be able to make a temporary access url. I understand the technical differences between the token we get with oauth and an actual private key, just seems like there would be a way without requiring private keys.

What are the benefit of the JSON service account key?

[dhermes]

The JSON service account key uses the PEM format, which is much simpler than PKCS12. Google essentially recommends that you use a JSON key.

The reason the signed URL differs from the access token is that it is generated completely on the client, using the private key to sign some content. This signed content is used directly from the signed URL:

https://storage.googleapis.com/{BUCKET-NAME}/{BLOB-NAME}?
Expires={SECONDS-SINCE-EPOCH}&
GoogleAccessId={SVC-ACCOUNT-EMAIL}&
Signature={SOME-BASE64-BYTES}

@thobrla Any thoughts?