hope you're doing well.
I'm writing an API using in Go that can works similarly to the terraform provider protocol
So I already have two endpoints working on my local machine over HTTPS:
https://myapi:9000/v1/provider/:namespace/:type/versions
https://myapi:9000/v1/provider/:namespace/:type/:version/download/:os/:arch
Let's say for example these full endpoint:
https://myapi:9000/v1/provider/myprovider/custom/versions
https://myapi:9000/v1/provider/myprovider/custom/0.1.0/download/linux/amd64
So I have the next .tf.json file:
{
"module": {
"linux": {
"source": "myapi:9000/v1/module/mymodule/custom",
"version": "0.1.2"
}
}
}
That uses this two files
provider.tf.json
provider "mycustomprovider" {
username = "abc"
password = "def"
host = "yjk"
}
versions.tf.json
terraform {
required_providers {
mycustomprovider = {
source: "myapi:9000/v1/myprovider/custom",
version: "0.1.0"
}
}
required_version = ">=1.0.2"
}
Then I simply run: terraform init to get my assets.
So, when I'm getting my custom module, that works fine.
Output (module download):
Initializing modules...
Downloading myapi:9000/mymodule/custom/gnu 0.1.2 for linux...
- linux in .terraform/modules/linux
Initializing the backend...
But when I'm getting my provdier I have this error:
Initializing provider plugins...
- Finding myapi:9000:9000/myprovider/custom versions matching "0.1.0"...
- Installing myapi:9000:9000/myprovider/custom v0.1.0...
╷
│ Error: Failed to install provider
│
│ Error while installing myapi:9000/myprovider/custom v0.1.0: error checking signature:
│ openpgp: invalid data: tag byte does not have MSB set
So, my provider versions endpoint is working. That's why terraform is able to recognize the version of my provider.
The problem should be on my download endpoint.
Before talking about this endpoint I want to add some context, I'm running a S3 client using localstack and exposing it through ngrok. These two things work and I able to upload or download files without problem.
Terraform custom providers should have three files (as I understand):
the provider in zip format (like in their example)
a provider_SHA256SUMS file with the shasums of each provider zip file (in this case I'm only have one)
a provider_SHA256SUMS.sig that is uses to recognize the integrity of the provider_SHA256SUMS file.
So to get these files I'm running these commands:
provider_SHA256SUMS
$ sha256sum 0.1.0.zip > 0.1.0_SHA256SUMS
provider_SHA256SUMS
$ gpg --gen-key # generating a new key
$ gpg --armor --output 0.1.0_SHA256SUMS.sig --detach-sig 0.1.0_SHA256SUMS
The response with my endpoint is something like this. (the gpg info is just sample data, no real warning here.)
{
"protocols": [
"5.0"
],
"os": "linux",
"arch": "amd64",
"filename": "0.1.0.zip",
"download_url": "https://d4f6-186-84-89-138.ngrok.io/terraform/v1/providers/myprovider/custom/0.1.0.zip",
"shasums_url": "https://d4f6-186-84-89-138.ngrok.io/terraform/v1/providers/myprovider/custom/0.1.0_SHA256SUMS",
"shasums_signature_url": "https://d4f6-186-84-89-138.ngrok.io/terraform/v1/providers/myprovider/custom/0.1.0_SHA256SUMS.sig",
"shasum": "1dd61b508aad0d65b32c71159775e409fd618adc5ba945cc2eebb42f29e085d3",
"signing_keys": {
"gpg_public_keys": [
{
"key_id": "9F21EA3C1C9F793C",
"ascii_armor": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nmQGNBGEuvh8BDADyT3JLsSqnSLk0I2MrFPJgCvCYpPFsFJgfDx2EwL7TGGDeslaN\ndoOq05X+9vKyM6qQ1jQmpfS5dzsQIsHtUlsU0nphS21ZvYm10aZUt3dXxlMwu2Is\nSO+q8O4WSMXclIsBUyhzP6TMQ7nISXHundVx7b/S/bEYucOIMeYmqg1PKId55U4G\n7y/8W6mmzX6NvF97fRyN37fBqvx8q2SxT5iB3C2Sbfd7i/sMvjC0tQOBv1EKh3RN\ncElP5NlJbv58Ysz+UTU21EPkkvPH4pLuUcB9/0uwzi5y/268EWTy3+UlWnoh12ds\nESZFgijzUsdvOmCOZkdd5X1Radzr+6VKVXHHIprKgO5AlvjFoLQK0NzzMiXjhyUF\nk9plo7kET4dy9ztySJYutx5eNMJInF5mYKNdH3H36ThXAIPptAu8WJjCtYok78C5\nilpv7cTiM9F5g7SlxnKU+xFmbzhSnYxth9DEhrO9ufliT1Df76iuqpc8B79sUtUH\nWvf7QIgkL6HtL5MAEQEAAbQYbXlhcGkgPG15YXBpQGN1c3RvbS5jb20+iQHUBBMB\nCgA+FiEEbk3GwdLc9Ypn5Wg9nyHqPByfeTwFAmEuvh8CGwMFCQPCZwAFCwkIBwIG\nFQoJCAsCBBYCAwECHgECF4AACgkQnyHqPByfeTzfQgwAyqcGJFbU2zN45F/2ECBs\nE6vYbfk9qRXpvU6PWodE0t5sqcxY2Oz0r29OGaW5mDyZRE+zRGir4yQki3RqI6vY\nh66uTWMybUV6qipv3qXHIqbSn3H/ss4Tuf9C2//Pz/LpMKiMiJilpXyCy8F8l504\nEsm+PU3CtNioNZCkoeH6kJWkjXDGQWQK58R4SFRfHcJMa03+gyPgv5Ba593/zGqh\nl2GmmwbAJHcnSH1EBAulcd48nQCMOYuvIqa40CDOhcz+rIlqivvP6KVX+qmRVmaV\nY1u391a40wfaRomuk46JCKFQVeElAZ4tac8UaOv2x0GOBzIw7/1CwulN2VvojiEJ\nVj0Q6sZ/K7+dU4H7NLQ1aIN+Vv3t7VIISu3wzraCT5c1aduH4YLio83W9rS4EcRj\nHmej5JG4B16HOMMrM1caq+cVPyymCzblEShplCdmQ7qcYOqvRYW8ewVPNqWJSTiR\nC/Kpq1N8OOKdG0Th8ja4jfRkfexloCdUOlSOKktK8uU+uQGNBGEuvh8BDADOMUX3\nxatbgt4sArBKNlnZWrZZCRFHxzeGaZsY9EsNY6D722iGoU40iYs6ky08bOQT/g8O\nFSooA6DKNhxVCM/r99rsiYrNIzT9s/ywKmUb6JipgAiGpd5W9lBAB/u6pQ039ni2\nQI+5cYZ+8i6v2b6oOGdnym8p2K14O+keAh7Z6aOnpb8YIq3B7khtcO+oHvp820sB\nOa0hlMs39qQHkG70ybAe0HdcZAhXVVSmrN6EdDZ8SZmRSAVbiv84a9t5b8swNhxZ\nT3csxqdAWbz3GOCIUmmaJUYOdGYLwAc2BnsRyxzNq66H962uK9hygrDrjJSpNnxU\n/VdXcRxYZcLqUHGPMds/gqwr/30JmXlkPqbG/3v4D+wy5OFsr+uquK4helqhJdQ9\nfOWrMyUxShZhZ476YURn1VbaF4a5x5zi2OBSxYK9VjSfAedisMtvsRIxOMgU0eXT\nNkRaTBoQTX2ZiVjy0fwVeHgNIuPsszQRokRZ2zFttC+tU5x/ffayBU3qZhUAEQEA\nAYkBvAQYAQoAJhYhBG5NxsHS3PWKZ+VoPZ8h6jwcn3k8BQJhLr4fAhsMBQkDwmcA\nAAoJEJ8h6jwcn3k8QU4MAJetwC4o/F9m0tJKO6DYqX5bsnGlp1u3oyG0ATvSvT9E\nBTxbQlpcIOrJ16Be/92SmfVaGbbqWywqjkNgK7s08Zbbk7WONZyAg8NR5/b5Cgi9\ncJrR73dbDnijvhjDkAAn414+M57DG65tPt1vlXDqa8LSQobDdszn1i/ugvqxqj1y\n6NmFvVPxor67n9r67Iq4PzWF3WK7tosPUaTbFczbS2xS4sINPCEddb2Ima5cixL0\nh2pni/jonYo4RCWmUvpMx48CevgXFCzWOGdaOSI75MklcaH4IBe2EFaCbN3IUMlA\nHI2TOuR0KXsX0R3jzmDzVJkXaXWMqPjcFlxvXuMTE4ooI6DiBN7+2xAqfYOURmy1\nwjfWwCVR3OaPY2cGvxWPnIz2mtKjhRIaYwfzDVdR5vlSU/YwkJUv11P6Y8YPX7jw\nRYVFtTkd7qghjvWMBMpABTxYWxvd74EgUUnYOfoei97nKOnb3loj+XdoZeGCmyL7\nCZWzoNTMkeFkob1UkxIe+Q==\n=wZDI\n-----END PGP PUBLIC KEY BLOCK-----"
}
]
}
}
The shasum prop in the response should be a shasum corresponding to the provider zip file, it can also be found inside the provider_SHA256SUMS file.
To get the key_id and ascii_armor props I'm running this commands:
$ gpg --list-secret-keys --keyid-format=long ## key_id
$ gpg --armor --export <MY_KEYID> > public.gpg ## export to public key to base64
$ cat public.gpg | sed -E ':a;N;$!ba;s/\r{0,1}\n/\\n/g' ## one-lined ascii_armor
Short Question
What am I doing wrong with the gpg key to get this error? Am I lacking of steps, or doing everything wrong?
openpgp: invalid data: tag byte does not have MSB set
----Update----
This is how I'm building the response in Go:
I'm not putting the full code because I belive that the problem is with the KeyID and ASCIIArmor atributes.
As you can see: Shasum, KeyID and ASCIIArmor are hardcoded.
response := ProviderDownloadResponse{
Protocols: protocols,
Os: os,
Arch: arch,
Filename: filename,
DownloadURL: downloadURL,
ShasumsURL: SHASUMsURL,
ShasumsSignatureURL: SHASUMSSignatureURL,
Shasum: "1dd61b508aad0d65b32c71159775e409fd618adc5ba945cc2eebb42f29e085d3",
SigningKeys: SigningKeys{
GpgPublicKeys: []GPGPublicKey{
{
KeyID: "9F21EA3C1C9F793C",
ASCIIArmor: "-----BEGIN PGP PUBLIC KEY BLOCK-----\n\nmQGNBGEuvh8BDADyT3JLsSqnSLk0I2MrFPJgCvCYpPFsFJgfDx2EwL7TGGDeslaN\ndoOq05X+9vKyM6qQ1jQmpfS5dzsQIsHtUlsU0nphS21ZvYm10aZUt3dXxlMwu2Is\nSO+q8O4WSMXclIsBUyhzP6TMQ7nISXHundVx7b/S/bEYucOIMeYmqg1PKId55U4G\n7y/8W6mmzX6NvF97fRyN37fBqvx8q2SxT5iB3C2Sbfd7i/sMvjC0tQOBv1EKh3RN\ncElP5NlJbv58Ysz+UTU21EPkkvPH4pLuUcB9/0uwzi5y/268EWTy3+UlWnoh12ds\nESZFgijzUsdvOmCOZkdd5X1Radzr+6VKVXHHIprKgO5AlvjFoLQK0NzzMiXjhyUF\nk9plo7kET4dy9ztySJYutx5eNMJInF5mYKNdH3H36ThXAIPptAu8WJjCtYok78C5\nilpv7cTiM9F5g7SlxnKU+xFmbzhSnYxth9DEhrO9ufliT1Df76iuqpc8B79sUtUH\nWvf7QIgkL6HtL5MAEQEAAbQYbXlhcGkgPG15YXBpQGN1c3RvbS5jb20+iQHUBBMB\nCgA+FiEEbk3GwdLc9Ypn5Wg9nyHqPByfeTwFAmEuvh8CGwMFCQPCZwAFCwkIBwIG\nFQoJCAsCBBYCAwECHgECF4AACgkQnyHqPByfeTzfQgwAyqcGJFbU2zN45F/2ECBs\nE6vYbfk9qRXpvU6PWodE0t5sqcxY2Oz0r29OGaW5mDyZRE+zRGir4yQki3RqI6vY\nh66uTWMybUV6qipv3qXHIqbSn3H/ss4Tuf9C2//Pz/LpMKiMiJilpXyCy8F8l504\nEsm+PU3CtNioNZCkoeH6kJWkjXDGQWQK58R4SFRfHcJMa03+gyPgv5Ba593/zGqh\nl2GmmwbAJHcnSH1EBAulcd48nQCMOYuvIqa40CDOhcz+rIlqivvP6KVX+qmRVmaV\nY1u391a40wfaRomuk46JCKFQVeElAZ4tac8UaOv2x0GOBzIw7/1CwulN2VvojiEJ\nVj0Q6sZ/K7+dU4H7NLQ1aIN+Vv3t7VIISu3wzraCT5c1aduH4YLio83W9rS4EcRj\nHmej5JG4B16HOMMrM1caq+cVPyymCzblEShplCdmQ7qcYOqvRYW8ewVPNqWJSTiR\nC/Kpq1N8OOKdG0Th8ja4jfRkfexloCdUOlSOKktK8uU+uQGNBGEuvh8BDADOMUX3\nxatbgt4sArBKNlnZWrZZCRFHxzeGaZsY9EsNY6D722iGoU40iYs6ky08bOQT/g8O\nFSooA6DKNhxVCM/r99rsiYrNIzT9s/ywKmUb6JipgAiGpd5W9lBAB/u6pQ039ni2\nQI+5cYZ+8i6v2b6oOGdnym8p2K14O+keAh7Z6aOnpb8YIq3B7khtcO+oHvp820sB\nOa0hlMs39qQHkG70ybAe0HdcZAhXVVSmrN6EdDZ8SZmRSAVbiv84a9t5b8swNhxZ\nT3csxqdAWbz3GOCIUmmaJUYOdGYLwAc2BnsRyxzNq66H962uK9hygrDrjJSpNnxU\n/VdXcRxYZcLqUHGPMds/gqwr/30JmXlkPqbG/3v4D+wy5OFsr+uquK4helqhJdQ9\nfOWrMyUxShZhZ476YURn1VbaF4a5x5zi2OBSxYK9VjSfAedisMtvsRIxOMgU0eXT\nNkRaTBoQTX2ZiVjy0fwVeHgNIuPsszQRokRZ2zFttC+tU5x/ffayBU3qZhUAEQEA\nAYkBvAQYAQoAJhYhBG5NxsHS3PWKZ+VoPZ8h6jwcn3k8BQJhLr4fAhsMBQkDwmcA\nAAoJEJ8h6jwcn3k8QU4MAJetwC4o/F9m0tJKO6DYqX5bsnGlp1u3oyG0ATvSvT9E\nBTxbQlpcIOrJ16Be/92SmfVaGbbqWywqjkNgK7s08Zbbk7WONZyAg8NR5/b5Cgi9\ncJrR73dbDnijvhjDkAAn414+M57DG65tPt1vlXDqa8LSQobDdszn1i/ugvqxqj1y\n6NmFvVPxor67n9r67Iq4PzWF3WK7tosPUaTbFczbS2xS4sINPCEddb2Ima5cixL0\nh2pni/jonYo4RCWmUvpMx48CevgXFCzWOGdaOSI75MklcaH4IBe2EFaCbN3IUMlA\nHI2TOuR0KXsX0R3jzmDzVJkXaXWMqPjcFlxvXuMTE4ooI6DiBN7+2xAqfYOURmy1\nwjfWwCVR3OaPY2cGvxWPnIz2mtKjhRIaYwfzDVdR5vlSU/YwkJUv11P6Y8YPX7jw\nRYVFtTkd7qghjvWMBMpABTxYWxvd74EgUUnYOfoei97nKOnb3loj+XdoZeGCmyL7\nCZWzoNTMkeFkob1UkxIe+Q==\n=wZDI\n-----END PGP PUBLIC KEY BLOCK-----",
},
},
},
}
x/crypto/openpgp that is used by terraform does not support reading armored messages, see issue, this is where the error comes from.
shasums_signature_url filed doc mentions:
binary, detached GPG signature
Also, see Manually preparing a release doc
which is a valid GPG binary (not ASCII armored) signature
So you should try signing without --armor flag.
Related
I am working with Google Cloud in Go and following this article by John Hanley:
https://www.jhanley.com/google-cloud-improving-security-with-impersonation/
and mashed it with this SO answer:
How to authenticate Google APIs (Google Drive API) from Google Compute Engine and locally without downloading Service Account credentials?
The credentials are successfully saved to, "application_default_credentials.json":
Notice: "type": "impersonated_service_account"
{
"delegates": [],
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[sa#example-2021.iam.gserviceaccount.com]:generateAccessToken",
"source_credentials": {
"client_id": "...apps.googleusercontent.com",
"client_secret": "...",
"refresh_token": "...",
"type": "authorized_user"
},
"type": "impersonated_service_account"
}
My code which produces an unknown credential type: "impersonated_service_account" error:
package main
import (
...
"cloud.google.com/go/storage"
"golang.org/x/oauth2"
"google.golang.org/api/docs/v1"
"google.golang.org/api/drive/v3"
"google.golang.org/api/impersonate"
"google.golang.org/api/option"
...
)
var Config.GoogleServiceAccount string = "sa#example-2021.iam.gserviceaccount.com"
func main(){
_ = getTokenAsImpersonator()
}
// From: https://pkg.go.dev/google.golang.org/api/impersonate#example-CredentialsTokenSource-ServiceAccount
func getTokenAsImpersonator() oauth2.TokenSource {
ctx := context.Background()
// Base credentials sourced from ADC or provided client options.
ts, err := impersonate.CredentialsTokenSource(ctx, impersonate.CredentialsConfig{
TargetPrincipal: Config.GoogleServiceAccount,
Scopes: []string{"https://www.googleapis.com/auth/cloud-platform"},
// Delegates: []string{"bar#project-id.iam.gserviceaccount.com"},
})
if err != nil {
log.Fatal(err)
}
return ts
}
The 'unknown credential type: "impersonated_service_account"' error:
google: error getting credentials using GOOGLE_APPLICATION_CREDENTIALS environment variable: unknown credential type: "impersonated_service_account"
Have I done something wrong or is this a bug?
UPDATE
Answering John's questions from the comments:
1.
a) What is the value of the environment variable GOOGLE_APPLICATION_CREDENTIALS?
GOOGLE_APPLICATION_CREDENTIALS=/Users/x/.config/gcloud/application_default_credentials.json
b) What command did you use to generate application_default_credentials.json?
gcloud auth application-default login --scopes=https://www.googleapis.com/auth/drive,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/accounts.reauth,openid --impersonate-service-account=[sa#example-2021.iam.gserviceaccount.com]
Response:
Credentials saved to file: [/Users/x/.config/gcloud/application_default_credentials.json]
c)Which OS and version?
MacOS 10.13.6
d)gcloud --version?
Google Cloud SDK 343.0.0
app-engine-go
app-engine-python 1.9.91
bq 2.0.69
cloud-datastore-emulator 2.1.0
core 2021.05.27
gsutil 4.62
If you can create a minimum example ...
I have updated the example code above.
At some point I had used the CLI to impersonate an account:
gcloud config set auth/impersonate_service_account <service account>
Then later on when trying to use the application default credentials command it wraps your credentials with the service account credentials.
gcloud auth application-default login
What you end up with is a file that looks like this:
{
"delegates": [],
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/example#example-project.iam.gserviceaccount.com:generateAccessToken",
"source_credentials": {
"client_id": "123abc.apps.googleusercontent.com",
"client_secret": "XXXXXXXXX",
"refresh_token": "XXXXXXXXX",
"type": "authorized_user"
},
"type": "impersonated_service_account"
}
This appears to cause a lot of problems with third party services such as terraform.
What is strange is that Terraform is just making API calls to Google using Google SDKs, so really its something to do with Google.
You need to remove the impersonation:
gcloud config unset auth/impersonate_service_account
And then run the application default credential command again:
gcloud auth application-default login
Now if you check your file it should look like this:
{
"client_id": "XXXXXXXXX",
"client_secret": "XXXXXXXXX",
"quota_project_id": "example-project",
"refresh_token": "XXXXXXXXXX",
"type": "authorized_user"
}
I was hitting the same issue when I was trying to impersonate an account so I could run Terraform commands as a service account instead of my personal account but it doesn't like that.
EDIT: Rereading you question it sounds like you're in the same boat as me. We want to use service accounts without physically downloading the keys. This is even mentioned by Google as best practice. But doing so is causing issues with their own SDKs.
I had the same issue running GCP Terraform provider tests. You can specify the Service Account Terraform have to impersonate setting the env variable GOOGLE_IMPERSONATE_SERVICE_ACCOUNT (documentation).
Configuration steps:
export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=SERVICE_ACCOUNT#PROJECT_ID.iam.gserviceaccount.com
gcloud auth application-default login
I tried what seemed to be the straight-forward approach, and added a Package resource in my node configuration for the MongoDB MSI. I got the following error: "Could not get the https stream for file".
Here's the package configuration I tried:
package MongoDB {
Name = "MongoDB 3.6.11 2008R2Plus SSL (64 bit)"
Path = "https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-3.6.11-signed.msi"
ProductId = "88F7AA23-BDD2-4EBE-9985-EBB5D2E23E83"
Arguments = "ADDLOCAL=`"all`" SHOULD_INSTALL_COMPASS=`"0`" INSTALLLOCATION=`"C:\MongoDB\Server\3.6`""
}
(I had $ConfigurationData references in there, but substituted for literals for simplicity)
I get the following error:
Could not get the https stream for file
Possible TLS version issue? I found that Invoke-WebRequest needed the following to get it to work with that same mongo download URL. Is there a way to do this with the package resource?
[Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"
Using nmap to interrogate both nodejs.org and fastdl.mongodb.org (which is actually on cloudfront) it was indeed true that TLS support differed. Node still supports TLS version 1.0, which so happens to work with PowerShell. But MongoDB's site only supports TLS versions 1.1 or 1.2.
As I mentioned in my question, I suspected that setting the .Net security protocol work, and indeed it does. There's no way to add arbitrary script to the DSC package resource, so I needed to make a script block just to run this code, and have the package resource depend on it.
This is what I got to work:
Node $AllNodes.Where{$_.Role -contains 'MongoDBServer'}.NodeName {
Script SetTLS {
GetScript = { #{ Result = $true } }
SetScript = { [Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls" }
TestScript = { $false } #Always run
}
package MongoDB {
Ensure = 'Present'
Name = 'MongoDB 3.6.11 2008R2Plus SSL (64 bit)'
Path = 'https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-3.6.11-signed.msi'
ProductId = ''
Arguments = 'ADDLOCAL="all" SHOULD_INSTALL_COMPASS="0" INSTALLLOCATION="C:\MongoDB\Server\3.6"'
DependsOn = '[Script]SetTLS'
}
...
When creating a Microsoft Bot Framework 4 project - the Startup.cs has the following code which can be uncommented.
const string StorageConfigurationId = "<NAME OR ID>";
var blobConfig = botConfig.FindServiceByNameOrId(StorageConfigurationId);
if (!(blobConfig is BlobStorageService blobStorageConfig))
{
throw new InvalidOperationException($"The .bot file does not contain an blob storage with name '{StorageConfigurationId}'.");
}
This code handles a way to configure an Azure Storage Account via Json Configuration.
However the project lacks an example on what the config Json looks like for the "is BlobStorageService" to work.
I have done various tries and searched for examples but cannot make it work.
Has anyone got the nailed?
Got it working using this json...
{
"type": "blob", //Must be 'blob'
"name": "<NAME OF CONFIG - MUST BE UNIQUE (CAN BE ID)>",
"connectionString": "<COPY FROM AZURE DASHBOARD>",
"container": "<NAME OF CONTAINER IN STORAGE>"
}
In DCOS, I want to deploy a mesos container with a self-defined image which stored in a local secure docker registry, and it has been secured by CA (not username and password!)
The json is
{
"id": "/gpu-tflinker",
"cmd": "while [ true ] ; do nvidia-smi; sleep 5; done",
"cpus": 0.1,
"mem": 1024,
"gpus": 1,
"instances": 1,
"constraints": [
[
"hostname",
"CLUSTER",
"10.140.0.22"
]
],
"container": {
"type": "MESOS",
"docker": {
"image": "tflinker:test-gpu",
"credential": null
}
}
}
The above json failed to run on marathon, and there is no content on mesos's stderr and stdout file, on mesos-agent log, the error message is :
E0721 05:01:57.726367 22498 slave.cpp:3976] Container 'e2c68720-0fb7-41bc-9d3b-a2b5e4793816' for executor 'gpu-t
flinker.b6f96725-6dd1-11e7-ba5d-0242b2c758c0' of framework 1079aaea-6dde-4dc1-8990-d926a895de78-0000 failed to s
tart: Unexpected HTTP response '401 Unauthorized' when trying to get the manifest
W0721 05:01:57.726478 22497 composing.cpp:541] Container 'e2c68720-0fb7-41bc-9d3b-a2b5e4793816' is already destr
oyed
I0721 05:01:57.726583 22497 slave.cpp:4082] Executor 'gpu-tflinker.b6f96725-6dd1-11e7-ba5d-0242b2c758c0' of fram
ework 1079aaea-6dde-4dc1-8990-d926a895de78-0000 has terminated with unknown status
I0721 05:01:57.726603 22497 slave.cpp:4193] Cleaning up executor 'gpu-tflinker.b6f96725-6dd1-11e7-ba5d-0242b2c75
8c0' of framework 1079aaea-6dde-4dc1-8990-d926a895de78-0000
I0721 05:01:57.726794 22497 slave.cpp:4281] Cleaning up framework 1079aaea-6dde-4dc1-8990-d926a895de78-0000
so it seems mesos failed to fetch the docker image. I've configed CA file for dockerd(move ca files to /etc/docker/certs.d/), so I can 'docker pull' the image to local machine, but I am not sure how to config CA file for mesos~
in mesos-agent configurations, there exist a item --docker_config=VALUE, but it seems this item can only be used for username/password secured registry, I don't know how to config for CA secured registry.
anybody can help me out?! thanks!
I think CA file is just for encryption. you will need username and password in ca file way I think.
In my way, I put auth file into the container to authorize my private registry.
I wrote a web service for downloading the auth file
xxx.tar.gz(format: .docker/config.json in the tar.gz)
in the config.json, {"auths": {"test.com:6999": {"auth": "(username:password) [base64 encode]"}}} like {"auths": {"test.com:6999": {"auth": "Y2NjOjEyMw=="}}}
use Mesos uris to download auth files prepared into the containers. then, it would authorized.
"uris": [
"http:your download url"
]
I am trying to install Parse Dashboard on AWS. The public directory works but the /apps directory is blank.
When looking at the logs I see
> parse-dashboard#1.0.14 start /var/app/current
> node ./Parse-Dashboard/index.js
Your config file contains invalid JSON. Exiting.
I am deploying the parse-dashboard from github. And I have entered in the values in the parse-dashboard-config.json that match the keys on parse.com.
This is the JSON that I am using
{
"apps": [
{
"serverURL": "xxxxxxxxxxxxxxxxxxxxxxx/parse",
"appId": "xxxxxxxxxxxxxxxxxxxxxxxxxxx",
"masterKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"appName": "xxxxxxxxxxxxxx"
}
],
"iconsFolder": "icons"
}
In index.js the log is being generated by
129 if (error instanceof SyntaxError) {
130 console.log('Your config file contains invalid JSON. Exiting.');
131 process.exit(1);
132 }
For anyone who's looking for an answer, the parse-dashboard-config.json file should be in the your current folder where you're running
parse-dashboard --config parse-dashboard-config.json