I am trying to get a list of all GCP compute instances in all projects that I have access to. However, there is one project that keeps giving me a "permission denied" answer, and I cannot figure out why...
In order to authenticate, I have created a service account in one of the projects (let's call it sa1, in project_a). Then, at the organization level, I have given that SA the Compute Admin role. And everything works fine, for all of the projects, except one (let's call this non-working one project_x).
Now I am using the computeService.Instances.AggregatedList call, which according to the docs needs the compute.instances.list permission. If I go to IAM&Admin/Policy Troubleshooter and check, I find out that it should work:
Access granted for API call for sa1#project_a, compute.instances.list, projects/project_x.
If this outcome was unexpected, contact your support team for further assistance.
However, if I try to actually run the code, as soon as I hit that project I get:
googleapi: Error 403: Permission denied on resource project project_x, forbidden
The code is written in Go, based on the sample in the docs. The actual call I am using looks like this:
func getInstances ( ctx context.Context, cs *compute.Service, projectName string) {
req := cs.Instances.AggregatedList(projectName)
err := req.Pages(ctx, func(page *compute.InstanceAggregatedList) error {
for name, instancesScopedList := range page.Items {
if len(instancesScopedList.Instances) > 0 {
fmt.Printf("===== %v =====\n", name)
for _, instance := range instancesScopedList.Instances {
fmt.Printf("-- %#v, %#v\n", instance.Name, instance.NetworkInterfaces[0].AccessConfigs[0].NatIP)
}
}
}
return nil
})
if err != nil {
fmt.Println("Error getting list!")
log.Fatal(err)
}
}
I honestly have no idea what could be causing this, and no idea how to troubleshoot further. Can anyone point me in the right direction?
Oh, and if it matters - there are actually no compute engine instances in project_x. But this should not be a problem - I have other projects with no instances, and I simply get an empty list from them.
Related
When ever I try to deploy my smart contract to test, I receive an error indicating "transaction type not supported". Below is the source code. I'm trying to deploy my simple smart contract using abigen's Go bindings.
Versions:
go1.16.7
Solidity 0.8.9+commit.e5eed63a.Darwin.appleclang
Solidity Source code. I've tested this in Remix and it has worked everytime:
contract SendMSG {
function Send(address sendTo, bytes calldata message) public {
OtherContract other = OtherContract(sendTo);
other.send(message);
}
}
This is the contract I'm using ignore syntax errors as it may be human error while anonymizing.
I then run this line to develop the abi bindings and put them in the right place. I can confirm this works as the go file is always created:
abigen --sol ../../contracts/Contract.sol --pkg Contract --out Contract.go
Go Code. I believe there shouldn't be any issues. I'm using a simulated backend/blockchain for testing:
package Contract
import (
"testing"
"github.com/ethereum/go-ethereum/accounts/abi/bind/backends"
"github.com/ethereum/go-ethereum/accounts/abi/bind"
"github.com/ethereum/go-ethereum/crypto"
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/common"
"math/big"
)
// Test inbox contract gets deployed correctly
func TestMain(t *testing.T) {
//Setup simulated block chain
key, _ := crypto.GenerateKey()
auth := bind.NewKeyedTransactor(key)
alloc := make(core.GenesisAlloc)
alloc[auth.From] = core.GenesisAccount{Balance: big.NewInt(133700000)}
gasLimit := 300000
sim := backends.NewSimulatedBackend(alloc, gasLimit)
//Deploy contract
address, _, _, err := DeploySameBindings(
auth,
sim,
)
// commit all pending transactions
blockchain.Commit()
if err != nil {
t.Fatalf("Failed to deploy the contract: %v", err)
}
}
Always, it gives the same err, "transaction type not supported". I know the line where the error origin [GitHub]. From there, maybe I didn't set a payment mechanism? But all the tutorials I've seen didn't include one and if anyone could provide a guide as to how to do that.
Thank you.
This was stupid.
Geth updated their code and there weren't any tutorials so for anyone hoping to run a simulated background here's the answer:
You have to set the gas price manually. Taking this on after defining client and auth fixes it.
gasPrice, err := client.SuggestGasPrice(context.Background())
auth.GasPrice=gasPrice
The official Go documentation on the datastore package (client library for the GCP datastore service) has the following code snippet for demonstartion:
type Entity struct {
Value string
}
func main() {
ctx := context.Background()
// Create a datastore client. In a typical application, you would create
// a single client which is reused for every datastore operation.
dsClient, err := datastore.NewClient(ctx, "my-project")
if err != nil {
// Handle error.
}
k := datastore.NameKey("Entity", "stringID", nil)
e := new(Entity)
if err := dsClient.Get(ctx, k, e); err != nil {
// Handle error.
}
old := e.Value
e.Value = "Hello World!"
if _, err := dsClient.Put(ctx, k, e); err != nil {
// Handle error.
}
fmt.Printf("Updated value from %q to %q\n", old, e.Value)
}
As one can see, it states that the datastore.Client should ideally only be instantiated once in an application. Now given that the datastore.NewClient function requires a context.Context object does it mean that it should get instantiated only once per HTTP request or can it safely be instantiated once globally with a context.Background() object?
Each operation requires a context.Context object again (e.g. dsClient.Get(ctx, k, e)) so is that the point where the HTTP request's context should be used?
I'm new to Go and can't really find any online resources which explain something like this very well with real world examples and actual best practice patterns.
You may use any context.Context for the datastore client creation, it may be context.Background(), that's completely fine. Client creation may be lengthy, it may require connecting to a remote server, authenticating, fetching configuration etc. If your use case has limited time, you may pass a context with timeout to abort the operation. Also if creation takes longer than the time you have, you may use a context with cancel and abort the mission at your will. These are just options which you may or may not use. But the "tools" are given via context.Context.
Later when you use the datastore.Client during serving (HTTP) client requests, then using the request's context is reasonable, so if a request gets cancelled, then so will its context, and so will the datastore operation you issue, rightfully, because if the client cannot see the result, then there's no point completing the query. Terminating the query early you might not end up using certain resources (e.g. datastore reads), and you may lower the server's load (by aborting jobs whose result will not be sent back to the client).
I want to query GSuite Admin SDK Directory API to return all users in a group, in Go, and authenticated as a GCP service account (the script will be executed in a Google Compute Engine VM or as a Google Cloud Function).
The service account I use (let's call it my-service-account#my-project.iam.gserviceaccount.com) have been granted necessary scopes in GSuite :
https://www.googleapis.com/auth/admin.directory.group.member.readonly
https://www.googleapis.com/auth/admin.directory.group.member
https://www.googleapis.com/auth/admin.directory.group.readonly
https://www.googleapis.com/auth/admin.directory.group
At my disposal, I also have a GSuite Admin account (let's call it my-admin#my-domain.com). This account will be used for delegation (see the docs on delegation).
I am able to return all users in a group with the following code (based on the code in the link provided above, and I have removed most of error handling to make the code shorter) :
package main
import (
"io/ioutil"
"log"
"os"
"golang.org/x/net/context"
"golang.org/x/oauth2/google"
admin "google.golang.org/api/admin/directory/v1"
)
func main() {
srv := createAdminDirectoryService(
os.Getenv("SERVICE_ACCOUNT_FILE_PATH"),
os.Getenv("GSUITE_ADMIN_USER_EMAIL"),
)
members := listUsersInGroup(srv, os.Args[1])
log.Println(members)
}
func createAdminDirectoryService(serviceAccountFilePath,
gsuiteAdminUserEmail string) *admin.Service {
jsonCredentials, _ := ioutil.ReadFile(serviceAccountFilePath)
config, _ := google.JWTConfigFromJSON(
jsonCredentials,
admin.AdminDirectoryGroupMemberReadonlyScope,
)
config.Subject = gsuiteAdminUserEmail
ctx := context.Background()
client := config.Client(ctx)
srv, _ := admin.New(client)
return srv
}
func listUsersInGroup(srv *admin.Service, groupEmail string) []string {
members, err := srv.Members.List(groupEmail).Do()
if err != nil {
log.Fatal(err)
}
membersEmails := make([]string, len(members.Members))
for i, member := range members.Members {
membersEmails[i] = member.Email
}
return membersEmails
}
As you can see, that code requires to have a JSON key file of my-service-account#my-project.iam.gserviceaccount.com. This is the only way I have found to create a jwt.Config object.
Moreover, note that delegation is done with the line config.Subject = gsuiteAdminUserEmail; without that, I got the error :
googleapi: Error 403: Insufficient Permission: Request had insufficient authentication scopes., insufficientPermissions
Anyway, running :
SERVICE_ACCOUNT_FILE_PATH=/path/to/json/key/of/my/service/account \
GSUITE_ADMIN_USER_EMAIL=my-admin#my-domain.com \
go run main.go my-group#my-domain.com
prints with success all users in my-group#my-domain.com.
However, since this code will be run from a Google Compute Engine VM (or a Google Cloud Function) with my-service-account#my-project.iam.gserviceaccount.com used as service account, it seems ridiculous to need a JSON key of that service account to authenticate (since I am already authenticated as my-service-account#my-project.iam.gserviceaccount.com, on the VM or inside the GCF).
I have tried to replace the content of createAdminDirectoryService() with the following code to authenticate as the user who launched the script (with default credentials) :
func createAdminDirectoryService() *admin.Service {
ctx := context.Background()
client, _ := google.DefaultClient(ctx, scopes...)
srv, _ := admin.New(client)
return srv
}
But listing users returns an error :
googleapi: Error 403: Insufficient Permission: Request had insufficient authentication scopes., insufficientPermissions
As this is the same error I have got when I removed the delegation, I think this is related. Indeed, I did not provide my GSuite Admin account anywhere during the admin.Service creation.
Can anyone help about one of these points :
How can I authenticate directly with the user running the go script on a Google Compute Engine VM, or a Google Cloud Function?
Do I really need a JSON key file to generate a jwt.Config object?
I have looked into the source code of golang.org/x/oauth2/google, I could get a oauth2.Config instead of jwt.Config, but it does not seem possible to "delegate" with a oauth2 token. How else can I perform the delegation?
This post is somewhat old, but hope this can help:
I had the same problem and made it work by changing the OAuth scopes in the Compute Instance (by default, the scopes only include https://www.googleapis.com/auth/cloud-platform). In my case, I had to add the https://www.googleapis.com/auth/admin.directory.group.readonly scope. It can be done using the gcloud compute instances set-service-account command. That way the service account bearer token received from the metadata server has the correct scopes to access the Admin Directory API.
Unfortunately, it cannot be done for Cloud Functions as the Cloud Functions OAuth scopes do not seem to be customizable.
See this post for more details.
Btw it is also possible now to give a service account access to the Admin Directory API without domain-wide delegation.
r, err := srv.Users.List().
ViewType("domain_public").
Customer("my_customer").
OrderBy("email").
Do()
Try using this.
I am using osin, Go Lang oAuth Server to try and build a oAuth sever.
So I have used, or i am trying to use the complete example given, to give me a good place to start playing with the code to see what I can do.
However, I have a lot of errors with the file. Now most seem to be about error checking and i have seem to fix them (I am using Visual Code, which as very good Go Lang support). However, no matter what I try I cant seem to fix the error handling for w.Write,
http.HandleFunc("/appauth/code", func(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm() //*1
if err != nil {
log.Panic(err)
}
code := r.Form.Get("code")
w.Write([]byte("<html><body>")) *2
*1 - This was also missing an error and found my understanding this is how that is meant to be dealt with, please tell me if I am wrong?
*2 - With this I am getting, a errors unhandled call within Visual Code, and no matter how I try to deal with it, it will not work?
This is what I have tried,
err := w.Write([]byte("<html><body>"))
w.Write([]byte("<html><body>") (int, error) ) <- the Write seems to return an int and error?
So I am not sure how to deal with this type of error?
Discard the int if you don't need it:
_, err := w.Write([]byte("<html><body>"))
I'm currently using gorm and gin framework. I wonder what would happen if an error happened and my app didn't handle it?
Example:
if err := db.Where("name = ?", "jinzhu").First(&user).Error; err != nil {
// error handling...
}
In the above example, the error is being handled.
if db.Model(&user).Related(&credit_card).RecordNotFound() {
// no credit card found error handling
}
In the next example above, only the RecordNotFound() error is being handled, but what if it throws a different error? what will happen?
Will my app automatically respond with a 500 server internal error and will the app keep on running properly?
In the next example above, only the RecordNotFound() error is being handled, but what if it throws a different error?
If you won't catch the error it will continue on the code. Error is not a special type it's a simple struct
err := FunctionThatReturnsError()
if err == myError.RecordNotFound() { // if err is not RecordNotFound then it won't enter the if simple as that.
// Do something.
}
// continue code.
Will my app automatically respond with a 500 server internal error and will the app keep on running properly?
There will be no response if the go routine doesn't panic or you return a response. If you want to handle it you can do:
err := FunctionThatReturnsError()
if err == myError.RecordNotFound() {
panic("RecordNotFound")
}
or
err := FunctionThatReturnsError()
if err == myError.RecordNotFound() {
c.JSON(500, "Record not found"}
}
I don't recommend the panic method. If you're curious google why.
Go doesn't have exceptions. Instead of catching exceptions, you get errors via return values from functions. So there's no throwing or anything special going on behind the scenes, just a function that returns an error value, and like any other return value - you can discard it.
I wouldn't recommend discarding errors though. If you feel lazy or lost about what to do with an error - just log it:
log.Error(err)
You never know if an error you discarded is causing this mysterious bug you can swear is coming from anywhere but your own code.
I wonder what would happen if an error happened and my app didn't handle it?
Then the state of the app is undefined. If you don't check error values your app will be using values which are undefined (probably nil for pointers and "zeros" for values) or assuming that side effect occurred but it might not.
Let's say you have a function with signature func CreateStruct() (T, err)
and call it like that t, _ := CreateStruct() (not checking for error) you should not expect t variable to have a proper value set.
If you have function like func Update() err and you call it without error checking then you can't know whether update was performed or not.
Of course everything depends on API and implementation. But you get the idea.
but what if it throws a different error?
It's impossible. There is not throwing error mechanism in Go. Error can only be returned as a normal value.
You should never be lazy with handling errors. It's very important part of programming and Go makes it easier to realize.