Endpoints Google ID JWT, and Golang oauth2: have id_token, need access_token? - go

I have an app engine app behind endpoints and I'm having trouble following the documentation around adding authentication. My preference is to allow service accounts within the project through, then perform more granular app-side authorization.
In my case, most clients will be outside GCP and will be automated programs not people, so I'm using JSON key files thinking that is the way to go (correct me if I'm wrong please). I also don't want to have to redeploy the app to change user configs, so I follow the "GOOGLE ID JWT" information in the documentation from here:
https://cloud.google.com/endpoints/docs/openapi/service-to-service-auth
This is the security sections of my swagger JSON:
"securityDefinitions": {
"api_key": {
"type": "apiKey",
"name": "key",
"in": "query"
},
"google_id_token": {
"authorizationUrl": "",
"flow": "implicit",
"type": "oauth2",
"x-google-issuer": "https://accounts.google.com"
}
},
"security": [
{
"api_key": [],
"google_id_token": []
}
]
This deploys OK, but I am stumped on what to do from the client side to make use of the service account JSON.
In Go, I use the following, based on my understanding of the oauth2/google documentation:
package main
import (
"context"
"fmt"
"io/ioutil"
"log"
"os"
"golang.org/x/oauth2/google"
)
func main() {
ctx := context.Background()
apiKey := os.Getenv("API_KEY")
basePath := "https://SERVICE_NAME-dot-PROJECT_NAME.appspot.com" // changed for privacy
creds, _ := google.FindDefaultCredentials(ctx)
jwt, _ := google.JWTConfigFromJSON(creds.JSON, basePath)
client := jwt.Client(ctx)
res, _ := client.Get(fmt.Sprintf("%s/REQUEST_PATH?key=%s", basePath, apiKey)) // changed for privacy
body, _ := ioutil.ReadAll(res.Body)
log.Printf("### http status: %d %s", res.StatusCode, res.Status)
log.Printf("### http body: %s", body)
}
I've removed error-checking for this code paste for brevity, there are no errors when I run this. The result of this being:
2017/10/16 07:32:38 ### http status: 401 401 Unauthorized
2017/10/16 07:32:38 ### http body: {
"code": 16,
"message": "JWT validation failed: Missing or invalid credentials",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "auth"
}
]
}
Upon inspection of the internals of what's happening, such as calling jwt.TokenSource(ctx).Token(), I see that Google is returning an id_token, but no access_token (spotted by adding some debug logs to the jwt package):
2017/10/16 07:38:47 ### oauth2/jwt -> tokenURL: https://accounts.google.com/o/oauth2/token, form: url.Values{"grant_type":[]string{"urn:ietf:params:oauth:grant-type:jwt-bearer"}, "assertion":[]string{"...cut..."}}
2017/10/16 07:38:47 ### oauth2/jwt <- response: {
"id_token" : "...cut..."
}
2017/10/16 07:38:47 ### oauth2/jwt <- token: &oauth2.Token{AccessToken:"", TokenType:"", RefreshToken:"", Expiry:time.Time{wall:0x0, ext:63643740079, loc:(*time.Location)(0x1424160)}, raw:map[string]interface {}{"id_token":"...cut..."}}
So when the oauth2 HTTP transport tries to add the auth header -- which is tied to the AccessToken field above -- it results in a string like Authorization: Bearer, thus the result fails.
I feel like I'm missing a step here which isn't obvious to me in the documentation.
Thanks for your time.

It would be expected for the oauth client to return id_token instead of access_token. You're using JWTConfigFromJSON, not ConfigFromJSON, which is correct. I think the second argument to JWTConfigFromJSON is incorrect, and you should be specifying scopes instead. In this case, try "email" instead of basePath.

Related

Youtube Data API with Kotlin: The request is missing a valid API key

So I'm trying to use the Youtube Data API with Kotlin + Spring Boot and I've been struggling a bit.
For now, I'm using hardcoded values for the api_key and the access_token for test purposes.
I'm trying to send a request to list my playlists but I keep getting this error:
"error": {
"code": 403,
"message": "The request is missing a valid API key.",
"errors": [
{
"message": "The request is missing a valid API key.",
"domain": "global",
"reason": "forbidden"
}
],
"status": "PERMISSION_DENIED"
Here's my code:
Controller:
#RestController
class PlaylistController(
private val youtubeService: YoutubeService
) {
#GetMapping("/playlists")
fun getPlaylists() : YoutubePlaylistResponse {
return youtubeService.getPlaylists()
}
}
Service:
#Service
class YoutubeService(
private val youtubeClient: YoutubeClient
) {
fun getPlaylists() = youtubeClient.getPlaylists(
access_token = "[ACCESS_TOKEN]",
api_key = "[API_KEY]"
)
}
Client
#FeignClient(name = "youtube", url = "https://www.googleapis.com/youtube/v3")
interface YoutubeClient {
#GetMapping("/playlists")
fun getPlaylists(
#RequestHeader("Authorization", required = true) access_token: String,
#RequestParam("api_key") api_key: String,
#RequestParam("part") part: String = "snippet",
#RequestParam("mine") mine: Boolean = true
): YoutubePlaylistResponse
}
Any thoughts on what I'm doing wrong?
PS: I'm getting the acess_token through the OAuth 2.0 Playground
Edit:
I was calling api_key but it's actually only key.
But now I'm getting a new problem:
"error": {
"code": 401,
"message": "The request uses the \u003ccode\u003emine\u003c/code\u003e parameter but is not properly authorized.",
"errors": [
{
"message": "The r... (464 bytes)]] with root cause
Apparently, it's because I'm trying to access my playlists and it says that I don't have the permission, but when I do the same request using cURL I get an appropriate response. Any thoughts on this?
According to the API documentation, the parameter should be called key rather than api_key:
Every request must either specify an API key (with the key parameter) or provide an OAuth 2.0 token. Your API key is available in the Developer Console's API Access pane for your project.
Source: https://developers.google.com/youtube/v3/docs

AWS API Gateway HTTP API how to pass string query params?

So I am making an app and need AWS API Gateway. I want to use HTTP API instead of REST API. My code looks like this
package main
import (
"database/sql"
"fmt"
"strings"
"github.com/aws/aws-lambda-go/lambda"
_ "github.com/lib/pq"
)
here I make a connection to the database
func fetch(inte string, input string) string {
if err != nil {
panic(err)
}
switch inte {
case "00":
{
res = append(res, response)
}
switch len(res) {
case 0:
return "401"
}
case "01":
}
switch len(res) {
case 0:
return "402"
}
}
return "404"
}
type LambdaEvent struct {
Req string `json:"req"`
Num string `json:"num"`
}
type LambdaResponse struct {
Res string `json:"res"`
}
func LambdaHandler(event LambdaEvent) (LambdaResponse, error) {
res := fetch(event.Num, event.Req)
return LambdaResponse{
Res: res,
}, nil
}
func main() {
lambda.Start(LambdaHandler)
}
So as you see this is not the full code. I make a connection to the database and and work with the requests string query. so I tried the same with http api but it just gives me the 404 meaning the http api doesn't pass the query string to the lambda so how do I make my api pass the data to the lambda. Rest api works HTTP doesn't.
Thanks for any help.
I'm not familiar with Serverless Frameworks for APIGW, but manipulating QueryString parameters is built into the APIGW Console. Just login to AWS and search for APIGateway. Edit your HTTP API and select Integrations from the menu on the left. Select the integration that maps to your Lambda function and Edit the Parameter Mappings on the right
If you are deploying your lambdas and api-gateway with serverless framework you can do something like this:
hello:
handler: src/hello.handler
name: hello
events:
- http:
path: car/{id}/color/{color}
method: get
Assuming you are planning to use Lambda Proxy Integration in API Gateway, here are the changes that needs to be done to access the query parameters.
import github.com/aws/aws-lambda-go/events (This has all the relevant structs)
Change the lambda handler to func handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
Now you can access the query parameters as a Map at request.QueryStringParameters and execute your selection logic
When you return a response for API Gateway, ensure you follow the events.APIGatewayProxyResponse struct i.e. at least return a status code along with optional body, headers etc.
No changes/config required at the API Gateway to pass the query parameters with Lambda proxy integration
You can use your own structs for request and response, but they need to use the appropriate keys as defined in the events.APIGatewayProxyRequest and events.APIGatewayProxyResponse.
e.g. add the following in LambdaEvent struct to access the query string parameters.
QueryStringParameters map[string]string `json:"queryStringParameters"`
If you are getting started with AWS Lambda, have a look at AWS SAM to keep things simple.

Programmatically get Account Id from lambda context arn

I have access to
com.amazonaws.services.lambda.runtime.Context;
object and by extension the invoked function Arn. The arn contains the account Id where the lambda resides.
My question is simple, I want the cleanest way to extract the account Id from that.
I was taking a look
com.amazon.arn.ARN;
It has a whole bunch of stuff, but no account ID (which i presume is due to the fact that not all arns have account ids ?)
I want to cleanly extract the account Id, without resorting to parsing the string.
If your lambda is being used as an API Gateway proxy lambda, then you have access to event.requestContext.accountId (where event is the first parameter to your handler function).
Otherwise, you will have to split the ARN up.
From the AWS documentation about ARN formats, here are the valid Lambda ARN formats:
arn:aws:lambda:region:account-id:function:function-name
arn:aws:lambda:region:account-id:function:function-name:alias-name
arn:aws:lambda:region:account-id:function:function-name:version
arn:aws:lambda:region:account-id:event-source-mappings:event-source-mapping-id
In all cases, account-id is the 5th item in the ARN (treating : as a separator). Therefore, you can just do this:
String accountId = arn.split(":")[4];
You no longer need to parse the arn anymore, sts library has introduced get_caller_identity for this purpose.
Its an overkill, but works!.
Excerpts from aws docs.
python
import boto3
client = boto3.client('sts')
response = client.get_caller_identity()['Account']
js
/* This example shows a request and response made with the credentials for a user named Alice in the AWS account 123456789012. */
var params = {
};
sts.getCallerIdentity(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
/*
data = {
Account: "123456789012",
Arn: "arn:aws:iam::123456789012:user/Alice",
UserId: "AKIAI44QH8DHBEXAMPLE"
}
*/
});
More details here & here
I use this:
ACCID: { "Fn::Join" : ["", [{ "Ref" : "AWS::AccountId" }, "" ]] }
golang
import (
"github.com/aws/aws-lambda-go/lambdacontext"
)
func Handler(ctx context.Context) error {
lc, ok := lambdacontext.FromContext(ctx)
if !ok {
return errors.Errorf("could not get lambda context")
}
AwsAccountId := strings.Split(lc.InvokedFunctionArn, ":")[4]

Auth with google sheets API V4 / REST / Go

I have read example https://developers.google.com/sheets/reference/rest/v4/spreadsheets.values/update from Google
Everything is great until I coming to authentication.
I would like to send this Put request to update a spreadsheet from my application from Golang:
key := "my key"
// I think I do not need key, because it have to be OAuth...
spreadsheetId := "myspreadsheetID"
link := fmt.Sprintf("https://sheets.googleapis.com/v4/spreadsheets/%s/values/A1?valueInputOption=RAW&fields=updatedCells&key=%s",
spreadsheetId, key)
request := gorequest.New()
resp, body, errs := request.Put(link).
Send(`{"values": [ ["hello","my", "friends" ] ]}`).
End()
if errs != nil {
fmt.Println(errs)
}
if resp.StatusCode != 200 {
fmt.Println(resp.Status)
}
fmt.Println(body)
The response is
401 Unauthorized
{
"error": {
"code": 401,
"message": "The request does not have valid authentication credentials.",
"status": "UNAUTHENTICATED"
}
}
I tried to grasp the Auth Guide but honestly I am not sure I figured out how can auth the request...
Any help would be appreciated very much.
Key is used for accessing Public data, Oauth2 requires that you authenticate and grant the application permission to access private data.
Worked in V3: If you want to use a key you can set the google sheet you are trying to access public and it will work.
Otherwise you are going to have to implement Oauth2. I cant help with go but a quick search on google turned up a bunch of Oauth2 tutorials for it.
Update:
I just did a quick check in the documentation. Method: spreadsheets.values.update
Authorization
Requires one of the following OAuth scopes:
* https://www.googleapis.com/auth/drive
* https://www.googleapis.com/auth/spreadsheets
which leads me to think that they removed the public trick from the v4 of the api. Looks like you are going to have to implement oauth2 or service accounts

Golang: implements http server health checking. gocraft/health

I want to check the health of my service,having the metrics of each endPoint.
My service calls some other services and recieves a Json code, I make templates with it, and then I send it to a http.ResponseWriter.
I searched and I found this package "gocraft/health" but I didn't really understand how it works.
Is there any other way or package to generate metrics or should I just use "gocraft/health.
Thank you in advance
Finally, I choose "gocraft/health", a great library.
Example of usage:
package main
import (
"log"
"net/http"
"os"
"time"
"github.com/gocraft/health"
)
//should be global Var
var stream = health.NewStream()
func main() {
// Log to stdout!
stream.AddSink(&health.WriterSink{os.Stdout})
// Make sink and add it to stream
sink := health.NewJsonPollingSink(time.Minute*5, time.Minute*20)
stream.AddSink(sink)
// Start the HTTP server! This will expose metrics via a JSON API.
adr := "127.0.0.1:5001"
sink.StartServer(adr)
http.HandleFunc("/api/getVastPlayer", vastPlayer)
log.Println("Listening...")
panic(http.ListenAndServe(":2001", nil))
}
Per the initialization options above, your metrics are aggregated in 5-minute chunks. We'll keep 20 minutes worth of data in memory. Nothing is ever persisted to disk.
You can create as many jobs as you want
func vastPlayer(w http.ResponseWriter, r *http.Request) {
job_1 := stream.NewJob("/api/getVastPlayer")
...
...
if bol {
job_1.Complete(health.Success)
} else {
job_1.Complete(health.Error)
}
}
Once you start your app, this will expose metrics via a JSON API. You can browse the /health endpoint (eg, 127.0.0.1:5001/health) to see the metrics. You will get something like that:
{
"instance_id": "sd-69536.29342",
"interval_duration": 86400000000000,
"aggregations": [
{
"interval_start": "2015-06-11T02:00:00+02:00",
"serial_number": 1340,
"jobs": {
"/api/getVastPlayer": {
"timers": {},
"events": {},
"event_errs": {},
"count": 1328,
"nanos_sum": 140160794784,
"nanos_sum_squares": 9.033775178022173E+19,
"nanos_min": 34507863,
"nanos_max": 2736850494,
"count_success": 62,
"count_validation_error": 1266,
"count_panic": 0,
"count_error": 0,
"count_junk": 0
},
"timers": {},
"events": {},
"event_errs": {}
}
}
]
}
For more information and functionality check this link:
https://github.com/gocraft/health
In case you came across this question because you were looking for exposing a /health endpoint, there is an upcoming RFC for health checks: https://github.com/inadarei/rfc-healthcheck
And there's a Go library health-go for exposing health endpoints compliant with that RFC: https://github.com/nelkinda/health-go
Example:
package main
import (
"github.com/nelkinda/health-go"
"net/http"
)
func main() {
// 1. Create the health Handler.
h := health.New(health.Health{Version: "1", ReleaseID: "1.0.0-SNAPSHOT"})
// 2. Add the handler to your mux/server.
http.HandleFunc("/health", h.Handler)
// 3. Start your server.
http.ListenAndServe(":80", nil)
}
It is extensible and supports a number of built-in checks such as uptime and sysinfo.
Disclaimer: I'm the author of health-go.

Resources