Like the title said, my question here is how to use Go to programmatically get the Github commit history of a given file in a given repository
It seems that you need to access GitHub api from golang. There are a plenty of libraries but I would recommend using go-github.
Here is how you can try doing that
package main
import (
"context"
"github.com/google/go-github/github"
)
func main() {
var username string = "MayukhSobo"
client := github.NewClient(nil)
commits, response, err := client.Repositories.ListCommits(context.Background(), username, "Awesome-Snippets", nil)
if err != nil && response.StatusCode != 200 {
panic(err)
}
for _, commit := range commits {
// commit.SHA
// commit.Files
// You can use the commit
}
}
If you are trying to access the some other public repo, you need to pass the owner name in the username and change the repo name.
If you face access issues, it can be probably it is a private repo. You can also use the key pair to set up the access.
Related
How do I write a unit test for my code that clones a repo using git-go
Below is a sample of the function I have created. I am cloning multiple repos and reading a particular file that is in that repo, I am unsure how to unit test this function.
func cloneRepository(repository string) (repo beans.Repo) {
dir, err := os.MkdirTemp("./", "temp") //To create a temp folder to clone repo in
if err != nil...
_, err := git.PlainClone(dir, false, &git.CloneOptions{
URL: repository,
Depth: 1,
})
var repo beans.Repo
if err = util.ParseYmlFile("filename.yml", &repo) // Custom util function to parse a file in the repository
if err = os.RemoveAll(dir); err != nil{...}
return repo
}
You can mock the git.PlainClone() function so it returns a custom file for your tests.
Take a look into spf13's lib, that provides a filesystem mocking solution!
What we did in the past was create a bare git repository with some predefined content, put it under e.g. testdata/myrepo.git and use it during unit-testing.
Commit the repo normally as part of your project.
I'm trying to write an application in GO that will get all the image vulnerabilities inside a GCP project for me using the Container Analysis API.
The GO Client library for this API has the function findVulnerabilityOccurrencesForImage() to do this, however it requires you to pass the URL of the image you want to get the vulnerability report from in the form resourceURL := "https://gcr.io/my-project/my-repo/my-image" and the projectID. This means that if there are multiple images in your project, you have to list and store them first and only after that you can recursively call the findVulnerabilityOccurrencesForImage() function to get ALL of the vulnerabilities.
So I need a way to get and store all of the images' URLs inside all of the repos inside a given GCP project, but so far I couldn't find a solution. I can easily do that in the CLI by running gcloud container images list command but I don't see a way how that can be done with an API.
Thank you in advance for your help!
You can use the Cloud Storage package and the Objects method to do so. For example:
func GetURLs() ([]string, error) {
bucket := "bucket-name"
urls := []string{}
results := client.Bucket(bucket).Objects(context.Background(), nil)
for {
attrs, err := results.Next()
if err != nil {
if err == iterator.Done {
break
}
return nil, fmt.Errorf("iterating results: %w", err)
}
urls = append(urls, fmt.Sprint("https://storage.googleapis.com", "/", bucket, "/", attrs.Name))
}
return urls, nil
}
I'm new to developing web applications. I'm working with golang and would like to serve user uploaded files securely, such as allowing them to view their own files only.
Now I have saved those files to a local file system with random names. If I serve the entire directory, malicious users may view other users files. This sounds like a common use case, I wonder what's the best approach to deal with it?
This question in pretty vague and architectural decisions must be made to optimize data access and secure the files.
However, here is a simple solution that might serve your use-case.
package main
import (
"fmt"
"mime"
"net/http"
"path/filepath"
)
//UserFilesMap is the map that contains
var UserFilesMap map[string]FilePermission
type FilePermission map[string]struct{}
//FileServer is the function that serves files
func FileServer(w http.ResponseWriter, r *http.Request) {
//get the file path the user wants to access
filename := r.URL.Path[9:]
var uname, pass string
var ok bool
if uname, pass, ok = r.BasicAuth(); !ok {
w.WriteHeader(http.StatusForbidden)
return
}
if !(uname == "user" && pass == "1234") {
w.WriteHeader(http.StatusForbidden)
return
}
//Checking if user has permission to the file
if _, ok := UserFilesMap[uname][filename]; !ok {
w.WriteHeader(http.StatusForbidden)
return
}
w.Header().Set("Content-Type", mime.TypeByExtension(filepath.Ext(filename)))
http.ServeFile(w, r, "files/"+filename)
}
func main() {
UserFilesMap = make(map[string]FilePermission)
// UserFilesMap["user"] = FilePermission{"xyz.txt": struct{}{}}
UserFilesMap["user"] = FilePermission{"abc.txt": struct{}{}}
http.HandleFunc("/getFile/", FileServer)
if err := http.ListenAndServe(":8080", nil); err != nil {
fmt.Println("Error in ListenAndServe")
}
}
Here, I used a map to store the permissions of files. I would suggest you go for a SQL table instead.
If ur filenames are random and long enough and use a secure random generator this is already secure (unless directory-listing is enabled) with some limits though.
https://golang.org/pkg/crypto/rand/
One user will have access to the file only if he has the url with the random name. The limitation is though that the URL will be saved in the browser history, if someone else finds it he will also have access to it.
I have the following function in golang:
import (
"github.com/aws/aws-sdk-go/service/iam"
"github.com/aws/aws-sdk-go/aws/session"
"fmt"
)
func NewIAM() *SphinxIAM {
// awsConfig := aws.NewConfig()
sess, err := session.NewSession()
if err != nil {
fmt.Println("Failed to create session,", err)
return nil
}
session := &SphinxIAM{iam: iam.New(sess)}
return session
}
Now, I am getting the following error when I run this:
cannot use sess (type *session.Session) as type "github.com/aws/aws-sdk-go/aws/client".ConfigProvider in argument to iam.New:
*session.Session does not implement "github.com/aws/aws-sdk-go/aws/client".ConfigProvider (wrong type for ClientConfig method)
have ClientConfig(string, ...*"stash/cloud/sphinx/vendor/github.com/aws/aws-sdk-go/aws".Config) "stash/cloud/sphinx/vendor/github.com/aws/aws-sdk-go/aws/client".Config
want ClientConfig(string, ...*"github.com/aws/aws-sdk-go/aws".Config) "github.com/aws/aws-sdk-go/aws/client".Config
I have to change the method getting imported but how do I exactly do that?
Thanks!
The problem here is that your github.com/aws/aws-sdk-go/aws/session package is vendored, it is loaded from the folder stash/cloud/sphinx/vendor/github.com/aws/aws-sdk-go/aws.
But the function you want to pass it: iam.New() is not vendored, it does not come from the same vendor folder (stash/cloud/sphinx/vendor/xxx) but it comes directly from github.com/aws/aws-sdk-go/service/iam.
Either put both packages under the same vendor folder, or none. It is possible that one of your dependency tool does this, (e.g. glide), in which case you should instruct your tool to handle both as vendored.
I'm trying to insert a simple record as in GoDoc. But this returns,
rpc error: code = 7 desc = "User can't access project: tidy-groove"
When I searched for grpc codes, it says..
PermissionDenied Code = 7
// Unauthenticated indicates the request does not have valid
// authentication credentials for the operation.
I've enabled Big table in my console and created a cluster and a service account and recieved the json. What I'm doing wrong here?
package main
import (
"fmt"
"golang.org/x/net/context"
"golang.org/x/oauth2/google"
"google.golang.org/cloud"
"google.golang.org/cloud/bigtable"
"io/ioutil"
)
func main() {
fmt.Println("Start!")
put()
}
func getClient() *bigtable.Client {
jsonKey, err := ioutil.ReadFile("TestProject-7854ea9op741.json")
if err != nil {
fmt.Println(err.Error())
}
config, err := google.JWTConfigFromJSON(
jsonKey,
bigtable.Scope,
) // or bigtable.AdminScope, etc.
if err != nil {
fmt.Println(err.Error())
}
ctx := context.Background()
client, err := bigtable.NewClient(ctx, "tidy-groove", "asia-east1-b", "test1-bigtable", cloud.WithTokenSource(config.TokenSource(ctx)))
if err != nil {
fmt.Println(err.Error())
}
return client
}
func put() {
ctx := context.Background()
client := getClient()
tbl := client.Open("table1")
mut := bigtable.NewMutation()
mut.Set("links", "maps.google.com", bigtable.Now(), []byte("1"))
mut.Set("links", "golang.org", bigtable.Now(), []byte("1"))
err := tbl.Apply(ctx, "com.google.cloud", mut)
if err != nil {
fmt.Println(err.Error())
}
}
I've solved the problem. It's nothing wrong with the code, but config json itself. So anyone who out there want to authenticate and came here by google search... This code is correct and working perfectly. What I've done wrong is follows.
First I made a service account and got the json. But google warned me that im not an owner of project hence it wont be added to accept list but anyway it let me download the json.
Then I deleted that key from console and requested project owner to create a key for me.
There he has created another key with the same name I given.. And since he's the owner no error/warning msgs displayed and successfully json file was downloaded.
When I tried with that... my question begun. That's when i posted this question.
After that with no solutions. I asked owner to delete that key and create another key but with a different name..
Then it worked! It seems if you try to create a key with non-owner account and then again create with same name ( after deleting original of course ) has no effect. Hope this helps everyone out there :)
Take a look at: helloworld.go or search.go which uses GOOGLE_APPLICATION_CREDENTIALS environment variable.
For most environments, you no longer even need to set GOOGLE_APPLICATION_CREDENTIALS. Google Cloud Platform, Managed VMs or Google App Engine all have the right thing set for you. Your desktop environment will also be correct if you've used gcloud init or it's predecessor gcloud auth login followed by gcloud config set project <projectID>.