Compile GO source code (on regular intervals) - go

I have a GO compiler available (i.e. installed in a dir ~/xxxx/bin/go).
Suppose to call this version GO.1
I now clone the GO repository from GitHub (ex. into ~/yyyy/)
Setting $GOROOT_BOOTSTRAP=~/xxxx/ I can compile it
as cd ~/yyyy/src && ./all.bash
obtaining the GO binary in ~/yyyy/bin (Suppose to call this GO.2)
I'd like now to use this new GO.2 binary ~/yyyy/bin/go to recompile its new source (~/yyyy/src) having a final 'good' GO.3
I cannot just set $GOROOT_BOOTSTRAP to ~/yyyy/ or play with some others env vars (at least I tried many attempts with no success).
The only way I've found to do this is:
copy ~/yyyy/ (about 800MB!) onto ~/xxxx/ (so that GO.2
overwrites GO.1 and the $GOROOT_BOOTSTRAP is unchanged)
redo cd ~/yyyy/src && ./all.bash
This cannot be the expected behaviour.
I'd like to have the most up-to-date binary GO.x just with a git pull && cd ./src && ./all.bash and no much more than this.
Any hint is welcome.

Related

go-swagger restapi/configure_todo_list.go - api.TodoGetHandler undefined error

I am a newbie in go and go-swagger. I am following steps in Simple Server tutorial in goswagger.io.
I am using Ubuntu 18.04, swagger v0.25.0 and go 1.15.6.
Following the same steps, there are a few differences of the files generated. For instance, goswagger.io's has find_todos_okbody.go and get_okbody.go in models but mine does not. Why is that so?
Link to screenshot of my generated files vs
Link to screenshot of generated files by swagger.io
Starting the server as written in the tutorial go install ./cmd/todo-list-server/ gives me the following error. Can anyone please help with this?
# my_folder/swagger-todo-list/restapi
restapi/configure_todo_list.go:41:8: api.TodosGetHandler undefined (type *operations.TodoListAPI has no field or method TodosGetHandler)
restapi/configure_todo_list.go:42:6: api.TodosGetHandler undefined (type *operations.TodoListAPI has no field or method TodosGetHandler)
The first step in goswagger.io todo-list is swagger init spec .... Which directory should I run this command in? I ran it in a newly created folder in my home directory. However, from the page, it shows the path to be ~/go/src/github.com/go-swagger/go-swagger/examples/tutorials/todo-list. I am not sure whether I should use go get ..., git clone ... or create those folders. Can someone advise me?
Thanks.
This is likely the documentation lagging behind the version of the code that you are running. As long as it compiles, the specific files the tool generates isn't so crucial.
This is a compilation error. When you do go install foo it will try to build the foo package as an executable and then move that to your GOPATH/bin directory. It seems that the generated code in restapi/configure_todo_list.go isn't correct for the operations code generated.
All you need to run this tutorial yourself is an empty directory and the swagger tool (not its source code). You run the commands from the root of this empty project. In order not to run into GOPATH problems I would initialise a module with go mod init todo-list-example before doing anything else.
Note that while the todo-list example code exists inside the go-swagger source, it's there just for documenting example usage and output.
What I would advice for #2 is to make sure you're using a properly released version of go-swagger, rather than installing from the latest commit (which happens when you just do a go get), as I have found that to be occasionally unstable.
Next, re-generate the entire server, but make sure you also regenerate restapi/configure_todo_list.go by passing --regenerate-configureapi to your swagger generate call. This file isn't always refreshed because you're meant to modify it to configure your app, and if you changed versions of the tool it may be different and incompatible.
If after that you still get the compilation error, it may be worth submitting a bug report at https://github.com/go-swagger/go-swagger/issues.
Thanks #EzequielMuns. The errors in #2 went away after I ran go get - u -f ./... as stated in
...
For this generation to compile you need to have some packages in your GOPATH:
* github.com/go-openapi/runtime
* github.com/jessevdk/go-flags
You can get these now with: go get -u -f ./...
I think it's an error of swagger code generation. You can do as folloing to fix this:
delete file configure_todo_list.go;
regenerate code.
# swagger generate server -A todo-list -f ./swagger.yml
Then, you can run command go install ./cmd/todo-list-server/, it will succeed.

I can't get new modules to load in metasploit

I have been trying to load a new module in MS. I followed the instructions and done a git clone [address of files]. I then copied the new files into /root/.msf4/modules/exploits/windows/smb and into /usr/share/metasploit-framework/modules/windows/smb and issued reload_all and restarted msfconsole but still no matter what way i try it it still says:
msf > use exploits/windows/smb/eternalblue_doublepulsar
[-] Failed to load module: exploits/windows/smb/eternalblue_doublepulsar
msf >
i tried every combo including:
use exploit/windows/smb/eternalblue_doublepulsar
use exploits/windows/smb/eternalblue_doublepulsar
use exploit/windows/smb/eternalblue_doublepulsar.rb
use exploits/windows/smb/eternalblue_doublepulsar.rb
and also the same with other downloadable modules
i double checked the file path, i tried:
chmod 644 eternalblue_doublepulsar.rb
to make the perms the same as the other modules in the same folder
which work btw
any help would be much appreciated
Thank you
Damian Moore
Did u clone the exploit from https://github.com/ElevenPaths/Eternalblue-Doublepulsar-Metasploit? a way to debug, choose an exploit which already exists in the folder(such as ms08_067_netapi.rb),change the name to eternalblue_doublepulsar,and reload_all,run it and see what will be happened.

Using forked package import in Go

Suppose you have a repository at github.com/someone/repo and you fork it to github.com/you/repo. You want to use your fork instead of the main repo, so you do a
go get github.com/you/repo
Now all the import paths in this repo will be "broken", meaning, if there are multiple packages in the repository that reference each other via absolute URLs, they will reference the source, not the fork.
Is there a better way as cloning it manually into the right path?
git clone git#github.com:you/repo.git $GOPATH/src/github.com/someone/repo
If you are using go modules. You could use replace directive
The replace directive allows you to supply another import path that might
be another module located in VCS (GitHub or elsewhere), or on your
local filesystem with a relative or absolute file path. The new import
path from the replace directive is used without needing to update the
import paths in the actual source code.
So you could do below in your go.mod file
module some-project
go 1.12
require (
github.com/someone/repo v1.20.0
)
replace github.com/someone/repo => github.com/you/repo v3.2.1
where v3.2.1 is tag on your repo. Also can be done through CLI
go mod edit -replace="github.com/someone/repo#v0.0.0=github.com/you/repo#v1.1.1"
To handle pull requests
fork a repository github.com/someone/repo to github.com/you/repo
download original code: go get github.com/someone/repo
be there: cd "$(go env GOPATH)/src"/github.com/someone/repo
enable uploading to your fork: git remote add myfork https://github.com/you/repo.git
upload your changes to your repo: git push myfork
http://blog.campoy.cat/2014/03/github-and-go-forking-pull-requests-and.html
To use a package in your project
https://github.com/golang/go/wiki/PackageManagementTools
One way to solve it is that suggested by Ivan Rave and http://blog.campoy.cat/2014/03/github-and-go-forking-pull-requests-and.html -- the way of forking.
Another one is to workaround the golang behavior. When you go get, golang lays out your directories under same name as in the repository URI, and this is where the trouble begins.
If, instead, you issue your own git clone, you can clone your repository onto your filesystem on a path named after the original repository.
Assuming original repository is in github.com/awsome-org/tool and you fork it onto github.com/awesome-you/tool, you can:
cd $GOPATH
mkdir -p {src,bin,pkg}
mkdir -p src/github.com/awesome-org/
cd src/github.com/awesome-org/
git clone git#github.com:awesome-you/tool.git # OR: git clone https://github.com/awesome-you/tool.git
cd tool/
go get ./...
golang is perfectly happy to continue with this repository and doesn't actually care some upper directory has the name awesome-org while the git remote is awesome-you. All import for awesome-org are resovled via the directory you have just created, which is your local working set.
In more length, please see my blog post: Forking Golang repositories on GitHub and managing the import path
edit: fixed directory path
If your fork is only temporary (ie you intend that it be merged) then just do your development in situ, eg in $GOPATH/src/launchpad.net/goamz.
You then use the features of the version control system (eg git remote) to make the upstream repository your repository rather than the original one.
It makes it harder for other people to use your repository with go get but much easier for it to be integrated upstream.
In fact I have a repository for goamz at lp:~nick-craig-wood/goamz/goamz which I develop for in exactly that way. Maybe the author will merge it one day!
Here's a way to that works for everyone:
Use github to fork to "my/repo" (just an example):
go get github.com/my/repo
cd ~/go/src/github.com/my/repo
git branch enhancement
rm -rf .
go get github.com/golang/tools/cmd/gomvpkg/…
gomvpkg <<oldrepo>> ~/go/src/github.com/my/repo
git commit
Repeat each time when you make the code better:
git commit
git checkout enhancement
git cherry-pick <<commit_id>>
git checkout master
Why? This lets you have your repo that any go get works with. It also lets you maintain & enhance a branch that's good for a pull request. It doesn't bloat git with "vendor", it preserves history, and build tools can make sense of it.
Instead of cloning to a specific location, you can clone wherever you want.
Then, you can run a command like this, to have Go refer to the local version:
go mod edit -replace github.com/owner/repo=../repo
https://golang.org/cmd/go#hdr-Module_maintenance
The answer to this is that if you fork a repo with multiple packages you will need to rename all the relevant import paths. This is largely a good thing since you've forked all of those packages and the import paths should reflect this.
Use vendoring and submodules together
Fork the lib on github (go-mssqldb in this case)
Add a submodule which clones your fork into your vendor folder but has the path of the upstream repo
Update your import statements in your source code to point to the vendor folder, (not including the vendor/ prefix). E.g. vendor/bob/lib => import "bob/lib"
E.g.
cd ~/go/src/github.com/myproj
mygithubuser=timabell
upstreamgithubuser=denisenkom
librepo=go-mssqldb
git submodule add "git#github.com:$mygithubuser/$librepo" "vendor/$upstreamgithubuser/$librepo"
Why
This solves all the problems I've heard about and come across while trying to figure this out myself.
Internal package refs in the lib now work because the path is unchanged from upstream
A fresh checkout of your project works because the submodule system gets it from your fork at the right commit but in the upstream folder path
You don't have to know to manually hack the paths or mess with the go tooling.
More info
https://git-scm.com/book/en/v2/Git-Tools-Submodules
How do I fix the error message "use of an internal package not allowed" when go getting a golang package?
https://github.com/denisenkom/go-mssqldb/issues/406
https://github.com/golang/go/wiki/PackageManagementTools#go15vendorexperiment
The modern answer (go 1.15 and higher, at least).
go mod init github.com/theirs/repo
Make an explicit init arg that is the ORIGINAL package names. If you don't include the repo name, it will assume the one in gopath. But when you use go modules, they no longer care where they are on disk, or where git actually pulls dependencies from.
To automate this process, I wrote a small script. You can find more details on my blog to add a command like "gofork" to your bash.
function gofork() {
if [ $# -ne 2 ] || [ -z "$1" ] || [ -z "$2" ]; then
echo 'Usage: gofork yourFork originalModule'
echo 'Example: gofork github.com/YourName/go-contrib github.com/heirko/go-contrib'
return
fi
echo "Go get fork $1 and replace $2 in GOPATH: $GOPATH"
go get $1
go get $2
currentDir=$PWD
cd $GOPATH/src/$1
remote1=$(git config --get remote.origin.url)
cd $GOPATH/src/$2
remote2=$(git config --get remote.origin.url)
cd $currentDir
rm -rf $GOPATH/src/$2
mv $GOPATH/src/$1 $GOPATH/src/$2
cd $GOPATH/src/$2
git remote add their $remote2
echo Now in $GOPATH/src/$2 origin remote is $remote1
echo And in $GOPATH/src/$2 their remote is $remote2
cd $currentDir
}
export -f gofork
You can use command go get -f to get you a forked repo
in your Gopkg.toml file add these block below
[[constraint]]
name = "github.com/globalsign/mgo"
branch = "master"
source = "github.com/myfork/project2"
So it will use the forked project2 in place of github.com/globalsign/mgo

Update Local Workspace to SVN Repository 'Head' With Ruby SVN Bindings

I am writing a program in Ruby that necessitates downloading the most current version of my team's software from SVN upon start up.
The checkout function (from the Ruby SVN bindings) is what I believe I want to use, because an update would not ADD any files that do not exist on my machine's local "trunk" workspace. A checkout statement would both update files that do not match to HEAD, and it would download ones that don't exist at all. Effectively, after running a fully recursive checkout, I would hope to have an exact copy of the most recent SVN repository.
According to this API, a checkout statement basically takes the following:
an exact SVN URL
a local root project directory
a revision (I would be using the string 'HEAD')
recursive (integer 1 or 0)
a pool object (I cannot determine what this is for exactly, but I don't think it affects me)
Here's what I wrote, inside a block that iterates for each file in the SVN repository:
if status != NORMAL #any file that changed or is 'missing'
ctx.checkout(status.entry.url, ROOTDIR, 'HEAD', 0, nil) #update abnormal file to HEAD
end
As a test, I erased a directory from my local workspace, and attempted to restore it with this command. It runs through until it reaches one of the missing files, at which point it raises an error:
`svn_client_checkout3': subversion/libsvn_fs_fs/tree.c:663: Svn::Error::FsNotFound: File not found: revision 0, path '/trunk/project-gadfly/SocketServer/DiscoveryServer.cpp' (Svn::Error::FsNotFound)
I do not understand why this error would be raised, because I thought that a checkout statement would see that the directory (i.e. file) does not exist locally and then create it. Perhaps I am doing something wrong?
Looking back on what I've written, I think all of this was a long-winded way of asking the following simple question: How do I get the most current version of SVN repository onto my local hard drive with an SVN Ruby command?
Thanks in advance,
Elwood Hopkins
I don't know about Ruby-specific part of the question, but it's clear that you asked SVN API to checkout "status.entry.url" at revision 0, which of course doesn't exist here.
It's also strange that you looked into Perl documentation for writing in Ruby. I would recommend you to look at Subversion sources instead.
Here's Ruby method declaration:
http://svn.apache.org/repos/asf/subversion/branches/1.7.x/subversion/bindings/swig/ruby/svn/client.rb
def checkout(url, path, revision=nil, peg_rev=nil,
depth=nil, ignore_externals=false,
allow_unver_obstruction=false)
revision ||= "HEAD"
Client.checkout3(url, path, peg_rev, revision, depth,
ignore_externals, allow_unver_obstruction,
self)
end
So as you can see, you've specified 0 as peg revision. But you should specify HEAD instead.
What about pools --- they are parts of SVN memory managements. Here's the explanation: http://subversion.apache.org/docs/community-guide/conventions.html#apr-pools

Mogenerator and Xcode 4

I just installed mogenerator+xmo'd on my development machine and would like to start playing with it. The only instructions I could really find online were from a previous SO post, and those don't work with XCode 4 (or at least ⌘I doesn't pull up metadata any more and I don't know how).
So to get things up and running, is all that needs to happen to add xmod in the .xcdatamodeld's comments (wherever they are) and the classes will be generated/updated on save from then on?
While trying to find this answer myself, I found MOGenerator and Xcode 4 integration guide on esenciadev.com. This solution is not a push-button integration, but it works. The link has detailed instructions, but generally you:
Copy the shell scripts into your project
Add build rules to your target to run the two shell scripts
When you build your project, the script runs MOGenerator on all .xcdatamodel files in your project directory. After the build, if the script generates new class files, you must manually add them to your project. Subsequent builds will remember existing MO-Generated files.
Caveats:
The example's build rule assumes you put the scripts into a /scripts/ file folder within your project directory. When I ignored this detail (creating a project folder but not a file folder) I got a build error. Make sure the build rule points to the script's file location.
The script uses the --base-class argument. Unless your model classes are subclasses of a custom class (not NSManagedObject), you must delete this argument from the script. E.g.,
mogenerator --model "${INPUT_FILE_PATH}/$curVer" --output-dir "${INPUT_FILE_DIR}/" --base-class $baseClass
Now that Xcode 4 is released Take a look at the Issues page for mogenerator
After I make changes to my model file, I just run mogenerator manually from the terminal. Using Xcode 4 and ARC, this does the trick:
cd <directory of model file>
mogenerator --model <your model>.xcdatamodeld/<current version>.xcdatamodel --template-var arc=YES
Maybe I'll use build scripts at some point, but the terminal approach is too simple to screw up.
I've found a Script in the "Build Phases" to be more reliable than the "Build Rules".
Under "Build Phases" for your Target, choose the button at the bottom to "Add Run Script". Drag the run script to the top so that it executes before compiling sources.
Remember that the actual data model files (.xcdatamodel) are contained within a package (.xcdatamodeld), and that you only need to compile the latest data model for your project.
Add the following to the script (replacing text in angle-brackets as appropriate)
MODELS_DIR="${PROJECT_DIR}/<path to your models without trailing slash>"
DATA_MODEL_PACKAGE="$MODELS_DIR/<your model name>.xcdatamodeld"
CURRENT_VERSION=`/usr/libexec/PlistBuddy "$DATA_MODEL_PACKAGE/.xccurrentversion" -c 'print _XCCurrentVersionName'`
# Mogenerator Location
if [ -x /usr/local/bin/mogenerator ]; then
echo "mogenerator exists in /usr/local/bin path";
MOGENERATOR_DIR="/usr/local/bin";
elif [ -x /usr/bin/mogenerator ]; then
echo "mogenerator exists in /usr/bin path";
MOGENERATOR_DIR="/usr/bin";
else
echo "mogenerator not found"; exit 1;
fi
$MOGENERATOR_DIR/mogenerator --model "$DATA_MODEL_PACKAGE/$CURRENT_VERSION" --output-dir "$MODELS_DIR/"
Add options to mogenerator as appropriate. --base-class <your base class> and --template-var arc=true are common.
Random tip. If you get Illegal Instruction: 4 when you run mogenerator. Install it from the command line:
$ brew update && brew upgrade mogenerator

Resources