I am currently using Jenkins to build C/C++ projects. The projects uses conan packages as dependencies. I am trying to create a package with specific commit ID which can be later used by other packages.
For example:
RepoA is dependent on package created by RepoB. When RepoA build is trigged in jenkins RepoB build is triggered first in the pipeline and the packages are deployed in local cache which can be used by the RepoA later from the local cache.
RepoA has branch name called X and RepoB will also have same branch name called X. I need to use packages created from RepoB branch X for RepoA. Is there a way in conan to make RepoA use package created from RepoB branch X. I already can use the branch name as channel and use the right packages accordingly for RepoA as the branch name will be kept same. But apart from using the branch name is there any other way to create packages with commit ID as package ID and use it later for RepoA. If anyone knows the answer would be a great help.
The link below explains different package revision but I am cannot get a grasp of how to use it. It would be great if someone can give an example for the solution
Conan package revisions
Related
I am getting this error from go mod tidy when I am trying to update my dependancies. I am primarily developing a webhook service to use with cert-manager and I cannot figure out how to resolve this dependency isssue since the packages I am dependent on are created by other developers and I cannot control those "sub dependency".
This is my output:
go.opentelemetry.io/otel/semconv: module go.opentelemetry.io/otel#latest found (v1.9.0), but does not contain package go.opentelemetry.io/otel/semconv
I looked the package up here: https://pkg.go.dev/go.opentelemetry.io/otel/semconv
and the problem for me seems like that the package have been restructured like this:
go.opentelemetry.io/otel/semconv/v1.9.0
as a subdirectory instead of package version.
Is there a way I can manipulate the underlying dependancy of the packages that my service is depending on?
Please leave a comment if you need addictional information.
Take a look at the accepted solution's comments
You may want to use a local copy of the module where you can fix the issue and use it. Steps for that
Clone the module repository git clone https://github.com/open-telemetry/opentelemetry-go.git
If needed, checkout to a branch/tag git checkout branch_name
In the go.mod file of your module, add the following lines
replace (
go.opentelemetry.io => /path/where/cloned/opentelemetry-go
)
Now you should be able to modify code in the cloned opentelemetry-go repo and use it in your module
I am using multiple packages that I import into different projects, these range from custom adapters for my business logic that are shared by lambda and google cloud functions and other public packages. The way I do this right now is that I vendor them and include them for cloud functions. For applications that can be compiled and deployed on a VM, I compile them separately. This works fine for me, however, its a pain developing these modules.
If I update the method signature and names in the package, I have to push my changes to github / gitlab (my package path is something like gitlab.com/groupName/projectName/pkg/packageName) and then do a go get -u <pacakgeName> to update the package.
This also, does not really update it, sometimes am stuck with an older version with no idea on how to update it. Is there an easier way of working with this I wonder.
For sake of clarity:
Exported package 1
Path: gitlab.com/some/name/group/pkg/clients/psql
psql-client
|
|_ pkg
|
|_psql.go
Application 1 uses psql-client
Path: gitlab.com/some/name/app1
Application 2 uses psql-client
Path: gitlab.com/some/name/app2
My understanding is that (a) you are using the new Go modules system, and that (b) part of the problem is that you don't want to keep pushing changes to github or gitlab across different repositories when you are doing local development.
In other words, if you have your changes locally, it sounds like you don't want to round-trip those changes through github/gitlab in order for those changes to be visible across your related repositories that you are working on locally.
Most important advice
It is greatly complicating your workflow to have > 1 module in a single repository.
As is illustrated by your example, in general it is almost always more work on an on-going basis to have > 1 module in a single repository. It is also very hard to get right. For most people, the cost is almost always not worth it. Also, often the benefit is not what people expect, or in some cases, there is no practical benefit to having > 1 module in a repo.
I would definitely recommend you follow the commonly followed rule of "1 repo == 1 module", at least for now. This answer has more details about why.
Working with multiple repos
Given you are using Go modules, one approach is you can add a replace directive to a module's go.mod file that informs that Go module about the on-disk location of other Go modules.
Example structure
For example, if you had three repos repo1, repo2, repo3, you could clone them so that they all sit next to each other on your local disk:
myproject/
├── repo1
├── repo2
└── repo3
Then, if repo1 depends on repo2 and repo3, you could set the go.mod file for repo1 to know the relative on-disk location of the other two modules:
repo1 go.mod:
replace github.com/me/repo2 => ../repo2
replace github.com/me/repo3 => ../repo3
When you are inside the repo1 directory or any of its child directories, a go command like go build or go test ./.... will use the on-disk versions of repo2 and repo3.
repo2 go.mod:
If repo2 depends on repo3, you could also set:
replace github.com/me/repo3 => ../repo3
repo3 go.mod:
If for example repo3 does not depend on either of repo1 or repo2, then you would not need to add a replace to its go.mod.
Additional details
The replace directive is covered in more detail in the replace FAQ on the Modules wiki.
Finally, it depends on your exact use case, but a common solution at this point is to use gohack, which automates some of this process. In particular, it creates a mutable copy of a dependency (by default in $HOME/gohack, but the location is controlled by $GOHACK variable). gohackalso sets your current go.mod file to have a replace directive to point to that mutable copy.
go get is transitive, so you can just add it to your build process. A typical Go project build is basically:
go get -u ./... && go test ./... && go build ./cmd/myapp
Which gets & updates dependencies, runs all project tests, then builds the binary.
Dep has a very useful option - lock given a package for branch via constraints. You just specify a branch for a package and dep ensure -update update this package only within a selected branch.
In go modules I have to manually type go get -u package#branch and if I want to update all my modules by go get -u it will change branch in package to branch with most recent commit if any.
Is it possible to have dep-like behavior or automate this workflow?
A request for this feature is tracked at https://golang.org/issue/26964.
My Problem
Elastic Beats is an open source project for log shippers written in Go. It features several log outputs, including console, Elasticsearch and Redis. I would like to add an output of my own - to AWS Kinesis.
I have cloned the repo to ~/github/beats, and tried building it:
$ cd filebeat; go build main.go
However, it failed due to a missing library which is a part of the project:
main.go:6:2: cannot find package "github.com/elastic/beats/filebeat/cmd" in any of:
/usr/local/go/src/github.com/elastic/beats/filebeat/cmd (from $GOROOT)
/Users/adam/go/src/github.com/elastic/beats/filebeat/cmd (from $GOPATH)
A directory of the project is dependent on a package from the same repo, but instead of looking one directory up the hierarchy it looks in the GOPATH.
So, go get github.com/elastic/beats/filebeat/cmd fetched the code, and now go build main.go works. Changing the code in my GOPATH is reflected in these builds.
This leaves me with an structural inconvenience. Some of my code is at a working directory, and some of it is at my GOPATH and included by the code in my working directory.
I would like to have all my code in a single directory for various reasons, not the least being keeping everything under version control.
What Have I Tried
Mostly searching for the problem. I am quite new to Go, so I might have missed the correct terminology.
My Question
What is the right way to edit the code of an imported library in Go?
One of the recommended ways to work with other's packages is:
Get the sources of the original package:
go get github.com/elastic/beats
As a result you will clone project's git repository to the folder
$GOPATH/src/github.com/elastic/beats
Make some fixes, compile code, fix, compile... When you make go install package will be compiled and installed to your system. When you need merge updates from original repository you can git pull them.
Everything is OK. What's next? How to share your work with others?
Fork project on github, suppose it will be github.com/username/beats
Add this fork as another remote mycopy (or any other name you like) to your local repository
git remote add mycopy git://github.com/username/beats.git
When all is done you can push updated sources to your repo on github
git push mycopy
and then open a pull-request to original sources. This way you can share your work with others. And keep your changes in sync with mainstream.
Previous answers to this question are obsolete when developing projects that using Go Modules.
For projects that using Go Modules, one may use the following command to replace an imported library(eg. example.com/imported/module) with a local module(eg. ./local/module):
go mod edit -replace=example.com/imported/module=./local/module
Or by adding the following line into the go.mod file:
replace example.com/imported/module => ./local/module
Reference Docs: https://golang.org/doc/modules/managing-dependencies#unpublished
A project working copy should be checked out into $GOPATH/src/package/import/path - for example, this project should be checked out into /Users/adam/go/src/github.com/elastic/beats. With the project in the correct location, the go tooling will be able to operate on it normally; otherwise, it will not be able to resolve imports correctly. See go help gopath for more info.
I have big problem with configuring TC. It's 10.0.2 version.
I want build chain like this:
Main - Restore nuget and rebuild solution.
Code analysis - Analyse code result(do not checkout) use Main as dependency.
Publish - Publish to Azure - Use result of Main.
I set Main to:
Build numer format:%build.counter%.%build.vcs.number....%
VCS checkout dir: auto
Code analysis
Build number format:%build.counter%.%dep.<mainId>%.%build.vcs.number...%
VCS checkout dir:%dep.<mainId>.build.default.checkoutDir%
And the main dir is: 55660246e9f668c3
And Code Analysis searching in: 9ccd5731845f5aba
So it's wrong. Why?
Why?
EDIT:
What I set VCS checkout directory in "Code Analysis" build configuration to hardcoded directiory name of "Main" e.x. to 55660246e9f668c3 then it work.
So the problem is with %dep.<mainId>.build.default.checkoutDir%
You can set up a snapshot dependency, that builds from the same chain. This will ensure that the same branch, from the same root, with the same revision number (point in time) is checked out to the directory. If you use an artifact dependency, in addition to the snapshot dependency, you can achieve the same point in time consistency. So after your step 1 build runs, regardless of what new changes exist, your second build will be working with the same files your first had.