Error scanning github.com/golang/protobuf/proto/testdata: cannot find package "." - go

When running glide install on my project, I get the following error:
[ERROR] Error scanning github.com/golang/protobuf/proto/testdata: cannot find package "." in:
/Users/bevernie/.glide/cache/src/https-github.com-golang-protobuf/proto/testdata
[ERROR] Failed to retrieve a list of dependencies: Error resolving imports
When checking protobuf's source code, I can in fact see that there is no such package. I however don't directly use protobuf, so the error must come from one of the dependencies I use.
When running glide tree on my project, there is only one instance of github.com/golang/protobuf/proto/testdata:
|-- github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto)
| |-- github.com/golang/protobuf/proto/test_proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto/test_proto)
| | |-- (Recursion) github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto)
| |-- github.com/golang/protobuf/ptypes/any (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/ptypes/any)
| | |-- (Recursion) github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto)
github.com/golang/protobuf/proto/testdata (glide get github.com/golang/protobuf/proto/testdata)
|-- github.com/golang/protobuf/ptypes/any (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/ptypes/any)
| |-- github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto)
| | |-- github.com/golang/protobuf/proto/test_proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto/test_proto)
| | | |-- (Recursion) github.com/golang/protobuf/proto (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/proto)
| | |-- (Recursion) github.com/golang/protobuf/ptypes/any (/Users/bevernie/Programmation/work/src/github.com/golang/protobuf/ptypes/any)
which doesn't really help me pinpoint the source of the problem.
Do you have any suggestions as of how to fix the issues?
My project was compiling just fine until a week or two ago (I use Docker to deploy in production, so the glide install was run every time and never failed before that, and I haven't added any new dependency recently).

Before your own PR (995), there was glide issue 968
It looks like it's caused by a repository's structure changing, i.e. a sub-package being moved, or removed entirely.
Workarounds proposed by Elliot Wright (seeruk):
If the package that has been updated is under your own control, then I've since found it easier to use some of the newer Go features like type aliases to ease the pain from refactoring.
So, instead of just moving a package, move it and then make aliases to the new location in the old one so that your older code still works.
Then, gradually move things over. Basically just mark things as deprecated but make sure they're still usable for a little while until you've ported new code over.
If the package is not in your control, then you can always clone the version you want manually to your vendor folder and make your updates in your code.
Once you're done, Glide should let you update again.
If it's much more complex, sometimes it's even easier to revert to using go get until you're done updating packages, and rely on your $GOPATH contents.
It's far from ideal, but there are ways you can work around it at least.
In the mean time, I've also made an issue about this on dep.
I think they're looking into a way of disabling this kind of check if you just want the tool to trust you as the developer.
So you can check if you have the same issue using godep, or even the bleeding-edge vgo.

Related

Why does Yarn throw "Invariant Violation: expected workspace package to exist" when I attempt to upgrade some dependencies?

I have a monorepo that is managed by lerna which relies on the Yarn workspaces feature. It looks like this:
/repo
|-- packages
| |-- pkg-a
| | |-- package.json
| |-- pkg-b
| | |-- package.json
| |-- pkg-c
| | |-- package.json
|-- package.json
I use yarn upgrade-interactive to update a selection of dependencies across my packages but I get this error:
Invariant Violation: expected workspace package to exist for "#babel/plugin-transform-classes"
However this isn't a direct dependency of mine. When I search for it in all my package.json files I get nothing:
$ find . -name "package.json" -and -not -path "*/node_modules/*" | xargs grep "#babel/plugin-transform-classes"
$
It is present in my yarn.lock file though:
$ grep "#babel/plugin-transform-classes" yarn.lock
"#babel/plugin-transform-classes#^7.10.4":
resolved "https://registry.yarnpkg.com/#babel/plugin-transform-classes/-/plugin-transform-classes-7.10.4.tgz#405136af2b3e218bc4a1926228bc917ab1a0adc7"
"#babel/plugin-transform-classes#^7.4.0":
resolved "https://registry.yarnpkg.com/#babel/plugin-transform-classes/-/plugin-transform-classes-7.4.0.tgz#e3428d3c8a3d01f33b10c529b998ba1707043d4d"
"#babel/plugin-transform-classes" "^7.4.0"
"#babel/plugin-transform-classes" "^7.10.4"
$
I suspected some sort of caching issue so I tried to start from scratch inside a Docker container:
Delete all node_modules folders
Mount the repo in a Docker container
Run yarn to install all dependencies
Run yarn upgrade-interactive
But I get the exact same error.
I went scavenging for answers and none have fixed my issues. What strikes me the most is that nobody seems to know why it is happening.
Can anybody explain why it is happening and how one could reproduce this issue? Once we know why, we stand a better chance to find a proper solution.
Technical details:
Node 12.18
Yarn 1.22
Lerna 3.20
It seems that it is a bug
https://github.com/yarnpkg/yarn/issues/7734#issuecomment-580012389
And the official recommendation is to downgrade to Yarn 1.19.x or upgrade packages manually (Yarn 1 is deprecated).
https://github.com/yarnpkg/yarn/issues/7734#issuecomment-671729912
Also, you could try to use https://www.npmjs.com/package/npm-check-updates CLI utility
Try after removing the packages from resolution section of the package.json file which you want to upgrade.

Go module for multiple packages, with new major number >1

I am trying to migrate an existing Go project to use modules but having difficulties working out the correct directory structure. The repo contains two regular packages (p1, p2) along with example programs to demonstrate how to use the packages. I am using Go 1.13.
The current layout is
<reponame>
|
+--- p1
| |
| +-- p1a.go
| +-- p1b.go
|
+--- p2
| |
| +-- p2a.go
| +-- p2b.go
|
+-- examples
|
+-- e1.go (which refers to packages p1 and p2)
The github repository is already public with a version tag > 1. I know I need a new major number, v5, so it looks like I need to move to having v5 somewhere in the path to the packages.
But should it be <repo>/v5/p1 or <repo>/p1/v5? Where do go.mod files go? I suspect I need one to cover the two packages, and one for the example programs.
All the examples I can find for version > 1 seem to only have a single package in the module.
And importantly, how do I get the examples to compile and run? I've tried all kinds of permutations of a go.mod file for the examples but they all keep failing with an error like "no matching versions for query 'v5'" even with the "replace" directive to try to point at the local directories.
It looks like it is expecting a v5 version of code to be already published to github before I've done local testing.
Modules are about versioning. If p1, p2 and example are versioned together then put the go.mod at the repo level. This is probably the what you want.
Make sure the module declares itself properly, i.e. module <repo>/v5 in the go.mod file.
You do not need the v5 in the file system path. There are basically three ways: A v5 folder containing the v5 stuff, a v5 branch containing the v5 stuff or none of the two in which case your repo will be v5 only.
Make sure the import paths are correct. E.g. in package examples you must import p1 via import "<repo>/v5/p1". (Same for p2 or even p1 importing p2, etc.)
Thats all.
cd into the example folder and go build: Go will look up the filesystem tree for a go.mod and will find it at the repo root. It thus knows that this package belongs to module <repo>/v5 and thus where to find all packages from the module <repo>/v5 and can import <repo>/v5/p1 without the need for any replace directive. Like this you can work locally on the v5 without need to push to a remote repo.
(A common mistake is to not declare the module as v5: In the go.mod file you must have a line module github.com/<user>/<repo>/v5. If you falsely just write module github.com/<user>/<repo> then when compiling e.g. examples the compiler thinks "well, the examples package belongs to module github.com/<user>/<repo> and I should import github.com/<user>/<repo>/v5/p1 so let's peek at github.com// to see what we find there..." and wont find anything and complain.)

Go package binary reference

I have built a go package (https://github.com/emicklei/go-restful) and I am now trying to reference the output of that package build from my own workspace.
$GOPATH
|--src
| |-- {my source}
|--pkg
| |-- {package binaries /linux_amd64/github.com/emicklei/go-restful.a}
|--bin
|-- {my output}
but this fails to compile stating can not find package. go build is only searching the src folder as AFAIK and all the solutions I can find around this involve having the package source code, but surely it's possible to reference package binaries?
Turns out it's not possible to reference a package binary, in order to reference a package the source code is required: http://zduck.com/2014/go-and-package-versioning/
============ EDIT ==============
Turns out you can hack it so that the compiler will use the package output by placing a dummy source file in the expected location with a last modified older than the package output. (see Volker answer to original question).

Maven - Custom output project structure

Good evening, people.
I've already wasted the whole day trying to manage with maven.
I have the following project structure:
base-pom.xml
|-module1
| |-module1-resources
|-module2
| |-module2-resources
|-module3
| |module3-resources
|
|-general-resources
I'm looking for some way of getting the following output structure:
output-folder
|-libs-folder/3rdrarty-jars
|-config-folder/general-resources not-packed
|-module1.jar
|-module2.jar
|-module3.jar
|-module1-resources not-packed
|-module2-resources not-packed
|-module3-resources not-packed
Also a thing to mention: classpath of my project modules should be something like this:
Class-Path: /config-folder module2.jar module3.jar libs/3rdparty_1.jar libs/3rdparty_2.jar libs/3rdparty_3.jar
I don't know if maven allows to have such a complicated output structure - all samples, i've found having wasted a lot of time, show nothing even similar to my requirements.

java.lang.VerifyError: Incompatible argument to function

I have a mavenized Spring 3 project that builds and runs fine on one machine. The exact same project builds fine on a second machine, but when I try to hit a page (one that works fine on the other machine), I get the following stacktrace:
java.lang.VerifyError: (class: org/apache/jsp/tag/web/generate_002dvalidation_tag, method: _jspx_meth_c_005fset_005f13 signature: (Ljavax/servlet/jsp/tagext/JspTag;Ljavax/servlet/jsp/PageContext;[I)Z) Incompatible argument to function
java.lang.Class.getDeclaredConstructors0(Native Method)
java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
java.lang.Class.getConstructor0(Class.java:2699)
java.lang.Class.newInstance0(Class.java:326)
java.lang.Class.newInstance(Class.java:308)
org.apache.jasper.compiler.TagFileProcessor.loadTagFile(TagFileProcessor.java:635)
org.apache.jasper.compiler.TagFileProcessor.access$000(TagFileProcessor.java:52)
org.apache.jasper.compiler.TagFileProcessor$TagFileLoaderVisitor.visit(TagFileProcessor.java:685)
org.apache.jasper.compiler.Node$CustomTag.accept(Node.java:1530)
org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2361)
org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2411)
org.apache.jasper.compiler.Node$Visitor.visit(Node.java:2417)
org.apache.jasper.compiler.Node$Root.accept(Node.java:495)
org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2361)
org.apache.jasper.compiler.TagFileProcessor.loadTagFiles(TagFileProcessor.java:703)
org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:210)
org.apache.jasper.compiler.Compiler.compile(Compiler.java:347)
org.apache.jasper.compiler.Compiler.compile(Compiler.java:327)
org.apache.jasper.compiler.Compiler.compile(Compiler.java:314)
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:589)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:317)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:238)
org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:250)
org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1047)
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:817)
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
The only difference that I can think of is the version of Java. On the machine where the project works, the version is 6 update 17, whereas on the second machine (where the project does not work), the version is 6 update 22. The pom's are exactly the same.
It looks like the problem is centered around a custom tag, but I can't figure out what it is. What could be causing this problem?
UPDATE
I took a look at the target directories on both machines and noticed the following:
On the machine where the project doesn't work, the lib directory has el-api-2.2.jar
On the machine where the project works, there is tomcat directory under target which contains the following:
`-- tomcat
|-- conf
| |-- tomcat-users.xml
| `-- web.xml
|-- logs
|-- webapps
`-- work
`-- localEngine
`-- localhost
`-- _
|-- org
| `-- apache
| `-- jsp
| |-- tag
| | `-- web
| | |-- generate_002dvalidation_tag.class
| | `-- generate_002dvalidation_tag.java
| `-- WEB_002dINF
| `-- jsp
| `-- starship
`-- SESSIONS.ser
This directory is not present on the machine where the project works
On the machine where the project works, there is a war directory under target, which is not present on the machine where the project does not work (however both machines produce a war file under the target directory)
On the machine where the build does not work, the war file is 4,135,195 bytes, whereas on the other it is 4,104,569 bytes. This difference comes from the inclusion of the el-api-2.2.jar file.
I'm not sure what this means.
According to this answer,
java.lang.VerifyError can be the
result when you have compiled against
a different library than you are using
at runtime.
I suggest you to compile it on each machine and compare the content within the war file (assumming, from the stacktrace, you are building war project).
Do you happen to compile it on linux vs Windowsy by any chance? It is possible that you may have the same library with the different version within the classpath. On different OS, the order at which the class are loaded are different. The correct one maybe loaded first on your machine running JDK 6u17.
I normally open the war file in a 7zip browser and check whether there are any same library of different versions. Some libraries use the different artifact name but actually the same, e.g. spring-bean and org.springframework.bean.
According to my experience, sometimes the verifications get complex generics-related things wrong. Also, sometimes instrumentation can pose problems to it. If you know that the verification error should not appear, but it still does, verification can be disabled with -noverify startup option.
Sometimes it is also disabled for other reasons such as performance.
For more details on disabling verification, see this thread.
More than java version, I think the problem is due to the difference in the versions of Java EE libraries.
Is it possible that the two machines have different app servers or different versions of the app server? Also, are the libraries provided by the container (like servlet-api.jar or jsp-api.jar) being packaged in the war?
Though the reason mentioned by gigadot is correct, but I would definitely check below before moving to something else:
Check the cglibs in my classpath.
Check the hibernate versions in my classpath.
Chances are good that having multiple or conflicting version of any of the above could cause unexpected issues like the one in question.

Resources