So I have written a module A and a dedicated test-file. This module can be used alone, but I wrote it to be used as a "base" for a second module I wrote. This second module absolutely needs the first one, and also has it's own test-file.
Finaly, I wrote a third module that is based on the first two modules. In other words :
module A, can be used without B or C.
module B needs module A, and can be used without C
module C needs module B (and so also A)
My question is how do treat all these modules ? For example, should I make each module a package, and then import A in B, and B in C ? Or should I put them all in a single package ? Also, what do I do of all the test files (put them next to the module, or all in a single test-folder) ?
As of today, I am treating each one as a package, but it seems a bit heavy to have to install A and B for using C:
+ moduleA
- moduleA.py
- test_moduleA.py
+ moduleB
- moduleB.py
- test_moduleB.py
+ moduleC
- moduleC.py
- test_moduleC.py
So I was thinking about merging all like this :
+ moduleC
- moduleA.py
- moduleB.py
- moduleC.py
+ tests
- test_moduleA.py
- test_moduleB.py
- test_moduleC.py
Is that the pythonic way to wrap my module C (and all its components) ? Or should I nest the modules in subpackages (moduleC.moduleB.moduleA) ?
The goal of all this is to export to github-like platform, and eventually pip.
The modules seem to be independent so they should be developed separately: separate development directories, separate git repositories.
For proper installation the modules should declare dependencies: B depends on A and C directly depends on B and indirectly (transitively via B) depends on A. Such dependencies should be declared in setup.py:
In B:
setup(
…
install_requires=[A]
…
)
In C:
setup(
…
install_requires=[B]
…
)
No need to declare dependency on A as B when being installed brings A with it.
This way when installing pip install A A will be installed alone;
when installing pip install B B will be installed with A;
when installing pip install C C will be installed with both A and B.
Related
I am developing multiple microservices, which require different modules (which shall be available like modules on github, but private)
My first tests with Go were all located within one package, which gets quite messy after some time
I'm coming from the Java side of programming - with loads of packages - which keep stuff clear and clean.
(This also works with public modules like on github.com/xyz/module1 github.com/xyz/module2 github.com/xyz/module3)
I just need this for private modules - how can I do so?
This is what I tried yet:
Directory of my go sources:
C:\my\path\to\go\src\
In this dir I have multiple subdirs containing modules (actually more than listed here)
my-module1
my-module2
my-module3
For each folder I called go mod init but I get the message that
package my-module1/util is not in GOROOT (c:\go\src\my-module1\util)
which is obviously right, as my private libraries reside in C:\my\path\to\go\src\
Importing packages from github with go get ... is working without troubles (those packages will be loaded but copied to c:\go\src)
Working with all files in one folder works but is not the desired solution (I need to create multiple microservices therefore I want to be able to create different projects with custom executeables and or tests)
What am I doing wrong?
If more information is needed I will provide it - just let me know what
NOTE: packages without go file in package main cannot be installed via go install. This system looks pretty confusing to me - as modules cannot be found...
finally i created a multimodule
means i have a root folder (lets call it root) which contains a go.mod file
this defines the root module
it requires all my submodules
and it replaces the absolute path into a relative.
root
* aa
* ab
* bc
main gets defined as
module root
go 1.15
require (
root/aa v0.0.0
root/ab v0.0.0
root/bc v0.0.0
)
replace (
root/aa => ./aa
root/ab => ./ab
root/bc => ./bc
)
all the submodules have a very simple definition (this is equal for all, as long as you do not have any dependencies)
module root/aa
after searching through all the related posts, I was still unable to find an answer to my exact problem, so I hope someone here can help me out.
What I have is a project with 3 modules, where 2 modules use the same 'base'module. eacht module must be deployed to a different server with different provided libraries.
It looks like this:
Project
|
-- Modules
|
-- Module Base
|
-- Module A
|
-- Module B
pom file of Project has dependency managment where library 1 is declared with version 1.0
pom file of Module Base has dependency on library 1.
pom file of Module A has dependency on Module Base.
pom file of Module B has dependency on Module Base.
Module A is deployed on a server X where library 1 is provided.
Module B is deployed on a server Y where library 1 is NOT provided.
On server X the size of Module A must be kept as small as possible, so library 1 cannot end up in the distributable of Module A.
How should I configure the pom files to get everything build and running correctly?
I tried a whole bunch of possibilities already but am unable to get it correct so that the assembly of A does not have library 1 present, and the assembly of B does have library 1 present.
All help would really be appreciated.
Well it very much depends on what in reality depends on what. What you've declared in the POMs may or may not represent the reality. As I don't know that, I'll just give you the options you have:
Option 1
Move the dependency on library 1 from Module Base to Module A. Since you expect module B to be able to work with Module Base but without library 1 it probably means that Module Base does not really need library 1
Option 2
If the above is not the case and Module Base does need library 1 sometimes but can work without it other times - you should make the dependency optional. Then you can add it to Module A and not add it to Module B
Option 3
In the weird case when library 1 is not optional for Module Base but you happen to somehow know that Module B does not touch code from Module Base that needs the library - exclude the dependency on library 1 when you define the dependency on Module Base in Module B
For more information about both option 2 and 3 please have a look at "optional and excludes dependencies" docs.
UPDATE:
After the clarifying comment it appears that library 1 is needed by all modules and the difference is only whether or not it is provided at runtime. In that case, you need to copy the dependency from Module Base to Module A and change the dependency scope to provided for Module A. That way, assuming your packaging mechanism understands Maven scopes, Module B will be using the compile scope via Module Base and so library 1 will be added to the final package while Module A will know it's provided and should not have it in the package.
For example, given a package A that depends on package B and package C, where package C also depends on package D - is there a way to output this information? (Using a vendoring tool or otherwise)
The vendor.yaml output by govend doesn't include transitive dependency information - neither does the Gopkg.toml file
output by dep, from what I can see. The go.mod file produced by Golang 1.11's mod and does annotate some dependencies as // indirect - but it does not annotate dependencies with any information about which dependency they were pulled in via.
Did you try https://github.com/KyleBanks/depth?
It does provide a decent dependency tree at first look I tried.
For OSGi bundle, we can add dependencies by using Required-Bundle or Imported-Package. Now I have below case:
Both bundle A and bundle B export same packages, but bundle B with higher version, like:
bundle A:
Exported Packages:
xxx.a,
xxx.b,
xxx.c
bundle B:
Exported Packages:
xxx.a; version="5.0.0",
xxx.b; version="5.0.0",
xxx.c; version="5.0.0"
And bundle C has the dependency to package a, b, c, so it adds A in its Required-Bundle list, like
bundle C:
Required Bundles:
A
Let's assume both bundle A and B are resolved in the framework. So when resolving bundle C, which packages/classes are been loaded by C, the classes in A or classes B? I think is A. Am i right?
Yes, the packages from A are used because you require bundle A.
Please note that Require-Bundle is considered deprecated by most OSGi developers.
I have bundles A, B and C. A contains a package 'a' that is depended on by code in packages b and c from bundles B and C respectively.
bundle A {
package a
export-package:a
}
bundle B {
package b.a
package b.b
package b.c
package b.d
package b.e
import-package: a
export-package:b.a;uses:=a,
b.b;uses:=b.a,
b.c;uses:=b.b,
b.d;uses:=b.c,
b.e;uses:=b.d
}
bundle C {
package c
import-package: b.e,
a
}
When I update all of these bundles together, I often get uses constraint violations (Felix reporting style):
Chain 1:
C [47.1]
import: (&(osgi.wiring.package=a)(version>=1.1.0))
|
export: osgi.wiring.package=a
A [9.1]
Chain 2:
C [47.1]
import: (&(osgi.wiring.package=b.e)(version>=1.0.0))
|
export: osgi.wiring.package=b.e; uses:=a
B [33.0]
import: (&(osgi.wiring.package=a)(version>=1.0.0))
|
export: osgi.wiring.package=a
C [9.0]
I was initially surprised that b.e generates a uses clause for 'a'. This is not declared in the Manifest nor the OBR repository.xml through which these bundles are provisioned. However, types in b.e expose types in a through their API, so I guess this is where it comes from.
The only way I find to resolve these is to increase the version number of the export and import of intermediate packages, e.g. b.e in this example. However, there are a lot of packages that ultimately use 'a', transitively.
This means every time I update 'a' I also need to update all transitive dependencies on a with new versions. Is it the correct thing to be doing to increase the version number of these packages? Is refactoring my code to be less interdependent the only other way?
The uses constraints are not handled transitively by the resolver.
If bundle C imports b.e and b.e used b.d, then bundle C also needs to import b.d. That is, the public signature of b.e includes a reference to something in b.d. This is why we say b.e uses b.d. Therefore, any bundle which imports b.e must also import b.d. Otherwise the bundle cannot properly use b.e since some type referenced in b.e is not visible to the bundle.
For the export of b.e, it should state it uses b.d, b.c, b.b, b.a and a since using b.d means you also use b.c and so on...
Did you hand generate your uses clauses? Or was this example derived from something bnd generated?