I'm having trouble with a directory dependency in a parallel build in SCons.
Consider two projects with a single SConstruct in the following (simplified) hierarchy:
- SConstruct
- project1
- src
- project2
- src
- build
- project1
- project2
- dist
- project1
- project2
Each of project1 and project2 are supposed to be built under the relevant build directory (using variant dir) and several targets needs to be installed under the relevant dist directory.
Project 2 depends on Project 1's dist. I've states this dependency explicitly using the Depends() statement like so:
Depends('project2', 'dist/project1')
When I use a non-parallel build, there's no problem. Project 1 is fully built, targets are installed in the dist directory, and only then project 2 is built. However, when I use multiple jobs (4), project 2 is being built simultaneously to the Install() builder being run for the files needed to be installed in project 1's dist directory.
So, my questions are:
Does the Depends(project2, dist/project1) statement refers to the creation of the dist/project1 directory or to the completion of building all the directory's children?
How should I solve this issue?
Thank you very much,
BugoK.
Instead of specifying the actual directories as strings in the Depends() function, try specifying the actual targets as returned by the SCons project1 and project2 builders. Every SCons builder (or at least most of them) returns the affected target as an object, and its better to use this object instead of the file/directoy name since if you dont use the exact same file/directory path, it wont be considered as the same target.
Here is an example, fill in content accordingly:
project2Target = Install()
# Im not sure how you're building project1, so replace the builder
project1Target = Proj1DistBuiler()
Depends(project2Target, project1Target)
Related
This question already has an answer here:
How to use an alternate go.mod file for local development?
(1 answer)
Closed 10 months ago.
I would like to know how can I build a go project using a different go.mod file. Suppose I want to build project A inside project B module using project B go.mod file without copying the files around. That means I want to use dependencies in Project B to build Project A.
Manual option
"Module files" refers to both go.mod and go.sum
Rename or move project A's module files to some temporary names / location
Copy project B's module files into project A
Edit the newly copied go.mod file in project A, and change the module name:
module github.com/x/b changes to module github.com/x/a
Build whatever you need to build in project A
Delete the active module files in project A
Restore the proper module files for project A that you renamed or moved in step 1
These steps could be automated with a shell script or batch file if you need to do it often.
With build command
Using the go help build command, we can see the build flag -modfile
-modfile file
in module aware mode, read (and possibly write) an alternate go.mod
file instead of the one in the module root directory. A file named
"go.mod" must still be present in order to determine the module root
directory, but it is not accessed. When -modfile is specified, an
alternate go.sum file is also used: its path is derived from the
-modfile flag by trimming the ".mod" extension and appending ".sum".
Using this, we can directly use an alternative set of module files to build things in project A.
First, the flag description indicates that it may write to the go.mod file, so it's probably still a good idea to create a copy of project B's module files to do this.
Second, using project B's module file is going to be a problem if: 1. project A and project B have a different module name declared in their module file, and 2. packages in project A import other packages in project A. The module name determines what the import path of packages in the module will be, so changing it could break imports.
So the best practice should still be to:
Make a copy of project B's module files
Change the module name in the copy
Then you can run the build command like this to build in project A:
go build -modfile path/to/projectb/go.mod
First, make a folder b somewhere. Then make a folder a inside b. Then make
b/b.go:
package b
const Something = 1
Then make b/a/a.go:
package a
import "b"
func something() int {
return b.Something
}
Then go back to b folder, and do go mod init b. Done.
I have a multi-project gradle project with following directory structure:
+ project_root
+ module1
+ src
build.gradle
+ module2
+ src
build.gradle
+ web
..
build.gradle
settings.gradle
In module1/build.gradle among other things I have specified:
compileKotlin2Js.kotlinOptions {
outputFile = "web/script.js"
}
It is a special Kotlin JS setting that specifies output file path of compiled JS file.
Now my problem is, that when I build the whole project (project_root/build.gradle) the file ends up in the right directory (project_root/web), but when I accidentally run build on the module alone the file is saved in module directory (project_root/module1/web).
How can I fix paths in my build scripts, so file output will be saved in exactly the same directory no matter which build script I run (without specifying full path, I want a relative path)?
I don't know what Gradle plugin requires the path parameter in your code example, but all regular (non-3rd-party) Gradle plugins evaluate path parameters via Project.files(Object...) to avoid different locations when calling Gradle from various working directories.
I would suggest to use the above method (or its single file version Project.file(Object)) as well. You can even omit the project part, because the build.gradle file gets executed in the project scope:
compileKotlin2Js.kotlinOptions {
outputFile = file('web/script.js')
}
This will always evaluate the path relative to the project directory of the project your build.gradle belongs to. To evaluate a file relative to the project directory of the root project, use rootProject.files(Object...), for a path relative to the project directory of a subproject or any project in the build, use project(':path:to:project').files(Object...).
I have a CMake project using Makefiles on Windows, with a folder structure that looks like this (the build takes place in build):
project
|- build
|- ...
|- otherfolder
|- stuff
|- more stuff
As a build step (pre- or post doesn't matter), I want to make a copy of project into build (excluding the build folder), like so:
project
|- build
|- ...
|- project
|- otherfolder
|- stuff
|- more stuff
|- otherfolder
|- stuff
|- more stuff
Other options might be acceptable as well, e.g. copying to a temporary directory outside the project root before moving it into place, but CMake seemingly has no builtin support for generating temporary directories.
Things I've tried: xcopy has support for excluding certain files and directories, but refuses to copy even if I explicitly exclude the build folder. cmake -E copy_directory does not (from what I'm able to find) support excluding certain directories.
CMake's file(COPY ... PATTERN build EXCLUDE ... copies successfully, but it runs at CMake configure time and I haven't been able to find a way to make it run at build time.
I might resort to using Python and shutil, but it would be nice if it could be done without additional dependencies, so I'd prefer a batch file solution.
There are several ways for doing selectable directory copiing.
You can use cmake -P for execute cmake script at any time. E.g:
copy_to_build.cmake:
file(COPY . DESTINATION build PATTERN build EXCLUDE)
CMakeLists.txt:
add_custom_command(... COMMAND cmake -P copy_to_build.cmake)
You can prepare list of subdirectories (and files) at configuration stage, and then copy every element of that list using xcopy. This approach uses fact, that everything outside of build directory is not changed. Here iteration is done on configuration stage (by CMake). I am not sure, whether "for" loop works under COMMAND of add_custom_command. If it works, you can use it for iterate over entries in the shell.
CMakeLists.txt:
# List of elements in source directory.
file(GLOB entries RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/*)
# List of commands for pass to `add_custom_command` as is.
# `COMMAND` keyword is included into list.
set(copy_commands)
foreach(entry ${entries})
list(APPEND copy_commands COMMAND xcopy /s /i
${CMAKE_CURRENT_SOURCE_DIR}/${entry} ${CMAKE_CURRENT_SOURCE_DIR}/build/${entry}
endforeach()
add_custom_command(... ${copy_commands})
I have a solution containing multiple projects. I can use project variables within Build Events, for example $(TargetDir) to access paths and files of the current project.
Solution 'MyApp'
MyApp
MyApp.Core
MyApp.Setup
I want to add a Pre-Build-Event in MyApp.Setup. The $(TargetDir) would return D:\MyApp\MyApp.Setup\bin\Debug, but I want to gather all files from MyApp-Output Directory and put them into the setup directory.
So is it possible to access variables from other projects within build events?
Something like that:
copy "$(MyApp.TargetDir)\*.*" "$(ProjectDir)\externals"
I propose you build all projects of solution into one folder, and inside that folder create specific project subfolders. For now your projects have TargetDir variable with values:
MyApp - D:\MyApp\MyApp\bin\Debug
MyApp.Core - D:\MyApp\MyApp.Core\bin\Debug
MyApp.Setup - D:\MyApp\MyApp.Setup\bin\Debug
But you can make common target directory, so TargetDir value:
MyApp - D:\MyApp\Debug\MyApp\bin
MyApp.Core - D:\MyApp\Debug\MyApp.Core\bin
MyApp.Setup - D:\MyApp\Debug\MyApp.Setup\bin
And in Pre-Build-Event you can refer to it:
copy "$(TargetDir)\..\MyApp\*.*" "$(ProjectDir)\externals"
I'm working on a gradle multi project (java), where some of the sub-projects are applications (have a main class, and are executable) and some others are libraries (e.g, ends up being packaged as a jar, which some other sub-projects define a dependency on). It's working fine.
I'd like to be able to package (tar) the entire project for production, according to some structure that will make it easy for my users to deploy and use later on.
Currently, the distTar task creates a build/distribution/project-name.tar for each application project, and a build/libs/project-name.jar for each non application project, under each project build directory. That's a step in the right direction, but I'd like to consolidate the contents into one thing I can distribute.
As an example, right now after running distTar:
myapp/
README
docs/
services/
service1/
build/
libs/service1.jar
<other build dirs I don't want to distribute>
service2/
build/
distributions/service2.tar
<other build dirs I don't want to distribute>
service3/
build/
distributions/service3.tar
<other build dirs I don't want to distribute>
and the contents of service2.tar are:
service2/lib/service2.jar
service2/lib/some-service2-dependency.jar
service2/bin/service2 (start script)
service2/config.yml
(and similarly for service3.tar).
I'd like my final result to be a single myapp.tar(.gz) that includes a similar directory structure, but only with the production files:
README
docs/
services/
service1/
lib/service1.jar
service2/
lib/service2.jar
lib/some-service-dependency.jar
bin/service2 (start script)
config.yml
service3/
lib/service3.jar
lib/some-service-dependency.jar
bin/service3 (start script)
config.yml
Not sure what is the best way to achieve such a task. Do I create a parent level task that depends on distTar and copies files around, untar'ing stuff, etc? (tried that unsuccessfully).
Many thanks!
UPDATE:
I started doing something along these lines:
distributions {
main {
contents {
into('scripts') {
from {'scripts/sbin'}
}
into('service1') {
from {tarTree(tarFileForProject("service1"))}
}
into('service2') {
from {tarTree(tarFileForProject("service2"))}
}
into ...
}
}
}
distTar.dependsOn([subprojects.clean, subprojects.distTar])
(where tarFileForProject is a simple function that returns the path to the build/distributions/*.tar file of the given subproject name).
It seems to work, but also seems ugly. I wonder if there's a cleaner way to do this.