In Gradle there is a copy task where I can specify a from, a into and a duplicatesStrategy. However, the duplicatesStrategy is only an enum with limited options like override, ignore or fail.
Instead of just overriding I would like to be able to call a "content merge function" which takes the two files (the existing and the new one) as input parameters and outputs the resulting file.
from("fromdir") {
into 'targetdir'
duplicatesStrategy = { newFile,existingFile ->
return mergeFiles(newFile,existingFiles)
}
}
Is that possible in any way?
Context: The task should copy doc book xml files from multiple projects into one. One Project can override parts of the documentation of another project, so there has to be some "logic" to merge the resulting documentation correctly.
I would solve it using
duplicatesStrategy 'exclude'
Then in a separate task iterate and check for differences and handle the merge of them there. Preferably in an own task implementation using a third-party merge tool. My searches found no existing merge tasks.
Related
By default, the ASP.NET Core SPA project templates have a section in their .csproj files like so:
<!-- Include the newly-built files in the publish output -->
<ItemGroup>
<DistFiles Include="$(SpaRoot)dist\**; $(SpaRoot)dist-server\**" />
<DistFiles Include="$(SpaRoot)node_modules\**" Condition="'$(BuildServerSideRenderer)' == 'true'" />
<ResolvedFileToPublish Include="#(DistFiles->'%(FullPath)')" Exclude="#(ResolvedFileToPublish)">
<RelativePath>%(DistFiles.Identity)</RelativePath>
<CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
</ResolvedFileToPublish>
</ItemGroup>
I'm not finding any good documentation for this on Google, but I think this section might help me with what I want to do. I want to modify it for my own purposes but first I need to fully understand it, so could someone explain to me the following:
Where does SpaRoot get set?
What exactly does ResolvedFileToPublish do?
Where does DistFiles get set?
Where does FullPath get set?
What does the #(DistFiles->'%(FullPath)' "arrow notation" mean?
What does Exclude="#(ResolvedFileToPublish)" do?
What does DistFiles.Identity refer to and where does it get set?
UPDATE: This page gives some documentation on this item but not much:
https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/visual-studio-publish-profiles?view=aspnetcore-2.2#selective-file-inclusion
Where does SpaRoot get set?
SpaRoot is set as a property in the project by the template.
Projects contain a "static" portion; that is root-level PropertyGroup and ItemGroup elements.
Note that properties are like a global key-value dictionary (and whenever an XML node in a PropertyGroup defines a property, it overwrites existing ones with the same name).
Items however are like lists; you can add (<MyItem Include="..."/>), remove (... Remove="...") and even update items (... Update="..." in static portions only, no include/remove inside targets means "update all" which you can only filter with a Condition attribute). Items are like objects, they have an "ID" which is called "Identity" and can have other properties, which are called "metadata". The "Identity" is the part that is specified in Include, which may or may not be a file name. If a file is referenced, some well-known metadata is added automatically (such as file modification dates and FullPath). Metadata can also be defined on item XML elements as either attributes (e.g. as seen in Version="1.2.3" on PackageReference items) or as child elements of the item element (e.g. RelativePath as seen above).
What exactly does ResolvedFileToPublish do?
The build is executed in the build engine by running targets that contain logic. All of the build logic that .NET projects run are controlled from MSBuild code that uses the same syntax as the project file. So by using imports or SDKs, the .csproj file itself is a build definition rather than a configuration file. Using mechanisms like BeforeTargets/AfterTargets, one can hook into the build process at specific points to execute code; in this case, the template contains a target that hooks into the publish logic.
ResolvedFileToPublish itself doesn't do anything special. The XML elements tell msbuild to add items to the ResolvedFileToPublish list based on file specifications, one of them only if the project is configured with server-side rendering (which is a property that AFAIK is also present in the static portion of the project in the template).
At a later stage during the build, targets coming from the .NET SDK use these items to compute files to copy during publish operations, tool packaging, and/or single-file publishing (3.0 feature), see Microsoft.NET.Publish.targets for the code that uses #(Microsoft.NET.Publish.targets) to access the list of items.
It is a convention that whenever properties or items are used by Microsoft-/3rd-party build logic that do not start with underscores (_), those are allowed/expected to be configured via build customizations, such as provided in the SPA templates. So we are meant to add ResolvedFileToPublish items which are considered "public API", but not _ResolvedFileToPublishAlways. By adding the built SPA files as items, we can tell the publish logic to include them during publish.
Where does DistFiles get set?
DistFiles is created by this template / logic itself. There is next to no restriction on which item or property names can be used. This could also have been named SpaDistFiles or similar. The template creates some intermediate items it can later use to create ResolvedFileToPublish items and hopes that the name doesn't conflict with any other name used in build logic.
Where does FullPath get set?
Full path is an automatic well-known property that msbuild adds to items that reference files on disk.
While an item's identity could be ClientApp\dist\myapp\index.html (or relative paths containing ..\), its FullPath metadata will be C:\path\to\proj\ClientApp\.....
What does the #(DistFiles->'%(FullPath)' "arrow notation" mean?
While properties can be accessed using the $() syntax, items are referenced using #().
When you have item MyItem with identities A and B, #(MyItem) (when evaluated to text) would be A;B. This could again be interpreted as an item specification, so passed to <OtherItem Include="#(MyItem)" />.
But the #() syntax also allows for item transformations or calling item functions (#(MyItem->Count())). Transformation is a projection of each item to another item, so in this example, #(MyItem->'X') would result in X;X since both items are transformed to the same value. To include parts of the original item, metadata values can be accessed via %(). So #(MyItem->'Hello %(Identity)') would result in Hello A;Hello B, since Identity is default metadata.
In this case, the DistFiles items which contain the path relative to the project file are transformed to reference the full path. While this is not well documented, this is needed since publishing logic expects ResolvedFileToPublish items to contain an absolute/full path since it can also be flown across project references - e.g. a library could contain publish-only assets and the consuming project needs to copy them during publish so it needs to pass the full path and not the relative path, which would not be found in the consuming project.
What does Exclude="#(ResolvedFileToPublish)" do?
An item Include="..." can be filtered to not add items that are part of the Exclude definition.
In this case, the action translates to "Add the full path of DistFiles items as ResolvedFileToPublish items unless there is already an ResolvedFileToPublish item with the same identity (i.e. referring to the same file on disk)".
This is useful so as to not confuse the publish logic with duplicate items. Not sure at the moment if this would actually cause problems, but in order to be a good citizen, it is better not to cause additional file copies / file uploads (web deploy) etc.
The reason the files could already be in there is that they may have already been included by one of the default item specifications defined in the Web SDK that includes e.g. files in wwwroot or similar for publishing, depending on how your project is set up. The template just doesn't want to cause conflicts.
What does DistFiles.Identity refer to and where does it get set?
As mentioned above, items have some default metadata and Identity is one of them. In this case, the DistFiles items are created from file specifications relative to the project, so the item's identities are the project-relative paths (ClientApp\dist\...).
Since ResolvedFileToPublish items contain absolute paths, the RelativePath metadata tells the publish logic where to place the file during publish. You could also use this to rename the files or place them in subfolders.
In a verbose log / structured log, you should see that the items being added are C:\path\to\proj\ClientApp\dist\index.html with RelativePath=ClientApp\dist\index.html and CopyToPublishDirectory=PreserveNewest metadata.
Item batching
In the above code, there is a reference to metadata from within an attribute:
<RelativePath>%(DistFiles.Identity)</RelativePath>
While this tells MSBuild to set the RelativePath metadata to the source DistFiles item's Identity, this also triggers a feature called batching.
For every loose %(Item.Metadata) specification MSBuild sees (note that this only works inside targets) MSBuild groups the referenced item(s) into "batches" having the same property. It then runs the task that this is used in (in our case an intrinsic item add task) once for each batch, in which the #() notation will only yield the items from that particular batch.
When only batching on %(XYZ.Identity), this doesn't really matter and can be seen as a simple "for all".
So to be exact, the <ResolvedFileToPublish Include=... part would translate to: "For each set of DistFiles with the same Identity metadata, transform these items to their full path and, unless a ResolvedFileToPublish with this file name already exists, create a ResolvedFileToPublish item for them with the metadata RelativePath set to the DistFile item's Identity value, and the CopyToPublishDirectory metadata set to PreserveNewest."
I have two separate projects which I want to keep separate. However, sometimes I want to be able to combine them, briefly, into a composite build. Sometimes it's nice if I can do that for a while without affecting other devs. So, I want something like this:
My main settings.gradle, which would be checked into version control, would look like this:
// normal stuff
if (File('extra-settings.gradle).exists()) {
// This is what I don't know how to do
includeOtherSettingsFile('extra-settings.gradle')
}
Then extra-settings.gradle, which is not checked into source control, might look like this:
includeBuild('../anxml') {
dependencySubstitution {
substitute module('com.analyticspot.ml:framework') with project(':framework')
}
}
This way I could add an extra-settings.gradle file to make a temporary composite build. Keep it that way for several commits without affecting other programmers or worrying that I'd accidentally commit my temporary changes to settings.gradle and then, when I'm done, I could just delete it.
I know about Prezi Pride and it seems great but won't work for our current build (we use buildSrc, rootDir, etc.)
Can it be done?
settings.gradle is executed against a Settings instance which has an apply(Map) method so I'm guessing you can do:
// use Settings.getRootDir() so that it doesn't matter which directory you are executing from
File extraSettings = new File(rootDir, 'extra-settings.gradle')
if (extraSettings.exists()) {
apply from: extraSettings
}
I am building a RESTful API using golang.
I am using gin-gonic framework and mgo for mongodb connection.
I am trying to build this modular... (I don't think I know exactly what I am talking about and trying to do. XD LOL)
Like all other software, my project has multiple collections and number of collections will increase in the future.
And I would like to separate each module as much as possible from other modules so that I could
incrementally add modules
reuse the code for future projects and simply add files to add feature (in this case, I probably need to make each module into a separate package)
Here is my current project structure.
main.go - contains main function
database.go - small function that opens connection with Db
users.go - users collection related file
users-controller.go - contains gin.HandlerFunc for users collection
contents.go - contents collection related file
contents-controller.go - contains gin.HandlerFunc for contents collection
Code
https://gist.github.com/letsdevus/5121381be7cb1065e62ae403f14cd562
users and contents modules are totally independent.
Removing contents*.go files from project will cleanly disable contents module in the project.
Adding some non-dependent modules is very easy as well. No code to touch but simply adding files is all I need to do.
I would give myself kudos for that.
But now I am trying to add a module that has to interact with other modules.
It's called 'activities' module.
This module has to interact with other modules to track of what activities are occurring.
For example, when insert happens on users collection, activities module needs to know it happened so that it can record the activity.
A simple way of implementing activities module is adding a line, activities.insert("NewUser", Id) after users.Insert() is called.
The problem of this approach is very obvious.
I have to do this for all... the collections.
And more importantly, when I remove activities files from the project, I need to comment out all the activities related lines from other modules.
I want you to hear my plan to tackle this problem and advise me if it's viable or if there is any other better way.
My current possible solution: event manager
eventmanager.go will be added in the project as one of the base files like main.go and database.go.
eventmanager.go
package main
type EventFunc func()
type EventManager struct {
events map[string][]EventFunc
}
func (i *EventManager) AddEvent(eventName string, fn EventFunc) {
i.events[eventName] = append(i.events[eventName], fn)
}
func (i *EventManager) RunEvents(eventName string) {
for _, fn := range i.events[eventName] {
fn()
}
}
var eventManager EventManager // Globally available
And in users.go Insert() function, I add
eventManger.RunEvent("usersPostInsert")
after dbcUsers.Insert(&user).
And in init() in activities.go file, I add
eventManager.AddEvent("usersPostInsert", newUserActivity)
In the end of activities.go
func newUserActivity() {
// Do something. For now just print a message
log.Print("New User is Inserted!")
}
This way when I take off activities module, no problem occurs.
I hope you get the idea of what I am trying to do.
I just really want to know if this is a more than an OK way of tackling the problem.
p.s. If you are about to down vote this question because it's more like a codereview, just leave a comment instead. (somehow down vote makes me really uncomfortable lol)
Problem
This is how the app behaves:
every folder corresponds to a dynamic "feature" in the app.
features are independent of each other. (Inside each folder, none of the files have dependencies on other folders. However, each folder may use the same common libraries like jquery.)
not all features are needed all the time
within each folder, the files are not necessarily dependent on each other (but it makes sense to bundle them together because that's how a "feature" works)
the number of features is not fixed ... so a manual process where you have to manually update various configurations/files each time you add/remove a feature is NOT ideal
Desired Solution
Given what I described, what seems to make sense is to create chunks for each feature/folder and then load each chunk asynchronously (on demand) as needed by the app.
The problem is that this doesn't seem easy with webpack. I'd like to be able to do this
require.ensure([], function() {
var implementation = require('features/*/'+feature);
//...
where each feature (folder) goes in one chunk.
Unfortunately, if I'm understanding require.context correctly, that's not possible.
An alternative?
A possible alternative is the following:
use a single config file that lists each feature (we actually already use this),
use the config file to auto-generate a module that requires each folder asynchronously. (Basically, the "module" is a factory that the app can use to load each feature.)
add an entry point to the auto-generated module (force the chunks to be built).
So to summarize, add a build step that generates a factory that asynchronously load features. The problem is that this seems unnecessarily complex and I'm wondering if there's an better way to implement the solution. Or am I on the wrong track? is there a completely different solution?
Code
Folder structure
features
sl.graded.equation
graded.equations.js
graded.equation.edit.js
graded.equation.support.js
sl.griddable
griddable.js
griddable.edit.js
//...
Config file
{
"sl.graded.equation": [ //basically lists the feature "entry points" (not every file)
"sl.graded.equation",
"sl.graded.equation.edit"
],
"sl.griddable": [
"sl.griddable",
"sl.griddable.edit"
]
//...
}
Auto-generated factory
module.exports = function(feature, callback) {
switch(feature) {
//we can get fancy and return a promise/stream
case 'sl.graded.equation':
require.ensure([], function() {
var a = require('sl.graded.equation');
var b = require('sl.graded.equation.edit');
callback(a,b);
}
//...
I use KendoUI library in my project, so it's already minified but incredibly big.
Is it possible to exclude it from being uglified when using grunt-usemin?
Thanks!
In your grunt configuration, use an explanation point to make an exclude. Place those at the end of your src array.
e.g., add to the end of the src array, add:
'!htdocs/js/kendo.all.min.js'
You'll have to modify your flow for js and use a custom post-processor, which basically consists on adding a flow property to your useminPrepare.options (follow the basic structure in usemin README file), but instead of just adding a step (e.g. 'uglify'), plug a custom post-processor:
name: 'uglify',
createConfig: function (context, block) {
...
}
To customize how it will handle files, copy the createConfig from the example file you find most useful (see files in grunt-usemin/lib/config/) and modify it as you need (i.e. excluding the file you want).
I used a custom post-processor to add ngAnnotate to the usemin flow for js, just changing name to ngAnnotate and copying the createConfig from uglify).