code reuse for file existence test in a Visual Studio vcxproj file - visual-studio

I have a vcxproj file that contains explicit Windows shell commands in the NMakeBuildCommandLine section:
<NMakeBuildCommandLine Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
move file1 file2
</NMakeBuildCommandLine>
I'm using MSBuild to execute the vcxproj, either directly or via a sln file. The problem is that when file1 does not exist, the output is very unhelpful and doesn't even list the file's name:
The system cannot find the file specified.
My naive solution is to replace move file1 file2 with:
if exist file1 (move file1 file2) else (echo file1 does not exist && exit 1)
(Note that I need to write && instead of &&.)
This works, but it's error-prone because I need to type file1 three times per line and make sure they all match. file1 is only one of many files that need to be moved. Furthermore, the series of commands is virtually identical across the various build configurations.
How can I avoid repeating myself both within a command line and across build configurations? I thought that the UserMacros property group might help me, but I couldn't figure out how to write such a macro via the IDE. (Right-clicking on the project in Visual Studio doesn't show a field for entering user macros.) Nor could I find any discussion of the syntax of this section on the Internet, so I didn't know how to write macros with a text editor (which I would actually prefer).
There may be yet a better way within a vcxproj file to identify files that need to exist, so I'm open to any alternatives.

MsBuild has 'variables' like most other programming languages called properties. You declare one in a PropertyGroup element in the project file xml and then reuse it using the $(PropertyName) syntax. Example for your case:
<PropertyGroup>
<Src>/path/to/my/src</Src>
<Dst>/path/to/my/dst</Dst>
</PropertyGroup>
<NMakeBuildCommandLine>
if exist $(Src) (move $(Src) $(Dst)) else (echo $(Src) does not exist && exit 1)
</NMakeBuildCommandLine>
If you want to use the IDE, which might get tedious if you have lots of values, you can indeed use so-called UserMacros but you have to declare those in a proprty sheet. Go to View->Property Manager, right-click on your project and select 'Add new Property Sheet'. Doubleclick on it, go to 'User Macros' and add key/value pairs there. If you save everything and look in the generated files you'll see the vcxproj now Imports the propertysheet, and the propertysheet itself has a PropertyGroup just like shown above - but editable thgough the IDE.
As an alternative which might be better (less duplication, easier to automate) in the long run you can use MsBuild code for checking file existence and moving files which has the benefit you only have to write the move command once as you can have MsBuild loop over items. Those are declared in an ItemGroup. Explaining everything here is a bit out of scope but an example should make things clear:
<Target Name="BatchMove">
<ItemGroup>
<SrcFiles Include="file1">
<Dest>file2</Dest>
</SrcFiles>
<SrcFiles Include="file3">
<Dest>file4</Dest>
</SrcFiles>
</ItemGroup>
<Warning Text="Source file %(SrcFiles.Identity) does not exist" Condition="!Exists(%(SrcFiles.Identity))" />
<Move SourceFiles="%(SrcFiles.Identity)" DestinationFiles="%(SrcFiles.Dest)" Condition="Exists(%(SrcFiles.Identity))" />
</Target>
This declares 2 source files file1/file3 and their respective destination files file2/file4. If the source does not exists (using standard MsBuild Exists check) a message is shown, else it is moved to the destination. Those % characters will make the line they occur in loop over each element of the SrcFiles collection. To add more files, just add to the ItemGroup. Last step is to get this target invoked from the nmake command line which is done simply by calling msbuild on the file itself and telling it to run the target:
<NMakeBuildCommandLine>
msbuild $(MSBuildThisFile) /t:BatchMove
</NMakeBuildCommandLine>

Related

How to bootstrap generated files task output with msbuild

I have a .csproj file where I declare some generated files:
<ItemGroup>
<AutoGenerated Include="generated\*.cs;generated\Models\*.cs"/>
</ItemGroup>
This is used as Outputs on a task:
<Target Name="GenerateFilesFromTemplate" BeforeTargets="BeforeBuild;BeforeRebuild"
Inputs="sometemplatefile"
Outputs="#(AutoGenerated)">
<Exec
Command="somegenerator.exe ... "
Outputs="#(AutoGenerated)"/>
</Target>
The generated/ is added .gitignore, so the generated files are not part of the source control.
My problem is that this setup does not bootstrap itself. As initially the generated folder is empty, the build skips the generate files task because it has no outputs:
GenerateFilesFromTemplate:
Skipping target "GenerateFilesFromTemplate" because it has no outputs.
I feel I'm doing this the wrong way, or I'm missing the obvious. I know I can remove the Inputs and Outputs on the Task and then it will generate the files on every build. But the generation is lengthy, and I want to avoid it if no necessary (the template file did not change). How can I make the build self-bootstrap and generate the files on first build, or when necessary?
The output items are meant to indicate known output files of a target. MSBuild then checks if all modification dates of the outputs are greater than the highest modification date of the input items to see if it can skip the target. It will also run the target if one of the outputs are missing.
If you cannot know in advance which files are generated by a target, a workaround is to produce a temporary file with a known location.
e.g. write use the WriteLinesToFile task (configured to overwrite) to update a file ($(IntermediateOutputPath)autogen.marker) and use the file as output element. It will then run the target on the first build and then only run if the input template file is newer than the marker.

Can Visual Studio read a set of include file paths from a text file for the Additional Include Directories?

I'm trying to figure out how to get Visual Studio to read a set of include files from a text file.
For example, I would like to create a text file called IncludePaths.txt that contains a list of include paths such as "/I ../../header"
I would then tell Visual Studio reference this file.
I believe you could do this by adding #IncludePaths.txt to the Additional Include Directory, but I cannot get this to work. I have seen this done in projects I have worked on in the past but I can't find any documentation or figure out the trick.
After a little more research and talking to a couple of other developers, I figured out the "trick"
1) Create a file called IncludePaths.txt next to my project file.
2) Add your include paths to this file...
/I "..\..\..\..\open\common\include"
/I "..\..\..\common\include"
/I "..\..\"
3) Go to Properties -> C++ -> Command Line
4) Under "Additional Options" add #IncludePaths.txt
Alternatively, you can use custom properties to get this to work too.

Adding new lang to ctags does not work

I am trying to add .volt extension to ctags language map, but it keep ignoring .volt file. This is content of my .ctags file:
--recurse=yes
--tag-relative=yes
--exclude=*.git*
--exclude=.DS_Store
--langmap=html:+.volt
When I do ctags --list-maps I will see .volt files being included in HTML:
HTML *.htm *.html *.volt
But still when I run ctags, it completely ignores .volt files. What I am doing wrong here?
The reasons for the unexpected behavior are most likely:
You are not using currently latest version 5.8 of Exuberant Ctags, but a version before 5.6.
Your .ctags file has --langmap=html:+.volt at end of file with no line termination.
Read the full story below on why I think those 2 reasons result in the unexpected behavior of Ctags on your computer.
I looked on your problem on Windows first using older version 5.5.4 of Exuberant Ctags installed with text editor UltraEdit and later also with version 5.8 downloaded directly from Exuberant Ctags project page.
I created a copy of one of my HTML projects with just 1 *.html file in parent directory of the test project, 3 *.html files in a subdirectory and two more *.html files also in the subdirectory with file extension changed from html to volt on both files which were just copies of 2 of the 3 *.html files in this subdirectory.
Next I created in parent directory of the project a ctags.conf file and copied the few lines you posted into this file. Additionally I inserted at top a line with --verbose as this is useful on looking for problems like that.
And last I copied ctags.exe (first v5.5.4, later v5.8) also into the test project directory just for making it easier to run it from command line.
I opened a command prompt window in test project directory and executed
ctags.exe -f test.tag --options=ctags.conf
I could see on verbose output that both *.volt files were opened for processing and created test.tag contained also all the tags from the 2 *.volt files, the same tags as the 2 *.html files from which the *.volt files were copied before.
So what could be the problem?
I'm not only familiar with HTML. My main job is programming in C/C++. Therefore I know about an often made mistake in C source code files on reading in text files: a wrong handling of text files with no line termination on last line of the file.
And I know that some text editors like gedit on Linux position the caret on Ctrl+End at beginning of the line below the last line in the file even when last line of the file does not have a line termination. The caret should be in this case positioned by the text editor at end of the string on last line instead of beginning on next line beyond real end of the file. This in my point of view wrong behavior lets a user of the text editor think that the text file has a line termination also on last line of the file even if this is not true.
So I thought that you have appended --langmap=html:+.volt perhaps at end of the file without a line termination and ctags.exe does not evaluate the line in this case because of not well done text file parsing in source code. Therefore I removed the line termination in ctags.conf from last line containing now only --langmap=html:+.volt
I executed same command line as before and AHA, both *.volt files are ignored because of unknown language.
This was the time as I downloaded version 5.8 of Ctags for Windows and copied it into the test project directory replacing executable of version 5.5.4.
I executed the command line again with not modified ctags.conf. Both *.volt files were processed by Ctags and test.tag contained again the tags from both *.volt files.
Appending on last line of file ctags.conf again a line termination and executing the command line once more did not result in a different output. So this bug with ignoring last line of the options file if no line termination present at end of the file is fixed in version 5.8 of Ctags.
I searched in Change Notes of Exuberant Ctags for last and found in changes notes block for ctags-5.6 (Mon May 29 2006)
Fixed problem reading last line of list file (-L) without final newline.
This is the confirmation for what I thought and could see. And of course the problem existed not only on reading the list file, but also on reading other text files like the options file, or C and Java files as the next line in the change notes informs
Fixed infinite loop that could occur on files without final newline [C, Java].
If the ctags binary is really universal ctags you need to put/link your config file here (man ctags-universal -> FILES):
~/.ctags.d/my-config.ctags
File extension .ctags is relevant.
In my case, I needed ctags to support the arduino (.ino) file type. Add --langmap=c++:+.ino to ~/.ctags.d/local.ctags (it only symlinks to ~/.ctags really).
Check:
ctags --list-maps | grep C++
C++ *.c++ *.cc *.cp *.cpp *.cxx *.h *.h++ *.hh *.hp *.hpp *.hxx *.inl *.C *.H *.CPP *.CXX *.ino
[...]
Notice *.ino at the end of the line listing known extensions.

Creating empty directories / folders in InstallAnywhere 2011

I have a script which collected together a number of files to be installed. This includes a number of empty directories.
Previously I would use the D flag in the manifest file which would copy empty directories. However due to the way I generate the manifest files (as part of our build process) I can sometimes end up with two D entries with the same destination folder. e.g:
D;${A_LIB}/all/pysys/${PYSYS_VERSION}/lib/python2.7/site-packages;./third_party/python/lib/python2.7/site-packages;COMMON;${UNIX}
D;${A_LIB_BT}/python/${PYTHON_VERSION};./third_party/python;COMMON;${ALL}
This causes InstallAnywhere to fail to build the installer.
To get around this I rewrote the manifest generation code to parse the directories previously pointed to by a D and replace the D entry with F entries for each file in the directory.
Unfortunately this will not include empty directories (which we may / may not need in the installer but in general it's just safer to create them than have some piece of code fail because they're not there).
I've tried the following in the manifest. Reference, Reference3 and Reference4 are empty, Reference2 contains a single directory (which is itself empty). Only Reference2 is present in the install - the other three which are empty directories seem to get excluded.
D,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference,./samples/pysys/cor_002/Reference
D,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference2,./samples/pysys/cor_002/Reference2
D,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference3/.,./samples/pysys/cor_002/Reference3/.
D,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference4/../Reference4,./samples/pysys/cor_002/Reference4/../Reference4
I've also tried increasing the log level but this has not revealed anything. Is there a way to increase this log level?
export LAX_DEBUG=true
Any suggestions?
DISCLAIMER: I've cross posted this to InstallAnywhere's forums but I will do my best to keep the answers in sync and spread the knowledge.
I can't speak to your manifest challenges. However, my first thought is to change the manifest generator to be sensitive to duplicate output locations -- maybe by storing them in a Map or Set -- and then handling collisions when they occur by failing the build or adjusting the output location(s).
On the other hand, I can tell you how to increase the verbosity of your installer.
Make the installer more verbose by adding:
-Dlax.debug.all=true -Dlax.debug.level=3
to Project > JVM Settings > Installer Settings (tab) > Optional Installer Arguments > Additional Arguments. You'll want to remove these before you ship. You can also add these to the command line when you start the installer. Level values of 4 and 5 work, too, and are even more verbose.
You can also make your installer print its progress to the console by going to Project > JVM Settings > Log Settings. Here, uncheck Include debug output (stderr and stdout). Then enter the word console in Send stderr to: and Send stdout to:. Rather than console, you can also set a specific file name. You'll also want to undo these settings before you ship.
The solution turns out to be so blindingly simple that I never tried it.
To get EMPTY directories installed by Install Anywhere you have to specify the directories as files in the manifest. So with the following directory structure:
Reference <empty>
Reference2
testdir <empty>
Reference3 <empty>
Reference4 <empty>
You need to specify the entries in the manifest as F. Specifying then as D will only result in the "Reference2" directory being included.
F,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference,./samples/pysys/cor_002/Reference
F,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference2,./samples/pysys/cor_002/Reference2
F,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference3/.,./samples/pysys/cor_002/Reference3/.
F,$IA_PROJECT_DIR$/samples/pysys/cor_002/Reference4/../Reference4,./samples/pysys/cor_002/Reference4/../Reference4
Sorry to answer my own question, really wasn't the plan!

Using CMake, how can I concat files and install them

I'm new to CMake and I have a problem that I can not figure out a solution to. I'm using CMake to compile a project with a bunch of optional sub-dirs and it builds shared library files as expected. That part seems to be working fine. Each of these sub-dirs contains a sql file. I need to concat all the selected sql files to one sql header file and install the result. So one file like:
sql_header.sql
sub_dir_A.sql
sub_dir_C.sql
sub_dir_D.sql
If I did this directly in a make file I might do something like the following only smarter to deal with only the selected sub-dirs:
cat sql_header.sql > "${INSTALL_PATH}/somefile.sql"
cat sub_dir_A.sql >> "${INSTALL_PATH}/somefile.sql"
cat sub_dir_C.sql >> "${INSTALL_PATH}/somefile.sql"
cat sub_dir_D.sql >> "${INSTALL_PATH}/somefile.sql"
I have sort of figured out pieces of this, like I can use:
LIST(APPEND PACKAGE_SQL_FILES "some_file.sql")
which I assume I can place in each of the sub-dirs CMakeLists.txt files to collect the file names. And I can create a macro like:
CAT(IN "${PACKAGE_SQL_FILES}" OUT "${INSTALL_PATH}/somefile.sql")
But I am lost between when the CMake initially runs and when it runs from the make install. Maybe there is a better way to do this. I need this to work on both Windows and Linux.
I would be happy with some hints to point me in the right direction.
You can create the concatenated file mainly using CMake's file and function commands.
First, create a cat function:
function(cat IN_FILE OUT_FILE)
file(READ ${IN_FILE} CONTENTS)
file(APPEND ${OUT_FILE} "${CONTENTS}")
endfunction()
Assuming you have the list of input files in the variable PACKAGE_SQL_FILES, you can use the function like this:
# Prepare a temporary file to "cat" to:
file(WRITE somefile.sql.in "")
# Call the "cat" function for each input file
foreach(PACKAGE_SQL_FILE ${PACKAGE_SQL_FILES})
cat(${PACKAGE_SQL_FILE} somefile.sql.in)
endforeach()
# Copy the temporary file to the final location
configure_file(somefile.sql.in somefile.sql COPYONLY)
The reason for writing to a temporary is so the real target file only gets updated if its content has changed. See this answer for why this is a good thing.
You should note that if you're including the subdirectories via the add_subdirectory command, the subdirs all have their own scope as far as CMake variables are concerned. In the subdirs, using list will only affect variables in the scope of that subdir.
If you want to create a list available in the parent scope, you'll need to use set(... PARENT_SCOPE), e.g.
set(PACKAGE_SQL_FILES
${PACKAGE_SQL_FILES}
${CMAKE_CURRENT_SOURCE_DIR}/some_file.sql
PARENT_SCOPE)
All this so far has simply created the concatenated file in the root of your build tree. To install it, you probably want to use the install(FILES ...) command:
install(FILES ${CMAKE_BINARY_DIR}/somefile.sql
DESTINATION ${INSTALL_PATH})
So, whenever CMake runs (either because you manually invoke it or because it detects changes when you do "make"), it will update the concatenated file in the build tree. Only once you run "make install" will the file finally be copied from the build root to the install location.
As of CMake 3.18, the CMake command line tool can concatenate files using cat. So, assuming a variable PACKAGE_SQL_FILES containing the list of files, you can run the cat command using execute_process:
# Concatenate the sql files into a variable 'FINAL_FILE'.
execute_process(COMMAND ${CMAKE_COMMAND} -E cat ${PACKAGE_SQL_FILES}
OUTPUT_VARIABLE FINAL_FILE
WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR}
)
# Write out the concatenated contents to 'final.sql.in'.
file(WRITE final.sql.in ${FINAL_FILE})
The rest of the solution is similar to Fraser's response. You can use configure_file so the resultant file is only updated when necessary.
configure_file(final.sql.in final.sql COPYONLY)
You can still use install in the same way to install the file:
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/final.sql
DESTINATION ${INSTALL_PATH})

Resources