Trying to add UTF-8 compatibility to existing config files - utf-8

First question for me here, so please bear with me.
I'm working with some software that uses proprietary config files (think INI files). These files are loaded in on program startup and the data is used during runtime. The data can be changed during runtime, and the config files themselves are updated on program shutdown.
We have been adding new language support to our software, and have run into a problem. When we give data to these config files at runtime that contain non-ASCII characters, the config files don't recognize them correctly because they aren't UTF-8 compatible.
I've attempted to modify the creation of the FILE objects that get filled out on startup to use the ccs=UTF-8 mode, and I've also tried to do a _setmode(_fileno(fin), _O_U8TEXT); call after the FILE stream has been opened, but both attempts cause the following assertion:
File: fgets.c
Line: 75
Expression: ( (_Stream->_flag & _IOSTRG) ||
((_textmode_safe(_fileno(_Stream)) == _IOINFO_TM_ANSI) &&
!_tm_unicode_safe(_fileno(_Stream))))
What is even stranger is that this assertion seems to be stemming from the
_VALIDATE_STREAM_ANSI_SETRET(stream, EINVAL, retval, NULL);
line in fgets.c, which is surrounded by an #ifndef _UNICODE block, and I have _UNICODE defined for my project.
Any insight into resolving this issue so that I can have UTF-8 compatibility through _setmode or the ccs mode flag would be appreciated.
Thanks.

Related

Getting rid of empty cpp files for interfaces in Rhapsody

In my project, I want to get rid of tons of empty and pointless cpp files for interfaces in IBM Rational Rhapsody.
Setting CPP_CG:File:Generate to Specification yields only header file generation of a class, which is almost what I want. But, the makefile (gpj) still looks for the *Ifc.cpp file. Is there a straight way to exclude these cpp files from makefile?
There is an option CG::File::AddToMakefile which does only work for component files. I found some info that it was working before but with Rhapsody 8, it stopped working.
You should be able to force the suppression of the either header or implementation file of the interface using those properties. However!
Rhapsody expects to find the cpp file of the interface and suppressing it will cause problems with roundtrip function - Roundtrip doesn't just occur explicitly, it will also trigger implicitly by default when you save the project or change focus from code editor to model browser.
During this roundtrip, Rhapsody will try to "fix" the model by replacing the missing cpp file. This will be followed by roundtrip error messages. Disregarding the errors and continuing with roundtrip will probably cause duplicate elements and all sorts of mess.
In other words, what you're trying to do is not really supported and is a bad idea.

VS2010 not embedding files into assembly

For some reason in VS2010 I can't embed any file into an assembly. These files have their Build Action property set to Embedded Resource as they should, but when the assembly is executed it founds no resources. For example, the following test code:
string[] list = System.Reflection.Assembly.GetExecutingAssembly().GetManifestResourceNames();
string msg = (list.Length > 0) ? "Full" : "Empty";
MessageBox.Show(msg);
always shows "Empty" because GetManifestResourceNames() always returns an empty list.
This issue affects a project using nettiers in which is not convenient for that particular case to include the stored procedures into the DB, so they must be taken from the Procedures.xml file, which BTW is automatically set with Build Action= Embedded Resource after the classes generation, and then of course when a function tries to get any SQL script from it, the program fails.
I also created a couple of test projects where I tried to embed an xml and a gif, always with no success.
Update: The problem seems to be only for C#. I reproduced the test project in VB.Net and there the file does get embedded. I've noticed that in the output window, in the C# case, the compiler command line doesn't include the /resource option (which should be there followed with the name of the file to embed) whereas for the VB case it does appear.
Are there any global settings or configuration files anywhere for C# I should check? (The build options for the project have nothing strange there)
Does anyone have an idea of what may be happening and how to fix this?
Finally, I found the cause:
There was a wrong line inside the Microsoft.CSharp.targets file (located inside c:\WINDOWS\Microsoft.NET\Framework\v4.0.30319). There was a tag written as:
<EmbeddedResourceF Include="#(_Temporary)" />
instead of
<EmbeddedResource Include="#(_Temporary)" />
Writing it right restored the ability to embed files
How that F appeared and why that file got altered in first place, I dont know... Despite being a .NET programmer for several years I just became aware today of the ".targets" files after searching the web about this issue, as never have had the need to look about the MSBuild stuff.

Visual Studio encoding problems

I have problems with files encoding in Visual Studio 2008. While compiling I'm getting such errors:
When I'm trying to open file where particular error occures, encoding window appears:
By defualt auto-detect is set. When I change encoding option to UTF-8, everything works. If I open each problematic file in my project using UTF-8 encoding, project starts to compile. The problem is I have too many files and there is ridiculous to open each file and set encoding to UTF-8. Is there any way to make this in a quick way ?
My VS settings are:
I'm using Windows Server 2008 R2.
UPDATE:
For Hans Passant and Noah Richards. Thanks for interaction. I recently changed my operating system so everything is fresh. I've also downloaded fresh solution from source control.
In OS regional settings I've changed system locale to Polish(Poland):
In VS I've changed international settings to the same as windows:
The problem is still not solved.
When I open some .cs files using auto-detection for encoding, and then check Files -> Advanced Save Options..., some of this .cs files have codepage 1250:
but internally looks following:
It is wired, because when I check properties of such particular files in source control, they seems to have UTF-8 encoding set:
I don't understand this mismatch.
All other files have UTF-8 encoding:
and opens correctly. I have basically no idea what is going wrong because as far as I know my friend has the same options set as me, and the same project compiles for him correctly. But so far he happily hasn't encountered encoding issues.
That uppercase A with circumflex tells me that the file is UTF-8 (if you look with a hex editor you will probably see that the bytes are C2 A0). That is a non-breaking space in UTF-8.
Visual Studio does not detect the encoding because (most likely) there are not enough high-ASCII characters in the file to help with a reliable detection.
Also, there is no BOM (Byte Order Mark). That would help with the detection (this is the "signature" in the "UTF-8 with signature" description).
What you can do: add BOM to all the files that don't have one.
How to add? Make a file with a BOM only (empty file in Notepad, Save As, select UTF-8 as encoding). It will be 3 bytes long (EF BB BF).
You can copy that at the beginning of each file that is missing the BOM:
copy /b/v BOM.txt + YourFile.cs YourFile_Ok.cs
ren YourFile.cs YourFile_Org.cs
ren YourFile_Ok.cs YourFile.cs
Make sure there is a + between the name of the BOM file and the one of the original file.
Try it on one or two files, and if it works you can create some batch file to do that.
Or a small C# application (since you are a C# programmer), that can detect if the file already has a BOM or not, so that you don't add it twice. Of course, you can do this in almost anything, from Perl to PowerShell to C++ :-)
Once you've opened the files in UTF-8 mode, can you try changing the Advanced Save Options for the file and saving it (as UTF-8 with signature, if you think these files should be UTF-8)?
The auto-detect encoding detection is best-effort, so it's likely that something in the file is causing it to be detected as something other than UTF-8, such as having only ASCII characters in the first kilobyte of the file, or having a BOM that indicates the file is something other than UTF-8. Re-saving the file as UTF-8 with signature should (hopefully) correct that.
If it continues happening after that, let me know, and we can try to track down what is causing them to be created/saved like that in the first place.

Qt: QSettings on Windows with INI files and comments

I have an application I'm writing using Qt 4.5.2 on Windows. I'm storing some settings in an INI file and using QSettings to load and save the settings. I'd like to have some comments in the INI file.
For example:
; Meta-info to store with the file
[General]
MainWindow\size=#Size(1280 600)
MainWindow\pos=#Point(0 300)
Debugging=true
However, I've found when I load the settings file with
QSettings settings( "settings.ini", QSettings::IniFormat );
the comments are stripped out of the file. The INI file is re-written after loading by a call to QSettings::sync() (this is done automatically by the constructor). Is there a way to preserve the comments after syncing?
Preemptive comments:
I want INI files in Windows for future cross-platform compatibility
I want to store meta-info in the file for reference outside of the application
I am considering making the meta-info a section of the INI and using the name=value rules but would prefer to keep the information as a comment
QSettings has no concept of "save". All the changes you do to it is considered to be final, and written to disk often and transparently.
In the documentation of QSettings, there is no mention about comments in ini files. It does makes some sense: after all, it can be a registry value, too. Treat it like a generated file: it is one.
Here's my suggestion:
[General]
Comment = #String(Meta-info to store with the file)
MainWindow\size=#Size(1280 600)
MainWindow\pos=#Point(0 300)
Debugging=true
I don't know if it works, play around with it to see how it actually stores the string. Oh, and make sure you either set it from code or document it properly, to avoid accidentally using the same identifier from within the program.

Embedding an application manifest into a VB6 exe

I have recently gone through a bunch of standalone utility apps written in VB6 to make sure that registry virtualization is turned off for Windows Vista and above. I created a standalone manifest file for each exe, set the requestedExecutionLevel appropriately (some of them need to modify HKEY_LOCAL_MACHINE registry keys, others do not), and tested them. They all appear to work correctly.
I have only one small problem remaining. Since they are standalone utilities, people are used to just copying them around the network and running them manually. If anyone forgets to copy the manifest file as well as the exe, then the exe will silently write to the virtualized registry key instead of the real one and cause hard-to-debug problems.
The obvious solution is to embed the manifest into the exe as a resource. All the articles I have read on the net tell you to embed the resource like this:
#define CREATEPROCESS_MANIFEST_RESOURCE_ID 1
#define RT_MANIFEST 24
CREATEPROCESS_MANIFEST_RESOURCE_ID RT_MANIFEST "app.manifest"
This should work just fine, except that the VB compiler always creates the application icon with resource ID = 1. When I tried the above code, Windows refused to run the exe, complaining about a resource error (I'll update this post with the details later). I tried changing the resource ID to another number, after which Windows ran the program successfully but did not recognise the manifest contents.
Does anyone know of a way to get an embedded manifest to work in a VB6 exe, or should I just stick with an external file?
UPDATE 1
The text given above is the whole content of the .rc file. I compile it to a .res file like this:
"%ProgramFiles%\Microsoft Visual Studio\VB98\Wizards\rc.exe" /r /fo "Resources.res" "Resources.rc"
And embed it in the VB6 project file like this:
Type=Exe
Reference=*\G{00020430-0000-0000-C000-000000000046}#2.0#0#..\..\..\..\..\..\..\..\WINDOWS\system32\stdole2.tlb#OLE Automation
Form=Main.frm
ResFile32="Resources.res"
IconForm="FMain"
Startup="FMain"
HelpFile=""
Title="Windows Vista Registry Test - VB6"
ExeName32="RegistryTestVB6.exe"
Path32=""
Command32=""
Name="RegistryTestVB6"
HelpContextID="0"
CompatibleMode="0"
MajorVer=1
MinorVer=0
RevisionVer=0
AutoIncrementVer=0
ServerSupportFiles=0
VersionComments="Windows Vista Registry Test - VB6"
VersionCompanyName=""
VersionFileDescription="Windows Vista Registry Test - VB6"
VersionLegalCopyright=""
VersionProductName="Windows Vista Registry Test - VB6"
CondComp=""
CompilationType=0
OptimizationType=0
FavorPentiumPro(tm)=0
CodeViewDebugInfo=0
NoAliasing=0
BoundsCheck=0
OverflowCheck=0
FlPointCheck=0
FDIVCheck=0
UnroundedFP=0
StartMode=0
Unattended=0
Retained=0
ThreadPerObject=0
MaxNumberOfThreads=1
When I read the compiled exe into the VS2008 resource editor, it looks like this:
RegistryTestVB6.exe
Icon
1 [Neutral]
RT_MANIFEST
1 [English (United States)]
Version
1 [English (United States)]
When I construct an exact equivalent VB.NET test app in VS2008, then load that into the resource editor, it looks like this instead:
RegistryTestNET.exe
Icon
32512 [Neutral]
RT_MANIFEST
1 [Neutral]
Version
1 [Neutral]
UPDATE 2
Testing - the .NET exe runs fine on both Windows XP and Windows 7. However, the VB6 exe produces the following error on XP:
This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem.
and the following error on 7:
The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log or use the command-line sxstrace.exe tool for more detail.
Looking in the event log I see the following entry:
Activation context generation failed for "RegistryTestVB6.exe". Error in manifest or policy file "RegistryTestVB6.exe" on line 10. Invalid Xml syntax.
Needless to say the XML isn't invalid, it's exactly the same file with the same encoding that I used for the .NET exe, and that one works.
RESOLUTION
The VB6 compiler does indeed require that an arbitrary text file included in a resource must be an exact multiple of 4 bytes. I simply added spaces to the XML until Notepad++ told me that the total file size including BOM was a multiple of 4.
Thanks to both Michael and Jim for pointing me in the correct direction. Just a pity I can't mark you both as the answer!
Interestingly enough, I had to do the exact same thing recently. Following the steps Christian described, I got it to work the first time through. For prosperity, here is the entire workflow I followed:
Created a RC file as described in the orginal question
Created a app.manifest, preserving whitespace characters, which are VERY IMPORTANT for this to work. As stated in previous answers, the file size must be a multiple of 4.
Ran RC.EXE as described in the original question against the rc to generate the .res file
Edited my Project.VBP file to include the following line near the top:
ResFile32="Resources.res"
Built EXE in standard vb6 environment. When deployed on a vista or win7 machine, the shield shows up and the user is prompted to run as administrator. When opening the EXE file in studio, I verified the resources.
Notepad++ tells me that the encoding on my app.manifest file is ANSI. It did not include an byte order mark at the start of the file.
If you are still having troubles, let me know and I'll share whatever I can with you. Other than that, I'm not sure what to tell you other than Works on my Machine!
VB6 has a quirk in that any resource element must be an exact multiple of 4 in length. Try padding the manifest file out with spaces to ensure this, and see if that changes the behavior.
This quirk was documented in Microsoft article Q297112 (archive).
Also, you might add the resource using the VB6 IDE instead of editing the VBP. The effect may be the same, but the resource editor is the standard means for doing this.
Using the Resource Compiler (rc.exe) is going the long way around. There is a much simpler option for embedding an application manifest within an executable, whether C++ or VB6 or just about any other language. The Manifest Tool (mt.exe) was written specifically to embed manifests within binaries and is provided free of charge with the Windows SDK. The added benefit of using mt.exe is that it automatically handles any necessary padding.
Simply run the following command line after the binary has been compiled. I have used the naming convention used internally by the Visual C++ 2005 compiler, where the manifest filename contains the full program name with ".intermediate.manifest" appended.
mt.exe -nologo -manifest "program.exe.intermediate.manifest" -outputresource:"program.exe;#1
Update: I have personally been using this in an automated build process with VB6 executables for over two years now. It has been so successful that we have eliminated OS compatibility tests - specific to manifests - from our regression testing.

Resources