I'm seeing some weird behavior when running a test install of my Qt program (tried using qt 5.5.1 and 7.0). I haven't noticed the issue when running in a debug/development environment - but see the issue when installed into "Program Files (x86)".
The issue is: I'm using QDirIterator to find database files within the "QStandardPaths::DataLocation" locations and loading them in via sqlite. The phantom files are located in Program Files (x86)//Library/.ndat What I'm seeing is that files from a previous install (which have been deleted) and ones that have been renamed, then deleted, still show up and are readable in the program. These "phantom" files have been blocking loading of the up-to-date file. It's really strange - I wonder if anyone has seen the issue?
I'm running Windows 10 Home on an SSD-based machine (if it matters). Same issue with Qt 5.5.1 and 5.7. I've replicated it on a different machine with similar configuration.
Any ideas?
Here's a summary of my code:
QStringList standardPaths = QStandardPaths::locateAll(QStandardPaths::DataLocation, "Library", QStandardPaths::LocateDirectory);
QStringList fileFilters;
fileFilters << "*.ndat";
foreach (const QString &dir, standardPaths) {
QDirIterator iterator (dir, fileFilters);
while (iterator.hasNext()) {
const QString &filePath = iterator.next();
QString databaseName = QFileInfo(filePath).baseName();
database_->open(filePath, baseName); // my function
}
}
boolDataManager::open (const QString &filePath, const QString &connectionName) {
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE", connectionName);
db.setDatabaseName (filePath);
if (!db.open()) {
ERROR(QString("Cannot open database %1 with error %2")
.arg(QFileInfo(filePath).baseName())
.arg(db.lastError().text()));
printError();
return false;
}
databaseNames_.append(connectionName);
return true;
}
This code seems to read in files that don't exist anymore - and strangely, reads contents of old files that have been overwritten in the same spot. It only seems to happen when the files are located within the "Program Files" directory; not in a user directory or what-not.
For example, version 1 of my code had a database called "database.dat" with 10 entries. Version 2 of my install overwrote the file with a file of the same name with 20 entries. Version 2 of my code finds the database.dat file but only reads in the older version with 10 entries - Really weird!
Update
It appears that these "phantom" files are stored at:
C:\Users/USERNAME/AppData/Local/VirtualStore/Program Files (x86)/PROGRAM NAME/database.dat
My guess is that I'm opening the file in my program not as read-only so Windows creates a working copy in a user-writable location. Will investigate.
The issue is Windows caching - I think - One cant really tell with software that doesn't provide any way to debug it - such as Windows.
I've heard that this solution can also be solved (or at least decreased) by turning on the "Application Experience" Service -- I still run into it from time to time, typically when doing too many Filesystem writes in too short of a time.
I dont know exactly what the cause is -- and I'm pretty sure nobody else does or it would have been fixed.. but as far as I know there is no fix for that (as of this answer's date)
--
Here's my solution to problems like this that works 100% of the time:
To avoid this problem, append the version number to the end of your database's filename each time you compile, in fact apppend it to all your files by using a
#define VERSION 4.22.21
and then just adding .append(QString("%1").arg(VERSION)); or something.
All you have to really do then is write up some quick code to import all the necessary data from an old database or from wherever, which you should have more or less anyways just from using the database.
Better to avoid situations like that than to try and figure them out -- not to mention you now have a perfect revision system without even trying.
UPDATE
Since theres no use case, no code, and no information about the project, I would have to guess at what you were trying to do -
QList<DataDir*> dataDirectories;
DataDir* new_dataDir;
QStringList standardPaths = QStandardPaths::locateAll(QStandardPaths::DataLocation, "Library", QStandardPaths::LocateDirectory);
QStringList fileFilters;
fileFilters << "*.ndat";
foreach (const QString &dir, standardPaths) {
QDirIterator iterator (dir, fileFilters);
while (iterator.hasNext()) {
const QString &filePath = iterator.next();
QString databaseName = QFileInfo(filePath).baseName();
database_->open(filePath, baseName); // my function
/* Do your database reading or writing and save the results
* into a QHash or something then do this: */
database_->close(); // super important
}
}
After a bunch of poking around, I found the source of the problem (and solution).
Windows (for backwards compatibility) has a VirtualStore function where if the program tries to write to a unwritable file (based on permissions, e.g. Program Files/Progname/test.txt), it'll copy that file into USER/AppData/Local/VirtualStore/Program Files/.... This new file is not deleted when the program is uninstalled, but looks to the QT program as residing at its original location.
The solution is to open the Sqlite database in read only mode:
QSqlDatabase db = QSqlDatabase::addDatabase("QSQLITE", connectionName);
if (!writable_)
db.setConnectOptions(QLatin1String("QSQLITE_OPEN_READONLY"));
db.setDatabaseName (filePath);
Now, I'm running into a problem determining whether the file is writable. This:
writable_ = fInfo.isWritable();
always returns true, even for files in Program Files. Even when enabling NTFS permissions checking:
extern Q_CORE_EXPORT int qt_ntfs_permission_lookup;
qt_ntfs_permission_lookup++; // turn permisssions checking on
the permissions check doesn't work. So now I'm simply doing this:
QString appDir = gApp->applicationDirPath();
QString relFilepath = QDir(appDir).relativeFilePath(filePath);
if (!relFilepath.startsWith(".."))
writable_ = false;
Database is read only (OK for my application) and no longer creates anything within VirtualStore
Related
I have QString variable named "src", which holds a filename. Operation QFile::copy(src, target) works Ok, until target is "C:" or "C:/" (I have the problem in Windows 10). In this case the operation returns true, even though I do not see any files actually copied to C:/ (in fact, normally I cannot copy anything to C:/ without administrator rights). Moreover, when I debug, I see that it says it has copied to C:// (two slashes). Is it a Qt bug or do I miss something?
UPD: Copying, e.g., to C:/Users which also requires administrator rights fails, as it should (returns false). Qt version is 5.7.
UPD:
QString src = "C:/Stuff/somefile.pdf";
QString target = "C:/somefile.pdf";
if (QFile::copy(src, target)) qDebug() << "Copy successful";
else qDebug() << "Copy failed";
This code yields "Copy succeful", whereas neither do I have write access to C:/ nor actually file somefile.pdf appears there.
When your application does not have permissions for writing to some directories, the files are stored into the Windows Virtual Store located at C:\Users\%USERNAME%\AppData\Local\VirtualStore. So in fact your file is successfully saved (albeit the path is redirected), that is why QFile::copy() returns true.
This redirection is called UAC Virtualization, and it works for C:, C:\Program Files, C:\Windows and HKLM\Software.
I guess the most genuine way to avoid such problems is to stick to saving configuration data in the designated locations such as application data (which in the context of the Qt framework will also meet the terms of cross-platform code).
Processing 3.0 launch function doesn't launch my .exe.
I am using the Launch() function (https://processing.org/reference/launch_.html)
launch("C:/Program Files (x86)/Google/Chrome/Application/chrome.exe");
Or
launch("C:/app/keyboard.exe");
Result: Chrome browser will open. keyboard.exe will not. I've tryed different locations and relative paths.
I only get a windows loader when the link is correct. So that is correct.
The function discriptions says this:
"Be sure to make the file executable before attempting to open it (chmod +x). "
https://superuser.com/questions/106181/equivalent-of-chmod-to-change-file-permissions-in-windows
I also made a .bat file to execute the .exe but the launch() function only works on exe files.
but that didnt work either.
System:
Processing 3.0
Java 8
Windows 10, 64 bit
So what am I missing?
It is a bit dodgy but works in windows 8:
PrintWriter output=null;
output = createWriter("myfile.bat");
output.println("cd "+sketchPath(""));
output.println("start archivo.exe");
output.flush();
output.close();
output=null;
launch(sketchPath("")+"myfile.bat");
And you can choose another relative or absolute path
for instance
output.println("cd ..");
output.println("cd directoriy");
...
As Samuil advises, Windows uses \ instead of a / as a separator character, which you'll need to escape, hence \\: launch("C:\\app\\keyboard.exe");
I recommend using File.separator:
launch("C:"+File.separator+"app"+File.separator+"keyboard.exe");
It's a bit longer, but will work regardless of the operating system(Linux/OSX/Windows/etc.).
Aside launch(), also try exec():
exec(new String[]{"start","C:"+File.separator+"app"+File.separator+"keyboard.exe");
also Process. (If you need to check the output, you may need to write your own thread that will pipe the output)
I have an algorithm in C++ (main.cpp) and I use CLion to compile and run it. Algorithm would read strings from text file, but there is a mistake:
Could not open data.txt (file exists and placed in one folder with main.cpp)
How can I fix it and make this file "visible" to CLion?
If you are using fopen or something similar and just passing "data.txt", it is assumed that that file is in the current working directory of the running program (the one you just compiled).
So, either
Give a full path instead, like fopen("/full/path/to/data.txt"), where you use the actual full path
(not preferable), Move data.txt to the directory where CLion runs its compiled programs from.
(for #2, here's a hacky way to get that directory)
char buf[1024]; // hack, but fine for this
printf("%s\n", getcwd(buf, 1024));
Run/Edit configurations...
Select your application (on the lefthandside of the window)
Specify Working directory
Apply
Now you can fopen relatively from working directory.
I found another way to solve this problem.
#Lou Franco's solution may affect the project structure. For example, if I deploy code on a server, I should move the resource file to specific directory.
What I do is modify the CmakeLists.txt, on Windows, using
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY "D:\\science\\code\\English-Prediction")
CMAKE_RUNTIME_OUTPUT_DIRECTORY is a CMake variable, it assigns the work directory of CLion work directory.
Continuing with the CMAKE_RUNTIME_OUTPUT_DIRECTORY CMakeLists variables, I do the following. In the root directory of my project, I create a directory, e.g., out. Then, in my CMakeLists.txt I set the CMAKE_RUNTIME_OUTPUT_DIRECTORY to that directory:
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR}/out)
Note, that must come before you have
add_executable(YourProject ${SOURCE_FILES})
I might also add that instead of using fopen() I would keep it more object-oriented by using std::ifstream:
std::ifstream inFile("data.txt");
// check if it opened without issue...
if (!inFile) {
processError(); // a user-defined function to deal with the issue
} else {
// All is good, carry on...
// and when you're done don't forget
inFile.close();
}
I have a web application that reads files from its local directory (in wamp/www/). This file needed to be accessed by several users so I synced and shared it using Dropbox. Now, is there a shortcut I can use in php commands such as fwrite such that the code is not strictly applicable to only one computer?
For example, I can't code it to fwrite("C:\Users\name\My Documents\") because that is pretty specific to one user and long. I was wondering if there was a shorthand I can use, like %appdata% or %programfiles%?
Try using
$_SERVER['HOMEDRIVE'] and $_SERVER['HOMEPATH']
For drive and path to user folder respectively
print_r($_SERVER)
Will display all the environment variables. There you can see which one to select.
$fp = fopen("{$_ENV['USERPROFILE']}\My Documents\somefile.txt", 'wb');
See $_ENV on the manual and also getenv().
Please note this will only work in limited circumstances. You can use this internal function instead:
#include<Shlobj.h>
PHP_FUNCTION(win_get_desktop_folder)
{
char szPath[MAX_PATH];
if (zend_parse_parameters_none() == FAILURE)
RETURN_NULL();
if (SUCCEEDED(SHGetSpecialFolderPathA(NULL, szPath,
CSIDL_MYDOCUMENTS, FALSE))) {
RETURN_STRING(szPath, 1);
} else {
RETURN_FALSE;
}
}
When the code below is ran, the watch is only triggered if I edit and save tmp.txt manually, using either my ide, TextEditor.app, or vim.
It doesn't by method of the write stream or manual shell output redirection (typing echo "test" > /path/to/tmp.txt").
Although if I watch the file itself, and not its dirname, then it works.
var fs, Path, file, watchPath, w;
fs = require('fs' );
Path = require('path');
file = __dirname + '/tmp.txt';
watchPath = Path.dirname(file); // changing this to just file makes it trigger
w = fs.watch ( watchPath, function (e,f) {
console.log("will not get here by itself");
w.close();
});
fs.writeFileSync(file,"test","utf-8");
fs.createWriteStream(file, {
flags:'w',
mode: 0777
} )
.end('the_date="'+new Date+'";' ); // another method fails as well
setTimeout (function () {
fs.writeFileSync(file,"test","utf-8");
},500); // as does this one
// child_process exec and spawn fail the same way with or without timeout
So the questions are: why? and how to trigger this event programmatically from a node script?
Thanks!
It doesn't trigger because a change to the contents of a file isn't a change to the directory.
Under the covers, at least as of 0.6, fs.watch on Mac uses kqueue, and it's a pretty thin wrapper around kqueue file system notifications. So, if you really want to understand the details, you have to understand kqueue, and inodes and things like that.
But if you want a short "lie-to-children" explanation: What a user thinks of as a "file" is really two separate things—the actual file, and the directory entry that points to the actual file. This is what allows you to have things like hard links, and files that can still be read and written even after you've deleted them, and so on.
In general, when you write to an existing file, this doesn't make any change to the directory entry, so anyone watching the directory won't see any change. That's why echo >tmp.txt doesn't trigger you.
However, if you, e.g., write a new temporary file and then move it over the old file, that does change the directory entry (making it a pointer to the new file instead of the old one), so you will be notified. That's why TextEditor.app does trigger you.
The thing is, you've asked to watch the directory and not the file.
The directory isn't updated when the file is modified, such as via shell redirection; in this case, the file is opened, modified, and closed. The directory isn't changed -- only the file is.
When you use a text editor to modify a file, the usual set of system calls behind the scenes looks something like this:
fd = open("foo.new")
write(fd, new foo contents)
unlink("foo")
rename("foo.new", "foo")
This way, the foo file is either entirely the old file or entirely the new file, and there's no way for there to be a "partial file" with the new contents. The renaming operations do modify the directory, thus triggering the directory watch.
Although the above answers seems reasonable, they are not fully accurate. It is actually a very useful feature to be able to listen to a directory for file changes, not just "renames". I think this feature works as expected in Windows at least, and in node 0.9.2 is also working for mac since they changed to the FSEvents API that supports the feature:
Version 0.9.2 (Unstable)