Windows / NTFS: Number of files in a directory, without enumerating them - windows

Is there a way in Windows to get the number of files in a directory (non-recursively is good enough) without the potentially significant overhead of enumerating them all through FindFirst/NextFile?
The purpose is for an estimate how many files need to be processed.
The directory structure is known and not deep, and folders usually don't contain unknown files.
Since the data is not essential to continue and the folder is virtually always on a local drive, an NTFS-specific command would be acceptable (provided that does not require elevation).
Any ideas?

Keeping track of the number of files in a directory is an overhead that doesn't provide any benefit for a file system, so none of them do it.
Counting files on a local drive should be fast enough for most purposes. It will certainly take a small fraction of the time needed to process the files.

Assuming that Indexing service is running and the directory is indexed, we can query the catalog and count the files on a given directory.
I dont know if querying the catalog is faster, its the only idea came on my mind.
Regards.
Sample code in VB.NET
Sub Main()
Try
' Catalog Name
Dim strCatalog As String = "System"
Dim strQuery As String
strQuery = "Select DocPageCount,DocTitle,Filename,Size,PATH,URL from SCOPE('shallow traversal of ""C:\Windows""')"
Dim connString As String = "Provider=MSIDXS.1;Integrated Security .='';Data Source='" & strCatalog & "'"
Dim cn As New System.Data.OleDb.OleDbConnection(connString)
cn.Open()
Dim cmd As New System.Data.OleDb.OleDbCommand(strQuery, cn)
Dim rdr As System.Data.OleDb.OleDbDataReader = cmd.ExecuteReader()
Dim files As Integer = 0
While rdr.Read()
files += 1
Console.WriteLine("Path: {0} Filename: {1} Size : {2}", rdr("PATH"), rdr("Filename"), rdr("Size"))
End While
rdr.Close()
rdr = Nothing
cmd.Dispose()
cmd = Nothing
cn.Close()
Console.WriteLine("Total Files: {0}", files)
Catch ex As Exception
Console.WriteLine(ex)
End Try
End Sub
More info: Searching your web site with the Microsoft Indexing Service and ASP.NET

Related

Looping over all databases in a directory

I need to loop over all databases in a certain directory, where the database name is ACPwxyz.mdb, where wxyz is the equivalent to an MMYY value for the period that database was used for.
E.g., the database for July 2017 would be ACP0717.mdb.
I've never written in VB6 before and I totally hate it, but it's an extension to an existing project so I'm stuck with it!
Is there a way of looping over all files in a directory, checking if the file name follows the format of ACPwxyz.mdb or not, and if it does, then opening a connection to it?
I've looked around a bit and see Dir(x, y), but I'm not sure if I can use this in this situation?
Any tips would be appreciated.
You can use Dir, yes.
If you use something like this:
Dim strFile As String
strFile = Dir(yourDBPath, "ACP????.mdb") ' mdb for MS-Access files
Do Until strFile = ""
If Len(strFile) = 11 Then ' Ensure the DB file name is 11 characters, which yours are
'Do something // You can also check the file name doesn't = a certain name if needed
End If
strFile = Dir
Loop
Dir accepts either an asterisk (*), or a question mark (?) as wildcards in file names, so this will look for any database in the set path that is called ACP followed by 4 characters.

Comparing files in two different folders with string variable names

Due to automatic naming conventions I'm stuck with, files are saved in a particular archive one of two ways. Either they have a 13-digit match to the record in the database, or they have a 10 digit match. The ones which have the 13 digit also have the ten digit match, and should be treated preferentially. I have two scripts right now that work: one just looks for 13-digit matches and moves them to another folder. The other moves by datemodified, which is not a very stable variable.
My attempts at combining look something like this:
'dimming of obj, folder paths, sql db info and connection open above
SQL = "SELECT ticket FROM orders WHERE dateinvoice>'6/1/16"
Recordset.Open SQL, Connection
Do While Not Recordset.EOF
ticket = Recordset("ticket")
id = Right(ticket, 2)
suffix = CInt(id)
compare = Left(ticket, Len(ticket) - 3)
search = compare & "-0" & suffix
For Each objFile In colFiles.Files
If Left(objFile.Name, 13) = search Then
Set objNewestFile = objFile
skip = 1
Else
skip = 0
End If
If Left(objFile.Name, 10) = search And skip = 0 Then
Set objNewestFile = objFile
End If
Next
Recordset.MoveNext
On Error Resume Next
objFSO.CopyFile objNewestFile.Path, strDestFolder
Set objFile = Nothing
Loop
This is the scaled-down version I went back to, having attempted various convoluted solutions like:
storing the skipped records in an array,
having a second loop through comparing files in folder1 to folder2 (every time I use FileExists it doesn't seem to recognize the destination),
creating a temporary SQL table to store the skipped records in.
The following have been very useful:
Compare two files in two different folders and replace it by the newer
https://blogs.technet.microsoft.com/heyscriptingguy/2005/06/20/how-can-i-determine-if-a-file-exists-and-if-it-does-exit-the-script/
Classic ASP 3.0 Create Array from a Recordset
ETA: There can be many duplicate files that share the 10 digit identifier. The only good rule is: if it has the 13 identifier version, that's the one to move. Part of my issue lies in setting paths if I use FileExists-- when I try wildcards, it doesn't recognize the path. If I don't use them, it doesn't compensate for all the variables.

Iterate over pre-filtered list of folders

I'm working with a client's flat folder structure that has a single folder containing 45k subfolders with 8-digit folder names, e.g. 51023231. I have a small script that ripples through them and copies them into a bin-sorted set of subfolders elsewhere on the network (for use with SharePoint), such that the first 5 digits are used as a parent folder, i.e. the contents of 51023231 are copied into 51023\51023231. It works perfectly well, and I've managed to some modest optimisation when dealing with folders new to the destination.
However, it can take an hour or so to run over the entire 45k set of folders doing folder by folder comparisons, I was wondering if it was possible to run a system-level query to return only an initial list of folders whose modification dates were after a given point, and then run the existing script over that. I've done the usual Google-is-your-friend type trawl, and keep hitting the idea of using WMI to do this, but I don't get much further. Is this because it simply isn't possible with VBScript?
Any pointers gratefully received.
It's possible with VBScript using WMI. It's not something you'd be able to do using the FileSystemObject, if that's what you're implying.
You can query WMI's Win32_Directory class to filter folders by modified date. The only tricky part is the datetime format used by WMI. But the SWbemDateTime class can convert a VBScript date to a datetime value.
Here's an example:
' Create a datetime value for use in our WMI query...
Dim dt
Set dt = CreateObject("WbemScripting.SWbemDateTime")
dt.SetVarDate DateSerial(2015, 8, 31)
Dim objWMI
Set objWMI = GetObject("winmgmts:\\.\root\cimv2")
' Find all subfolders of 'c:\some\path' that were modified > 2015/08/31
Dim objFolders
Set objFolders = objWMI.ExecQuery("select * from Win32_Directory" _
& " where Drive='C:' and Path='\\some\\path\\' and LastModified>'" & dt & "'")
Dim objFolder
For Each objFolder in objFolders
WScript.Echo objFolder.Name
Next

Search filesystem for filepath using VBA

I am looking for a way to search a specific folder for a subfolder containing a certain string. Below I have listed a function I typed off the top of my head to see if it would work. Well, it does work but when I am talking about searching through 6,000 folders on a network drive it just isn't fast enough.
I'm sure that there is a better way to do it but I can't seem to dig anything up on Google.
Is there an object that allows me to leverage the windows built in file system searching and indexing capabilities?
As an alternative, does someone have a way to optimize my code? The main bottleneck is the usage of instr.
Here is the code:
Function findPath(strId As String) As String
checkObj
Dim strBase As String
strBase = opt.photoBasePath
Dim fs As Object
Set fs = CreateObject("Scripting.FileSystemObject")
Dim baseFolder As Object
Set baseFolder = fs.getfolder(strBase)
Dim folder As Object
For Each folder In baseFolder.subfolders
If InStr(1, folder.name, strId) > 0 Then
findPath = strBase & "\" & folder.name
Exit Function
End If
Next folder
End Function
P.S. I'm sure someone will suggest modifying my folder structure so that I can programmatically predict the path but for various reason that isn't possible in my case.
You could use the FindFirstFile Win32 API, which allows you to search for files or sudirectories matching a specified name. Additionally, you could also use the FindFirstFileEx function, along with a FINDEX_SEARCH_OPS parameter of FindExSearchLimitToDirectories, which would limit your search to a file that matches a specified name and is also a directory (if the file system supports directory filtering). For more information on using these functions from VB/VBA see the following:
http://www.xtremevbtalk.com/showpost.php?p=1157418&postcount=4
http://support.microsoft.com/kb/185476
http://www.ask-4it.com/how-to-use-findfirstfile-win32-api-from-visual-basic-code-2-ca.html
Consider splitting traversing the folders from finding your key. Instead of the instr test, store the folder names in a table, then use a query on the table to find your target. It might be slower, but searching should be faster.

Visio to image command line conversion

At work we make pretty extensive use of Visio drawing as support for documentation. Unfortunately vsd files don't play nicely with our wiki or documentation extraction tools like javadoc, doxygen or naturaldocs. While it is possible to convert Visio files to images manually, it's just a hassle to keep the image current and the image files are bound to get out of date. And let's face it: Having generated files in revision control feels so wrong.
So I'm looking for a command line tool that can convert a vsd file to jpeg, png, gif or any image that can be converted to an image that a browser can display. Preferably it will run under unix, but windows only is also fine. I can handle the rest of the automation chain, cron job, image to image conversion and ssh, scp, multiple files, etc.
And that's why I'm turning to you: I can't find such a tool. I don't think I can even pay for such a tool. Is my Google-fu completely off? Can you help me?
I mean, it has got to be possible. There has to be a way to hook into Visio with COM and get it to save as image. I'm using Visio 2007 by the way.
Thanks in advance.
I slapped something together quickly using VB6, and you can download it at:
http://fournier.jonathan.googlepages.com/Vis2Img.exe
You just pass in the input visio file path, then the output file path (visio exports based on file extension) and optionally the page number to export.
Also here is the source code I used, if you want to mess with it or turn it into a VBScript or something, it should work, though you'd need to finish converting it to late-bound code.
hope that helps,
Jon
Dim TheCmd As String
Const visOpenRO = 2
Const visOpenMinimized = 16
Const visOpenHidden = 64
Const visOpenMacrosDisabled = 128
Const visOpenNoWorkspace = 256
Sub Main()
' interpret command line arguments - separated by spaces outside of double quotes
TheCmd = Command
Dim TheCmds() As String
If SplitCommandArg(TheCmds) Then
If UBound(TheCmds) > 1 Then
Dim PageNum As Long
If UBound(TheCmds) >= 3 Then
PageNum = Val(TheCmds(3))
Else
PageNum = 1
End If
' if the input or output file doesn't contain a file path, then assume the same
If InStr(1, TheCmds(1), "\") = 0 Then
TheCmds(1) = App.Path & "\" & TheCmds(1)
End If
If InStr(1, TheCmds(2), "\") = 0 Then
TheCmds(2) = App.Path & "\" & TheCmds(2)
End If
ConvertVisToImg TheCmds(1), TheCmds(2), PageNum
Else
' no good - need an in and out file
End If
End If
End Sub
Function ConvertVisToImg(ByVal InVisPath As String, ByVal OutImgPath As String, PageNum As Long) As Boolean
ConvertVisToImg = True
On Error GoTo PROC_ERR
' create a new visio instance
Dim VisApp As Visio.Application
Set VisApp = CreateObject("Visio.Application")
' open invispath
Dim ConvDoc As Visio.Document
Set ConvDoc = VisApp.Documents.OpenEx(InVisPath, visOpenRO + visOpenMinimized + visOpenHidden + visOpenMacrosDisabled + visOpenNoWorkspace)
' export to outimgpath
If Not ConvDoc.Pages(PageNum) Is Nothing Then
ConvDoc.Pages(PageNum).Export OutImgPath
Else
MsgBox "Invalid export page"
ConvertVisToImg = False
GoTo PROC_END
End If
' close it off
PROC_END:
On Error Resume Next
VisApp.Quit
Set VisApp = Nothing
Exit Function
PROC_ERR:
MsgBox Err.Description & vbCr & "Num:" & Err.Number
GoTo PROC_END
End Function
Function SplitCommandArg(ByRef Commands() As String) As Boolean
SplitCommandArg = True
'read through command and break it into an array delimited by space characters only when we're not inside double quotes
Dim InDblQts As Boolean
Dim CmdToSplit As String
CmdToSplit = TheCmd 'for debugging command line parser
'CmdToSplit = Command
Dim CharIdx As Integer
ReDim Commands(1 To 1)
For CharIdx = 1 To Len(CmdToSplit)
Dim CurrChar As String
CurrChar = Mid(CmdToSplit, CharIdx, 1)
If CurrChar = " " And Not InDblQts Then
'add another element to the commands array if InDblQts is false
If Commands(UBound(Commands)) <> "" Then ReDim Preserve Commands(LBound(Commands) To UBound(Commands) + 1)
ElseIf CurrChar = Chr(34) Then
'set InDblQts = true
If Not InDblQts Then InDblQts = True Else InDblQts = False
Else
Commands(UBound(Commands)) = Commands(UBound(Commands)) & CurrChar
End If
Next CharIdx
End Function
F# 2.0 script:
//Description:
// Generates images for all Visio diagrams in folder were run according to pages names
//Tools:
// Visio 2010 32bit is needed to open diagrams (I also installed VisioSDK32bit.exe on my Windows 7 64bit)
#r "C:/Program Files (x86)/Microsoft Visual Studio 10.0/Visual Studio Tools for Office/PIA/Office14/Microsoft.Office.Interop.Visio.dll"
open System
open System.IO
open Microsoft.Office.Interop.Visio
let visOpenRO = 2
let visOpenMinimized = 16
let visOpenHidden = 64
let visOpenMacrosDisabled = 128
let visOpenNoWorkspace = 256
let baseDir = Environment.CurrentDirectory;
let getAllDiagramFiles = Directory.GetFiles(baseDir,"*.vsd")
let drawImage fullPathToDiagramFile =
let diagrammingApplication = new ApplicationClass()
let flags = Convert.ToInt16(visOpenRO + visOpenMinimized + visOpenHidden + visOpenMacrosDisabled + visOpenNoWorkspace)
let document = diagrammingApplication.Documents.OpenEx(fullPathToDiagramFile,flags)
for page in document.Pages do
let imagePath = Path.Combine(baseDir, page.Name + ".png")
page.Export (imagePath)
document.Close()
diagrammingApplication.Quit()
let doItAll =
Array.iter drawImage getAllDiagramFiles
doItAll
You can try "Visio to image" converter
http://soft.postpdm.com/visio2image.html
Tested with MS Visio 2007 and 2010
There has to be a way to hook into Visio with COM and get it to save as image.
Why not try writing something yourself, then, if you know how to use COM stuff? After all, if you can't find anything already made to do it, and you know you can figure out how to do it yourself, why not write something to do it yourself?
EDIT: Elaborating a bit on what I stated in my comment: writing a script of some sort does seem to be your best option in this situation, and Python, at least, would be quite useful for that, using the comtypes library found here: http://starship.python.net/crew/theller/comtypes/ Of course, as I said, if you prefer to use a different scripting language, then you could try using that; the thing is, I've only really used COM with VBA and Python at this point (As an aside, Microsoft tends to refer to "Automation" these days rather than specifically referencing COM, I believe.) The nice thing about Python is that it's an interpreted language, and thus you just need a version of the interpreter for the different OSes you're using, with versions for Windows, OSX, Linux, Unix, etc. On the other hand, I doubt you can use COM on non-Windows systems without some sort of hack, so you may very well have to parse the data in the source files directly (and even though Visio's default formats appear to use some form of XML, it's probably one of those proprietary formats Microsoft seems to love).
If you haven't used Python before, the Python documentation has a good tutorial to get people started: http://docs.python.org/3.1/tutorial/index.html
And, of course, you'll want the Python interpreter itself: http://python.org/download/releases/3.1/ (Note that you may have to manually add the Python directory to the PATH environment variable after installation.)
When you write the script, you could probably have the syntax for running the script be something like "python visioexport.py <source/original file[ with path]>[ <new file[ with path]>]" (assuming the script file is in your Python directory), with the new file defaulting to a file of the same name and in the same folder/directory as the original (albeit with a different extension; in fact, if you wish, you could set it up to export to multiple formats, with the format defaulting to that of whatever default extension you choose and being specified by an alternate extension of you specify one in the file name. As well, you could likely set it up so that if you only have the new file name after the source file, no path specified, it'll save with that new file name to the source file's directory. And, of course, if you don't specify a path for the source file, just a file name, you could set it up to get the file from the current directory).
On the topic of file formats: it seems to me that converting to SVG might be the best thing to do, as it would be more space-efficient and would better reflect the original images' status as vectored images. On the other hand, the conversion from a Visio format to SVG is not perfect (or, at least, it wasn't in Visio 2003; I can't find a source of info similar to this one for Visio 2007), and as seen here, you may have to modify the resultant XML file (though that could be done using the script, after the file is exported, via parts of the Python standard library). If you don't mind the additional file size of bitmaps, and you'd rather not have to include additional code to fix resultant SVG files, then you probably should just go with a bitmap format such as PNG.

Resources