Batch file to compress and copy a folder

Recently, I needed to compress all contents of a folder (including subfolders) to an archive file, and copy it to a remote (network) location, all from a command line. In other words, I wanted to do this:

compressAndCopyFolder <sourceFolder> <destinationFolder> <archiveFileName>

Here’s how each parameter would work:

  • sourceFolder is the folder, along with all subfolders, to be added to the archive.
  • destinationFolder is the folder where the archive would be created.
  • archiveFileName is the file name of the archive

I prefer 7zip for archiving, so I could have done this simply using one command:

7z.exe" a -r "%destinationFolder%\%archiveFileName%.7z" %sourceFolder%\*

The problem with this is that it is inefficient to work with a large archive file over the network — there’s a lot of chatter going on to add files to an archive file. Much faster to create the archive locally, then copy the final archive file to the network. So, instead of one command, we have two:

echo off
if "%3"=="" (
	echo usage: 7zfolder ^<sourcefolder^> ^<destinationfolder^> ^<destinationfilename^>
	echo Note: The suffix .7z will be added at the end of destinationFilename.
	goto :eof
)
if not exist "%1" (
	echo ERROR: Source folder does not exist: %1
	goto :eof
)
if not exist "%2" (
	echo ERROR: Destination folder does not exist: %2
	goto :eof
)
if exist "%2\%3.7z" (
	@echo ERROR: Destination file already exists in destination folder: %2\%3
	goto :eof
)

@"c:\Program Files\7-Zip\7z.exe" a -r "%temp%\%3.7z" %1\*
@copy "%temp%\%3.7z" %2\%3.7z /z

Now, from a command line, I just do this…

7zfolder c:\mysource \\myserver\mydest archive

… and all files in c:\mysource will be added to an archive \\myserver\mydest\archive.7z.

Don’t you just love batch files?

Separating SQL script files generated by Microsoft SQL (by type)

There’s one great feature in SQL: the “Generate Scripts” command. Unfortunately, it has one limitation: the default filenames of scripts look something like this:

dbo.fnc_PlayerValue.UserDefinedFunction.sql
dbo.UserSelect.StoredProcedure.sql

I’d much prefer the filenames to match the object name, without the owner (‘dbo’) or object type. In other words, I’d prefer the above two files to look like this:

UserDefinedFunction\fnc_PlayerValue.sql
StoredProcedure\UserSelect.sql

How do we get from point A to point B without a lot of manual file copies and renames? We use the FOR command!

First, create a subdirectory for each object type (Table, StoredProcedure, UserDefinedFunction, View, Schema, Trigger, and User), then run the following from a command prompt.

for %i in (*.User.sql) do for /f "delims=., tokens=1-3" %j in ("%i") do move %i %k\%j.%l
for %i in (*.Schema.sql) do for /f "delims=., tokens=1-3" %j in ("%i") do move %i %k\%j.%l
for %i in (*.Trigger.sql) do for /f "delims=., tokens=1-3" %j in ("%i") do move %i %k\%j.%l
for %i in (*.sql) do for /f "delims=., tokens=1-4" %j in ("%i") do move %i %l\%k.%m

The command above will look for each file with a *.sql extension in the current directory. For each of those files, it copies it to a directory based on the type of file, and ensures the new file only includes the object name and the .sql extension. So much cleaner now!

Note that this will not work if any folder in the full path to the SQL files has a period in it!

Updated 2009.11.10 to support Trigger, User, and Schema scripts.

Automatically generate (partial) XML format files for BCP

I’ve been working with a lot of raw data files lately — data files that come in some delimited format which I must move into a SQL database. Most of the files are CSV files, and some of them have over 100 columns.

Using BCP to import CSV files into SQL is a pleasure. The only annoyance is writing those XML format files. Actually, the annoyance is not writing them, it’s going through the CSV files (which, of course, are undocumented) to determine what delimiters are used in each row.

Here’s a sample of what I mean:

2934,128321,2782,"2007-04-32","Excluded",2321,,22

Fortunately, most CSV files are consistent in their use of quotes, but going through dozens of columns to determine the terminators is a pain. The terminators in the above example aren’t just commas; they could also be quotes. Column three, for example, ends with a comma followed by a quote.

To generate the <record> section of my format files, I wrote the following script, which reads the first line of a text file, finds each comma, determines if it is preceded or followed by a quote, and generates the appropriate XML.

//get filename
var args = WScript.Arguments;
if (args.length == 0)
{
	WScript.Echo('Usage: getdelims.vbs <filename>');
	WScript.Quit();
}

var filename = args(0);

//read file
var fso = new ActiveXObject("Scripting.FileSystemObject");
var file = fso.OpenTextFile(filename,1);
var contents = file.ReadLine();
file.Close();
file = null;
fso = null;

//find commas
var cnt = 0;
for (var i = 1; i < contents.length; i++)
{
	if ( contents.substr(i,1) != ',' ) continue;
	cnt++;
	delim = ",";
	if ( contents.substr(i-1,1) == '"' )
		delim = '&quot;,';
	if ( i+1 < contents.length && contents.substr(i+1,1) == '"' )
		delim += '&quot;';
	WScript.Echo('\t<FIELD ID="' + cnt + '" xsi:type="CharTerm" TERMINATOR="' + delim + '" />');
}

The output can be copy/pasted right into your format file. The example content above would generate the following.

        <field ID="1" xsi:type="CharTerm" TERMINATOR="," />
        <field ID="2" xsi:type="CharTerm" TERMINATOR="," />
        <field ID="3" xsi:type="CharTerm" TERMINATOR=",&quot;" />
        <field ID="4" xsi:type="CharTerm" TERMINATOR="&quot;,&quot;" />
        <field ID="5" xsi:type="CharTerm" TERMINATOR="&quot;," />
        <field ID="6" xsi:type="CharTerm" TERMINATOR="," />
        <field ID="7" xsi:type="CharTerm" TERMINATOR="," />

A few minutes writing a script, and I won’t be looking at CSV files with too many columns ever again (at least, not for the reason of writing XML format files).

Scripting SQL database builds

A project I’m working on requires importing data from hundreds of megabytes of delimited text files into a staging database, then converting and migrating the data into an operations database. (Forgive the naming conventions. The “staging” database is used as an intermediary between the text files and the final database. The “operations” database is the one the application uses.)

What’s the best way to do automate this? Ideally, I need to be able to re-run the import at any time, preferably with a single command. The process should purge the staging database, rebuild tables (since schemas may change), reload all data, and run the necessary migration scripts. Since the final product is merely a web interface to the imported data, there is no harm in wiping and recreating everything from scratch, as there is no “user data” of note. (Actually there is, but it is isolated from this process.)

The solution I came up with used a series of SQL scripts executed using sqlcmd (part of MS SQL 2005; use osql for older versions) and data import tasks using bcp (part of MS SQL), all tied together using batch files. Continue reading

Toggling your HOSTS file

I use my laptop (a highly-recommended Toshiba Portege M100) for most of my development work. Typically, I develop on my laptop (a sandbox environment), then push data to another server (either a staging or production server). One of the problems I run into is my ever-changing physical location. When I’m at home, certain Web sites need internal IP addresses; at other places, they need external IP addresses.

The solution to this problem is by toggling the use of your HOSTS file. I enterd all the static entries into my HOSTS file, and created a small batch file to “toggle” the HOSTS file on and off. (When the HOSTS file exists, it is used, so renaming it to something other than HOSTS – with no file extension – turns it “off”.)

The batch file for toggling the HOSTS file is:

if exist c:\windows\system32\drivers\etc\hosts goto :hosts_off
:hosts_on
ren c:\windows\system32\drivers\etc\hosts.toggle hosts
goto :end
:hosts_off
ren c:\windows\system32\drivers\etc\hosts hosts.toggle
goto :end
:end
ipconfig /flushdns

The batch file simply changes the file from hosts to hosts.toggle and back again. The command ipconfig /flushdns is executed at the end to ensure the DNS cache is flushed.

Renaming files in a subdirectory tree

A coworker had an odd little requirement that they asked me if I could help with. She has image files stored in a directory structure, and she wanted to be able to pull the files out and rename them with the parent directories. Confused? I was, too. Essentially they want a file whose path is \A\01\23\45.tif to be renamed to A012345.tif.

Two steps to solving this. First, I needed a script file that would make a copy of a file and rename it by prepending the names of its parent folders to the filename. Second, I needed a script file which would find every file under a given subdirectory and call the first script file for each file found.

The first script file is a Windows Script Host file, written in VBScript. I name this file sdrename.vbs (as in subdirectory rename). The basic flow is:

  1. Retrieve the full, absolute path to the file from command line arguments.
  2. Determine the root folder (the folder that the script file is executed in).
  3. Extract the filename (only) from the full path of the file.
  4. Extract the names of all subfolders between the root folder and the file.
  5. Strip backslashes out of the subfolders extract (step 4).
  6. Determine the new filename (step 4 + step 3).
  7. Copy the file to the root folder.
  8. Rename the copied file to the new filename (step 6).

Option Explicit
 
'Filenames are required
If WScript.Arguments.Count = 0 Then
  WScript.Echo ">>> Must provide file name!!! <<<"
  WScript.Quit
End If
 
Dim strFile, strRootFolder, strFilename, strNewFilename, objFSO
 
WScript.Echo "*** SUBDIRECTORY FILE RENAMING ***"
WScript.Echo ""
 
strFile = WScript.Arguments(0)
WScript.Echo "Processing:   " & strFile
 
strRootFolder = Left( WScript.ScriptFullName, InStr( WScript.ScriptFullName, WScript.ScriptName ) - 1)
WScript.Echo "Root Folder:  " & strRootFolder
 
strFilename = Mid( strFile, InStrRev( strFile, "\" ) + 1 )
WScript.Echo "Filename:     " & strFilename
 
strNewFilename = Mid( strFile, Len( strRootFolder ) + 1 )
WScript.Echo "Subfolders:   " & strNewFilename
 
strNewFilename = Replace( strNewFilename, "\", "" )
WScript.Echo "New Filename: " & strNewFilename
 
Set objFSO = CreateObject( "Scripting.FileSystemObject" )
 
objFSO.CopyFile strFile, strFilename
WScript.Echo "File copied successfully."
 
objFSO.MoveFile strFilename, strNewFilename
WScript.Echo "File renamed successfully!"
 
WScript.Echo "*** PROCESS COMPLETE ***"

The second script is a simple one-line batch file, which I call renloop.bat (as in rename loop). This calls the shell’s for command to loop through all subdirectories, retrieving the full file name and path for each file that matches the search criteria.

for /r %%i in (%1) do cscript.exe sdrename.vbs "%%i"

To use, copy both script files to the starting directory – the top directory you want to include in the rename. For example, if I wanted to copy and rename all files within c:\mydocs\images\, both files would be copied into this directory. Then, run the following from a command prompt:

renloop *.gif

The result: Every file that matches the search criteria (*.gif) will have the sdrename.vbs script executed against it. Enjoy all your newly created, renamed files!

Updating the system path using Windows Script Host

On a recent business trip, I needed to add the full path to Microsoft Office and Lexis-Nexis CompareRite into the system path. (All workstations were running the French edition of Windows 2000.) Doing this manually – finding the installation path for the applications (since I wasn’t sure if all installations were to the same location on the local hard drives), then manually pasting or typing the paths into the System control panel – was too time-consuming for me, so I wrote a script to do it.

The Windows Script Host (WSH) engine is included with Windows 2000, so I wrote a WSH script using VBScript to do the necessary work:

  1. Read the current system path from the registry.
  2. Read the path to Microsoft Office from the registry.
  3. If the Office path is not already in the system path, append it.
  4. Read the path to CompareRite from the registry.
  5. If the CompareRite path is not already in the system path, append it.
  6. Update the system path in the registry if it changed.

The script follows:


'FILENAME: updpath.vbs

Option Explicit

'Stop at all errors by default
On Error Goto 0

'AddToPath: If sAdd is not in the string sPath, append sAdd to sPath and return the result;
' otherwise, return sPath unchanged.
Function AddToPath ( sAdd, sPath )
If InStr( 1, sPath, sAdd, 1 ) = 0 Then
AddToPath = sPath & ";" & sAdd
WScript.Echo sAdd & " added to path."
Else
AddToPath = sPath
End If
End Function

'DoScript: The main script procedure.
Sub DoScript
Dim WshShell, strPath, strOrigPath, strAppPath
Set WshShell = WScript.CreateObject("WScript.Shell")

On Error Resume Next

'Read the system path. Exit on errors.
strPath = WshShell.RegRead("HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\Path")
If Err.Number <> 0 Then Exit Sub
strOrigPath = strPath

'Read the Office path and call AddToPath.
strAppPath = WshShell.RegRead("HKLM\SOFTWARE\Microsoft\Office\8.0\BinDirPath")
If Err.Number = 0 Then
strPath = AddToPath(strAppPath, strPath)
End If

'Read the CompareRite path and call AddToPath.
strAppPath = WshShell.RegRead("HKLM\SOFTWARE\LEXIS-NEXIS\Office\7.0\CompareRite\Directory")
If Err.Number = 0 Then
strPath = AddToPath(strAppPath, strPath)
End If

'Update the system path if it's changed.
If strPath <> strOrigPath Then
WshShell.RegWrite "HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment\Path", strPath, "REG_EXPAND_SZ"
If Err.Number <> 0 Then Exit Sub
WScript.Echo "System path updated to:" & vbCrLf & strPath
Else
WScript.Echo "Path not updated - no updates necessary."
End If
End Sub

Call DoScript()

There are two things to consider with this script. First, the changes do not take effect until the user logs off and logs back on (no reboot, just re-logon). Second, the script requires Administrator-level privileges, because the registry key where the system path is stored is read-only to non-administrators. Using the Windows 2000 runas command, we can run the script when logged in as a non-administrator user by running the following line from a command prompt. Just change the user parameter to an administrator-level account, and the path to the actual location of the updpath.vbs file.

runas /profile /user:domain\administrator "wscript.exe \\server\share\updpath.vbs"

Change wscript.exe to cscript.exe if you want the output to appear in a console (command prompt) window.

Using the FOR command to copy files listed in a text file

A coworker asked me for a script. Here’s the request:

… would want to copy all files on this list [an attached text document] to another location (doesn’t really matter where for now). All are currently located in \\server\share\folder. The path in the new location should begin with the part after “folder”…

In a nutshell, here is the high-level description of what the script must do:

Given a text file which provides a list of files, copy the files from a fixed source to a fixed destination, recreating the directory trees on the destination. A simple file copy won’t work because there may be files in the source folders which should not be copied.

Sample content of the text file is:

client\CD120\Samarai Legends\Drafts\drafts folder.txt
client\CD120\Samarai Legends\Inbox\Legends.doc
client\CD120\Bushido Warriors\Inbox\Warrior Code.doc

The solution to this is to make a simple batch file that parses the content of the text file, generating the appropriate xcopy command to copy the file. We’ll call the batch file xcopylist.bat; its one line of content is below. Change c:\temp\ to whatever path you want the files copied to. (I used c:\temp\ for testing.) Change \\server\share\folder to the root folder of the files to copy.

for /f "delims=" %%i in (filelist.txt) do echo D|xcopy "\\server\share\folder\%%i" "c:\temp\%%i" /i /z /y

Put the file list in the same directory as the batch file and name the file list filelist.txt. Then run the batch file and viola! You’ve got to love the for command, which lets you (among other things) parse text files and use the line-by-line output. Another trick in here is the output parser pipe, which allows us to automatically press the “D” key with each xcopy command.

Note: The batch file overwrites files in the destination automatically. To turn this off (have it prompt you), change /y to /y- in the batch file. However, if you’re using Windows NT 4.0, just delete the /y switch altogether – it’s only supported in Windows XP and Windows 2000.

Moving a FrontPage Web site to a non-FrontPage Web site

For a web application I am working on, there are two Web sites – a development site and a production site. Both reside on the same server. Development is done using Microsoft Visual InterDev, so to connect to the development site, I have FrontPage extensions installed. (I also integrate with Microsoft Visual SourceSafe, but that’s meaningless to this discussion.)

Moving files from the development environment to the production environment usually entails manual copying, because I do not have FrontPage extensions installed on the production site (for obvious reasons; I don’t really need them!). I finally found a way to do it – the “old-fashioned” way – using a simple command: XCOPY.

XCOPY, which has been a part of Microsoft’s command shell (also known as DOS prompt) for years, copies files from a source to a destination with a wealth of options – most important for me, the ability to copy changed files only and to exclude files and folders.

First, you need an exclusions file – essentially a list of file name patterns (not wildcards). If a file or folder matches any pattern in the exclude file, it is skipped. Here’s my exclude file – which I named exclude.txt:

\temp\
\aspnet_client\
.vss
_vti
.scc

I’m telling XCOPY to skip the \temp\ and \aspnet_client\ folders, and skip any files that have .vss, .scc, or _vti in the filename. This ensures that unnecessary files are not copied to the production Web site.

The XCOPY command to run, which should work in Windows 2000 or Windows XP (not sure if Windows NT 4.0 supports all the XCOPY extensions here) is:

xcopy e:\devsite\*.* e:\website\ /D /P /E /V /R /-Y /EXCLUDE:exclude.txt

For you to use this, just change the source and destination paths, and viola! Dump this line into a batch file (call it dev2prod.bat) and make sure the exclude.txt file is in the same folder as the batch file.

Exporting directories to delimited text files

A coworker of mine needed some way to get a list of all files in a directory (and its subdirectories) in a delimited text file. Specifically, they needed a pipe-delimited text file with the following fields:

  • folder path (the absolute path to the folder containing the file)
  • modified date
  • modified time
  • size, in bytes
  • file name

The solution was a VBScript using Windows Scripting technologies. It’s rather simple, utilizing the Scripting.FileSystemObject object. We retrieve the current directory using the GetFolder() function, which returns a Folders collection (i.e., a collection of Folder objects). This object is passed as a parameter to the ParseFolder() function in our script.

The ParseFolder() function gets the data from each File object in its Files collection and outputs it to a pipe-delimited text file (called diraudit.txt. It then calls itself (a recursive function) for each Folder object in its Folders collection. The recursion takes care of all the subsequent subfolders.

The actual code of the script, which I named diraudit.vbs, follows.

Option Explicit

Const ForWriting = 2

Sub ParseFolder ( objFolder )

	Dim objFiles, f
	Set objFiles = objFolder.Files
	
	For Each f In objFiles
		objOutputFile.WriteLine Mid( f.Path, 1, InStr( f.Path, f.Name ) - 1 ) & "|" & _
			FormatDateTime( f.DateLastModified, vbShortDate ) & "|" & _
			FormatDateTime( f.DateLastModified, vbShortTime ) & "|" & _
			f.Size & "|" & f.Name
	Next

	Set objFiles = Nothing

	Dim objFolders
	Set objFolders = objFolder.SubFolders

	For Each f In objFolders
		ParseFolder( f )
	Next

	Set objFolders = Nothing

End Sub

Dim objFSO, objFolder, objOutputFile

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder( "." )

Set objOutputFile = objFSO.OpenTextFile("diraudit.txt", ForWriting, True)
objOutputFile.WriteLine "path|date|time|size|name"

Call ParseFolder( objFolder )

objOutputFile.Close

Set objOutputFile = Nothing
Set objFolder = Nothing
Set objFSO = Nothing