Windows NT Shell Scripting

Tim Hill

Mentioned 4

Windows NT Shell Scripting is a comprehensive reference for network professionals. It is the only book available on the practical use of the Windows NT shell scripting language. The book begins with a high-level introduction to the shell language itself, then describes the shell commands that are useful for controlling or managing different components of a network, i.e. file management, etc. The second part of the book is a comprehensive reference of all the commands, organized by function, for easy reference by the reader.

More on Amazon.com

Mentioned in questions and answers.

I have a Windows batch file I'm creating, but I have to ECHO a large complex string, so I'm having to put double quotes on either end. The problem is that the quotes are also being ECHOed to the file I'm writing it to. How do you ECHO a string like that and strip the quotes off?

UPDATE:

I've spent the last two days working on this and finally was able to kludge something together. Richard's answer worked to strip the quotes, but even when I put the ECHO in the subroutine and directly outputted the string, Windows still got hung up on the chars in the string. I'll accept Richard's answer since it answers the question asked.

I ended up using Greg's sed solution, but had to modify it because of sed/windows bugs/features (it didn't help that it came with no documentation). There are a few caveats to using sed in Windows: you have to use double quotes instead of single quotes, you can't escape the double quotes in the string directly, you have to endquote the string, escape using the ^ (so ^") then beqin quote for the next section. Also, someone pointed out that if you pipe input to sed, there's a bug with a pipe being in the string (I didn't get to verify this since in my final solution, I just found a way not to have all quotes in the middle of the string, and just removed all quotes, I never could get the endquote to be removed by itself.) Thanks for all the help.

The call command has this functionality built in. To quote the help for call:

 Substitution of batch parameters (%n) has been enhanced.  You can
 now use the following optional syntax:

 %~1         - expands %1 removing any surrounding quotes (")

Here is a primitive example:

@echo off
setlocal
set mystring="this is some quoted text"
echo mystring=%mystring%
call :dequote %mystring%
echo ret=%ret%
endlocal
goto :eof

:dequote
setlocal
rem The tilde in the next line is the really important bit.
set thestring=%~1
endlocal&set ret=%thestring%
goto :eof

Output:

C:\>dequote
mystring="this is some quoted text"
ret=this is some quoted text

I should credit the 'environment variable tunneling' technique (endlocal&set ret=%thestring%) to Tim Hill, 'Windows NT Shell Scripting'. This is the only book I have ever found that addresses batch files with any depth.

I want to have an executable file that will call some other programs. The way I would do this in Linux is with a simple bash script that looks like this:

#!/bin/bash
echo "running some-program"
/home/murat/some-program arg1 arg2

What's the best way to do this kind of thing under Windows?

The Windows NT family of Operating Systems (NT 3.51, NT 4.0, 2000, XP, Vista) have a reasonably powerful Command Shell even if it is a bit esoteric. This shell does not have the power of the typical UNIX shells however it is ubiquitous on Windows machines and it is pretty amazing what can be achieved. A great book for this is Windows NT Shell Scripting by Timothy Hill. I always found Rob van der Woude's Scripting Pages a great resource for batch files.

PowerShell is Microsoft's latest offering in this field and takes scripting to a whole new level and gives you access to all of the power of the .NET framework as well as WMI, COM and any other interface into Windows. There is a pretty steep learning curve with this but it is definitely the way to go and I think it has the potential to become one of those foundational technology skills (like RegEx, XPath etc) in the Microsoft world. A good starting point is Microsoft Windows PowerShell Technology Center but there are a huge number of resources out there for this.

What are the best free resources for learning advanced batch-file usage?

It's not free, but it's probably the best. "Windows NT Shell Scripting" by Tim Hill.

That said, whenever I try to do something 'advanced' in cmd batch files, I always end up regretting it.

Always.

edit: some explanation of the shortcomings of batch files:

  • math capabilities are pathetic
  • quoting support is an afterthought - if you need to pass a quoted string as an argument to a command that needs to be quoted itself, reserve a spot at the asylum (actually, I'm not sure it's possible)
  • string manipulation is a patchwork of half implemented functionality

Then there are the seemingly never-ending bits of oddities, corner cases, and inconsistencies that you run into at every turn.

The only thing going for batch files is that they're supported on every Windows box out there. If you just want to automate executing a few commands as a group, great. Maybe add a simple loop, a couple of subroutines, and some environment variables to parameterize things. Beyond that I strongly recommend you use something else.

I've run into a weird error with a Qt program running on Windows. The program uses QProcess to spawn a child process wit two arguments. The program and arguments passed to the QProcess::start() method are of the form:

"batchfile.bat" "--option1=some_value" "--option2=some_other_value\with_a\path"

For some reason by the time those options get to the batchfile for processing the equals signs have been converted to spaces and it now looks like:

"batchfile.bat" "--option1 some_value" "--option2 some_other_value\with_a\path"

because of this, the processing fails. Any ideas what could be causing the equal signs to be replaced by spaces? I'm using the mingw build of the QT 4.6.3 framework found on the Qt download page.

EDIT: Here's the actual code. I didn't write it (I'm a complete Qt noob) but I've got to try to get it working. It's part of an automated build system that runs on two versions of RHEL (4 and 5), OS X, and Windows. And it works fine everywhere but Windows.

QProcess sconsProcess;
sconsProcess.setWorkingDirectory(build.getBuildLocation());
sconsProcess.setProcessChannelMode(QProcess::MergedChannels);

qDebug()<<"Starting scons process:"<<build.getSconsLocation()<<QString("--variant=%1-%2").arg(build.getOs()).arg(build.getVariant())<<
          QString("--source-release=%1").arg(build.getSettings().getSetting("sourceReleaseLocation", QStringList()<<"BUILDLOCATION"<<"VERSION",
                    QStringList()<<build.getBuildLocation()<<build.getBuildPackage().getVersion()).toString());
sconsProcess.start(build.getSconsLocation(), QStringList()<<QString("--variant=%1-%2").arg(build.getOs()).arg(build.getVariant())<<
          QString("--source-release=%1").arg(build.getSettings().getSetting("sourceReleaseLocation", QStringList()"BUILDLOCATION"<<"VERSION",
                    QStringList()<<build.getBuildLocation()<<build.getBuildPackage().getVersion()).toString()));
qDebug()<<"Source release build process started";

The actaul values that translates into in Windows (the bit that gets printed out in the first qDebug() print call) is:

DEBUG: Starting scons process: "V:\Glast_Software\Toaster\tools\Python2.5\Scripts\scons-1.3.0.bat" "--variant=Windows-i386-32bit-vc71-Debug" "--source-release=V:\Glast_Software\Toaster\ReleaseManagerBuild\Windows-i386-32bit-vc71\Debug\ScienceTools\LATEST-1-3163\ScienceTools-LATEST-1-3163-source.zip"

However inside the scons-1.3.0.bat (I had it echo all the commands executed) the passed parameters look like:

"--variant Windows-i386-32bit-vc71-Debug" "--source-release V:\Glast_Software\Toaster\ReleaseManagerBuild\Windows-i386-32bit-vc71\Debug\ScienceTools\LATEST-1-3163\ScienceTools-LATEST-1-3163-source.zip"

with the equal signs missing.

EDIT (6/29/10): I should add that this system is designed to run on a small Windows batch farm using the LSF batch queuing system. It only fails when the process is running as a batch job. When I run this program from the command line on one of the batch machines, it works perfectly and does exactly what it is supposed to do. So maybe it is an environment problem.

There's a good chance that this is because the quotes aren't making it through (they may need to be escaped, see the docs for QProcess::start()).

cmd.exe treats equals signs in command line options that aren't quoted as a separator between arguments similar to a space or tab. Just one of very many bits of oddness in Windows cmd scripting:

C:\test>type c:\util\cmdechoargs.cmd
@echo off
setlocal
set /a i=0
echo args[*]: %*
:loop
if {%1} == {} goto :eof
echo argv[%i%]: %1
set /a i=%i% + 1
shift
goto :loop


C:\test>cmdechoargs testing=123
args[*]: testing=123
argv[0]: testing
argv[1]: 123

C:\test>cmdechoargs "testing=123"
args[*]: "testing=123"
argv[0]: "testing=123"

The best documentation I've come across for how to handle command line arguments in Windows cmd scripts is Tim Hill's "Windows NT Shell Scripting" - get one used for only a penny!

Based on the examples given in your update, I think you might want your options that have equals signs in them to have quotes embedded inside them:

"\"--variant=%1-%2\""
"\"--source-release=%1\""

Edit -- new material

The following script has a routine that will strip the quotes off of an argument passed to a cmd script. The routine returns the 'dequoted' argument in an environment variable named RET using an idiom/technique from Tim Hill's book I mentioned above. I stole some of the dequoting code from an example here: http://ss64.com/nt/syntax-esc.html, but made it a bit more robust to handle empty quotes.

@echo off
setlocal
set /a i=0
echo args[*]: %*
:loop
if {%1} == {} goto :eof
echo.
echo argv[%i%]: %1

call :dequote %1
set dequoted_arg=%RET%
echo argv[%i%] ^(dequoted^): %dequoted_arg%

set /a i=%i% + 1
shift
goto :loop


:dequote
setlocal
SET _string=###%1###
if {%_string%} == {######} goto :dequote_empty
if {%_string%} == {###""###} goto :dequote_empty
SET _string=%_string:"###=%
SET _string=%_string:###"=%
SET _string=%_string:###=%
goto :dequote_done

:dequote_empty
set _string=

:dequote_done
endlocal & (set RET=%_string%) & goto :eof

This kind of thing is why you want to avoid (in my opinion) cmd scripts except for the simplest of tasks. But, I hope this helps you pass unquoted arguments to your scons process through your batch file.