Working with Pipelines
A Pipeline Example
The pipeline is an old concept developed in
the UNIX command language. It is a
series of commands connected by pipeline operators “|”. Each pipeline operator
sends the results of the preceding command to the next command. The result is a very powerful command chain
or "pipeline" that is comprised of a series of simple commands. where
the output of one command becomes the input for the next, like stages on an
assembly line. As the data passes
through the pipeline it is refined at each stage. In a pipeline, the commands are processed
from left to right in the order that they appear. The processing is handled as
a single operation and output is displayed as it is generated. For example:
Type:
get-process taskmgr |
Stop-Process
Parameter binding is done either by “by
value”, which means that the piped object must be of the same type as the
parameter object or be able to be converted to it, or “by PropertyName, which
means the piped object must have the same name as the parameter of the input
command.
To find out which parameters are able to be
used in a pipeline, type the following:
Error Checking the Pipeline
Type: get-help
stop-process –parameter *
Figure 2: Finding Cmdlet Parameters
We see that the Name parameter accepts
pipeline input, but only as a propertyname which means that the get-process
command must find the string name of the process and pass that to the name
parameter of the stop-process. We can
see the pipeline process in real time using the trace-command. This is an excellent command to debug the
pipeline process when errors occur.
Type:
trace-command -name parameterbinding
-expression {get-process taskmgr | `
stop-process} -pshost -filepath debugTaskmgr.txt
Note: the line continuation character “`”
when you have long lines of text.
Figure
3: Using the Trace-Command to Discover Pipeline Errors
Open the
debugTaskmgr file. We can see that the
binding failed because the name parameter “taskmgr” could not be found by the
get-process cmdlet so the Stop-Process name parameter had nothing to bind to
and could not execute. The reason of
course is that we didn’t open the application Task Manager before running the
command. Open Task Manager and run the
Get-Process taskmgr | Stop-Process
pipeline again.
Notice that
each command in the pipeline is executed separately; the get-process command
searches the running processes and finds the name and ID of the taskmgr process
and passes this information to
Figure 4: Get-Process Taskmgr Output
the stop-process command. The stop-process
binds the input name to its property name and then uses the associated ID
number to close the program.
By combining cmdlets together using the pipe operator, we can create
powerful compound commands that process the data as it travels through the
pipeline. A graphical example is
below. Suppose there was a cmdlet called
get shapes. This command returns a
collection of object shapes installed on the local system.
|OUT-FILE
OrangeShapes.txt
|
WHERE-OBJECT
{ $_ .Color -eq “orange”} |
|
Figure 4:Graphical Example of Pipelining
The output of
this command becomes the input for the Where-Object cmdlet which filters the
collection based on the expression contained in the script blocks. In this case, it selects only the objects
which are equal to orange in color.
Notice a new variable “$_”;
this is an automatic variable which is created by PowerShell to hold each item
in the pipeline. Each item in the
collection is processed one at a time; then the output is piped to the
Sort-Object cmdlet which arranges the orange objects in ascending order. This sorted list is then sent to the Out-file
cmdlet which takes the input and formats it into a table contained in a
file. You can see that when working with
pipes the order of the commands is important, for example you can’t Sort-Object | Get-Sharps, or Out-File| Sort
Object.
A Practical Pipeline Example
A typical network job is finding specific
processes running on a local system. The
Get-Process cmdlet returns a collection of running processes. Suppose you wanted to find all the Internet
explore processes and output the information to a file.
Type: Get-Process
Type: Get-Process | Where-Object {
$_.Name –eq “iexplore”} | Sort-Object | Out-File IEProcess.txt
Note: an alternate method to use the “`” escape character as an end of
line marker, is to press the enter key after a command and PowerShell will
display an additional line prompt”>>” to complete the command.
The next command Sort-Object filters the
collection looking only for iexplore processes.
Sort-Object takes the entire filtered collection and sorts it ascending
by name, which is the default, since we did not add any parameters. The Sort-object command is called a
sequential mode pipeline because sorting cannot begin until the entire collection
has been received, working on each item one at a time. This mode is slow and hogs memory because results are returned only after
all commands finish their work and the pipeline has to store the entire results
of each command. Normally, however, the pipeline works on each item-by-item in what
is called streaming mode. In this mode,
every single result is passed directly onto the subsequent command. This mode
is quick and saves memory because results are output while the pipeline
commands are still performing their tasks; the pipeline only stores a single
item at a time in the $_ variable. The
final result is piped to a file called IEProcess.txt
We need to be careful when pipelining
specially on commands like Sort-object because if the collection is large, it can
take long time to complete. PowerShell
will look like it is frozen and you could run out of memory and crash your
system. Commands like Get-ChildItem C:\ -recurse | Sort-Object --- Would likely crash your system-- Do Not
execute this command!
Using the Out-Host –Paging CmdLet to Save
System Resources
The Out-Host
-Paging command is a useful pipeline element whenever you want to display a
lengthy output slowly. It is especially useful if the operation is very
CPU-intensive because processing is transferred to the Out-Host cmdlet page by
page, and the cmdlet that precedes it in the pipeline halt their operation
until the next page of output is available. You can see this if you use the
Windows Task Manager to monitor the CPU and memory consumed by Windows
PowerShell. For example, run the following commands:
Type:
taskmgr . Click on the Performance
tab to monitor CPU usage.
Type: Get-ChildItem
C:\windows –recurse Record the
usage
Type: Get-ChildItem c:\windows –recurse |
Out-Host –paging Record the usage.
Figure
5: Comparing CPU Usage Using Out-Host -Paging Cmdlet
The Out-Host
–paging command provides a significant saving in system resources, but requires
user interaction during the pipeline to process each page.
Modifying Pipeline Execution: Whatif
Confirm and Suspend
PowerShell provides some common parameters
to modify the execution of the pipeline.
|
|
|
Parameter
|
Meaning
|
|
-whatif
|
Tells the cmdlet not to execute; instead it will
tell you what would happen if the cmdlet were to actually run.
|
|
-confirm
|
Tells the cmdlet to prompt prior to executing the
command.
.
|
|
-verbose .
|
Instructs the cmdlet to provide a higher level of
detail than a cmdlet not using the verbose parameter.
|
|
-debug
|
- Instructs the cmdlet to provide debugging
information.
|
|
-erroraction
|
- Instructs the cmdlet to perform a certain action
when an error occurs. Allowable actions are: continue, stop,
SilentlyContinue, and inquire.
|
|
-errorvariable
|
Instructs the cmdlet to use a specific variable to
hold error information. This is in addition to the standard $error variable.
|
|
-outvariable
|
Instructs the cmdlet to use a specific variable to
hold the output information.
|
|
-outbuffer
|
Instructs the cmdlet to hold a certain number of
objects prior to calling the next cmdlet in the pipeline
|
For example, suppose we want to stop all
notepad processes on the local system.
To illustrate open two instances of notepad:
Type: notepad
Type: notepad
Type: Get-Process
|Where-Object { $_.Name –eq “Notepad”}
Figure 6: Output of the Notepad pipe
This confirms two sessions are currently
active. Now suppose we type a command like the following:
Type:
Get-Process | Where-Object {
$_.Name –like “n*”} | Stop-Process –whatif
Figure 7: Using the Whatif Common Parameter
Your system display will be different than the above screenshot.
The –whatif parameter gives you the opportunity to model the command
prior to execution showing you what will happen. Now you can modify the command to get the
desired result.
The –confirm parameter is similar to the
whatif, but prompts you for a yes selection prior to execution.
Type:
Get-Process | Where-Object {
$_.Name –like “n*”} | Stop-Process –Confirm
Figure 8:Using the Confirm Common Parameter
Adding
the confirm parameter also provides a “quick and dirty” menu for user
preference. You can also use the –suspend parameter which suspends the
execution of the command and opens a nested command prompt so you can execute
another command. When that command is
finished, you type exit to return to the suspended command. For example:
The
use of whatif,
confirm and suspend parameters are essential tools for administrators. Using them to
model commands before execution can save hours of work each year.
|
Type: Get-Process
| Where-Object { $_.Name –like “n*”} | Stop-Process –Confirm
Type:
press s for
suspend -- notice the prompt now looks like this “>>”. This means a child
shell has been launched inside the parent shell.
Type:
mspaint
Type: Get-Process
| Where-Object { $_.Name –eq “ms*”} | Stop-Process -confirm
Type: 8 (enter key)
Type:
exit – this returns to the parent shell and the
suspended command
Type:
8 (enter key) twice
to close the 2 notepad sessions.
Using the Tee-Object and Pass Through
Parameters
So far the
pipeline is one linear action, however, there are times when an administrator
needs to access the items travelling through the pipeline. The most common example is logging. Suppose you are administering a network and
you want to get a listing of all the bad processes which will be stopped. If you wait to the end of the pipeline, it’s
too late to create a log file. The bad processes
don’t exist anymore. You need to create
the log file in the middle of the pipeline.
This is a job for the Tee-Object; it makes the pipeline go in two
directions at once.
Type: mspaint
Type: Get-Process
mspaint| Tee-Object -file kill.log
|Stop-Process
Type: Get-Content kill.log
An alternate method of saving the bad process
to a log file is to use the passthru parameter with the Stop-Process
cmdlet. This is a linear process, but the
passthru parameter allows the system prompting to pass through the pipeline
which can be captured to a file at the end of the pipeline. Without the passthru parameter or the
Tee-Object cmdlet there would be nothing to log after the Stop-Process command
finished.
Type:mspaint
Type: Get-Process
mspaint| Stop-Process –passthru > kill.log
Measuring and Comparing Objects
A common scenario
for a network administrator is to provide the number, maximum and minimum size
of files and comparing the differences between two objects. The first procedure is best performed with
the Measure-Object cmdlet. For simple
statistics, this is the best choice. It
is much easier to use than a foreach loop.
The compare-Object cmdlet uses one object as a reference to find
differences in the other object. This is
an excellent tool to compare scripts when one is working and the other is not.
The
Measure-Object cmdlet calculates three types of measurements on objects. It can
count objects and calculate the minimum, maximum, sum, and average of the
numeric values. For text objects, it can count and calculate the number of
lines, words, and characters.
Make sure you are
in your documents folder.
Type: Get-ChildItem –recurse |Measure-Object –property length
–min –max –average –sum
The command looks
at the length property of all the files in the documents folder and subfolders
and calculates the minimum length, maximum length, average and sum of all
files.
Type Get-Content echoname.cmd | Measure-Object
–character –line –word
All scripters will eventually have to
compare two objects or collections to find the differences, things added or
deleted. The Compare-Object cmdlet knows
how to loop over the objects or collections, without having to write an
iterative control structure. One object
is declared a “reference set” which Compare-Object uses to identify the
“difference set”. Let’s compare the two scripts we wrote in week 2 echoname.cmd
and echoname.ps1.
Type: Compare-Object
-referenceobject $(get-content .\echoname.cmd)8
–differenceobject $(get-content
.\echoname.ps1) –includeequal
to go back in directory you can type cd ..
