Speed up my batch file parsing - parsing

I have a batch file that takes input from a txt file that looks like this..
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.
Server name lak-print01
Printer name Microsoft XPS Document Writer
Share name
Driver name Microsoft XPS Document Writer
Port name XPSPort:
Comment
Location
Print processor WinPrint
Data type RAW
Parameters
Attributes 64
Priority 1
Default priority 1
Average pages per minute 0
Printer status Idle
Extended printer status Unknown
Detected error state Unknown
Extended detected error state Unknown
Server name lak-print01
Printer name 4250_Q1
Share name 4250_Q1
Driver name Canon iR5055/iR5065 PCL5e
Port name IP_192.168.202.84
Comment Audit Department in Lakewood Operations
Location Operations Center
Print processor WinPrint
Data type RAW
Parameters
Attributes 10826
Priority 1
Default priority 0
Average pages per minute 0
Printer status Idle
Extended printer status Unknown
Detected error state Unknown
Extended detected error state Unknown
Server name lak-print01
Printer name 3130_Q1
Share name 3130_Q1
Driver name Canon iR1020/1024/1025 PCL5e
Port name IP_192.168.202.11
Comment Canon iR1025
Location Operations Center
Print processor WinPrint
Data type RAW
Parameters
Attributes 10824
Priority 1
Default priority 0
Average pages per minute 0
Printer status Idle
Extended printer status Unknown
Detected error state Unknown
Extended detected error state Unknown
and parses it to get certain things in the list, like server name, printer name, driver name, etc.. and then puts each block entry into its own comma deliminated row. So i can have multiple rows, each one for a block of text, which each column having the particular information. Some of these txt files have 100+ entries. When it gets to parsing, each file I try to parse takes 5-10 minutes
The Parse code is as follows.
:Parselak-print01
SETLOCAL enabledelayedexpansion
:: remove variables starting $
FOR /F "delims==" %%a In ('set $ 2^>Nul') DO SET "%%a="
(FOR /f "delims=" %%a IN (lak-print01.txt) DO CALL :analyse "%%a")>lak-print01.csv
attrib +h lak-print01.csv
GOTO :EOF
:analyse
SET "line=%~1"
SET /a fieldnum=0
FOR %%s IN ("Server name" "Printer name" "Driver name"
"Port name" "Location" "Comment" "Printer status"
"Extended detected error state") DO CALL :setfield %%~s
GOTO :eof
:setfield
SET /a fieldnum+=1
SET "linem=!line:*%* =!"
SET "linet=%* %linem%"
IF "%linet%" neq "%line%" GOTO :EOF
IF "%linem%"=="%line%" GOTO :EOF
SET "$%fieldnum%=%linem%"
IF NOT DEFINED $8 GOTO :EOF
SET "line="
FOR /l %%q IN (1,1,7) DO SET "line=!line!,!$%%q!"
ECHO !line:~1!
:: remove variables starting $
FOR /F "delims==" %%a In ('set $ 2^>Nul') DO SET "%%a="
GOTO :eof
and the output I get is
lak-print01,Microsoft XPS Document Writer,Microsoft XPS Document Writer,XPSPort:,,,Idle
lak-print01,4250_Q1,Canon iR5055/iR5065 PCL5e,IP_192.168.202.84,Operations Center,Audit Department in Lakewood Operations,Idle
lak-print01,3130_Q1,Canon iR1020/1024/1025 PCL5e,IP_192.168.202.11,Operations Center,Canon iR1025 ,Idle
lak-print01,1106_TRN,HP LaserJet P2050 Series PCL6,IP_172.16.10.97,Monroe,HP P2055DN,Idle
lak-print01,1101_TRN,HP LaserJet P2050 Series PCL6,IP_10.3.3.22,Burlington,Training Room printer,Idle
lak-print01,1096_Q3,Canon iR1020/1024/1025 PCL5e,IP_192.168.96.248,Silverdale,Canon iR 1025,Idle
lak-print01,1096_Q2,Kyocera Mita KM-5035 KX,IP_192.168.96.13,Silverdale,Kyocera CS-5035 all in one,Idle
lak-print01,1096_Q1,HP LaserJet P4010_P4510 Series PCL 6,IP_192.168.96.12,Silverdale,HP 4015,Idle
lak-print01,1095_Q3,HP LaserJet P4010_P4510 Series PCL 6,IP_192.168.95.247,Sequim,HP LaserJet 4015x,Idle
Everything is perfect, and the code works as intended.. but its just super freaking slow!
How do I speed this up? the problem is there is no true delim and the tokens vary.. for instance comment needs token 2, but printer name, needs token 3.
Any help to increase the speed of parsing.. the program works perfectly, but super slow during parsing.

If speed is what you need, I'd suggest Marpa, a general BNF parser, in Perl — code, output.
It would take some time to get used to, but does the job and gives you a very powerful tool you can use easily — note how natural the grammar resembles the input.
Hope this helps.

Using Call is very slow - see if this gives you the output you need, and it will be interesting to hear how much quicker it is in comparison.
#echo off
:Parselak-print01
SETLOCAL enabledelayedexpansion
(FOR /f "delims=" %%a IN (lak-print01.txt) DO (
for /f "tokens=1,2,*" %%b in ("%%a") do (
if "%%b"=="Server" set "server=%%d"
if "%%b"=="Printer" if "%%c"=="name" (set "printer=%%d") else (set "printerstatus=%%d")
if "%%b"=="Driver" set "driver=%%d"
if "%%b"=="Port" set "port=%%d"
if "%%b"=="Location" for /f "tokens=1,*" %%e in ("%%a") do set "location=%%f"
if "%%b"=="Comment" for /f "tokens=1,*" %%e in ("%%a") do set "comment=%%f"
if "%%b"=="Extended" for /f "tokens=1-4,*" %%e in ("%%a") do if "%%f"=="detected" set "extendeddetected=%%i"
)
if defined extendeddetected (
echo !server!,!printer!,!driver!,!port!,!location!,!comment!,!printerstatus!,!extendeddetected!
set "server="
set "printer="
set "driver="
set "port="
set "location="
set "comment="
set "printerstatus="
set "extendeddetected="
)
))>lak-print01.csv
attrib +h lak-print01.csv
pause

The solution below assume that the input file have a fixed format, that is, that it has two header lines followed by blocks of 18 lines placed always in the same order. If this is true, this solution generate the output in a very fast way; otherwise, it must be modified accordingly...
#echo off
setlocal EnableDelayedExpansion
rem Create the array of variable names for the *desired rows* of data in the file
set "row[1]=Server name"
set "row[2]=Printer name"
set "row[4]=Driver name"
set "row[5]=Port name"
set "row[6]=Comment"
set "row[7]=Location"
set "row[15]=Printer status"
set i=0
(for /F "skip=2 delims=" %%a in (lak-print01.txt) do (
set /A i+=1
if defined row[!i!] (
set "line=%%a"
for %%i in (!i!) do for /F "delims=" %%v in ("!row[%%i]!") do set "%%v=!line:*%%v =!"
)
if !i! equ 18 (
echo !Server name!,!Printer name!,!Driver name!,!Port name!,!Location!,!Comment!,!Printer status!
set i=0
)
)) > lak-print01.csv

Related

Batch: Don't parse comment of key-value

I am using the following code to parse an config.ini-file:
#setlocal enableextensions enabledelayedexpansion
#echo off
set file=%~1
set area=[%~2]
set key=%~3
set currarea=
for /f "usebackq delims=" %%a in ("!file!") do (
set ln=%%a
if "x!ln:~0,1!"=="x[" (
set currarea=!ln!
) else (
for /f "tokens=1,2 delims==" %%b in ("!ln!") do (
set currkey=%%b
set currval=%%c
if "x!area!"=="x!currarea!" if "x!key!"=="x!currkey!" (
echo !currval!
)
)
)
)
endlocal
It works fine as long as there are no comments in the same line as they keys and values.
For example:
[BACKUP]
HOST=128.110.111.11 ;Comment is included in !currval!
PORT=5901
USER=user1
Unfortunately I can't find a way to exclude everything after the last character of the string "128.110.111.11"..
Any help is appreciated. Thanks!
The best batch has to offer towards achieving the goal is a combination of for loops to process the string. There is no innate command that can achieve this in a single step.
In a way though, you can make a command to complete the necessary set of commands by a assigning them to a variable as a macro
For example, in the below script the macro completes the necessary steps for this goal by:
Delimitng the Variables content using ;
Iterating over the length of the string from end to start - The example assumes a maximum string length of 250 characters; an arbitrary number for the point of the example.
Remove only trailing spaces using If condition logic and substring modification
Stop further modification of the variables content by using a true/false switch to flag that the last digit of the string contains a non-space character
Note : Substring modification is used at the point of the the macros expansion to supply the name of the variable to be processed.
#Echo off & Setlocal enableDelayedexpansion
Set "RemTrail=Set "end=0"&(For /F "Tokens=1 Delims=;" %%G in ("^^!$v^^!")Do Set "$V=%%G")&For /L %%i in (250,-1,0)Do (if "^^!$V:~%%i,1^^!"==" " (If not "^^!End^^!"=="1" Set "$V=^^!$V:~0,%%i^^!")Else (If not "^^!$V:~%%i,1^^!"=="" Set "End=1"))"
rem // usage example
Set "string=trail of spaces ; comment string "
Set "string2=uncommented string with trailing spaces and poison chars < & " " | * > "
Echo/[!string!]
Echo/[!string2!]
%RemTrail:$V=String%
%RemTrail:$V=String2%
Echo/[!string!]
Echo/[!string2!]
A slighty modified version that Allows the Delimiter to be modified at expansion, at the expense of returning the modified result in a fixed return variable ($V) instead of the original variable name:
#Echo off & Setlocal enableDelayedexpansion
Set "RemTrail=For %%n in (1 2)Do if %%n==2 (Set "end=0"&(For /F "Tokens=1 Delims=DLM" %%G in ("^^!$V^^!")Do Set "$V=%%~G")&For /L %%i in (250,-1,0)Do (if "^^!$V:~%%i,1^^!"==" " (If not "^^!End^^!"=="1" Set "$V=^^!$V:~0,%%i^^!")Else (If not "^^!$V:~%%i,1^^!"=="" Set "End=1")))Else Set $V="
rem // usage example
Set "string=trail of spaces ; comment string "
Set "string2=uncommented string with trailing spaces + poison chars < & | * " " > "
Echo/[!string!]
Echo/[!string2!]
%RemTrail:DLM=;%"!string!"
Echo/[!$V!]
%RemTrail:DLM=;%"!string2!"
Echo/[!$V!]
%RemTrail:DLM=+%"!string2!"
Echo/+ Delim example&Echo/[!$V!]
Solution 1:
for /f "usebackq delims=;" %%a in ("!file!") do (
Including the semicolon as a delimiter ensures only that part of the line up to, but not including the semicolon is assigned to token 1 which is assigned by default to the metavariable %%a
Disadvantage : the spaces between the end of the string and the semicolon are retained in %%a and hence currval and will be echoed.
Solution 2 :
for /f "tokens=1,2 delims== " %%b in ("!ln!") do (
Inclusion of the space as an extra delimiter will assign the value between the first delimiter found (=) and the second ([space]) to %%c.
Disadvantage : The value displayed as %%c will be truncated at the space
Solution 3:
Use solution 1 and then change
set currval=%%c
to
CALL :setcurrval %%c
AND then endlocal becomes
endlocal
goto :eof
:setcurrval
SET "currval=%*"
goto :eof
which terminates the local environment and exits the batch.
CALLing the internal subroutine :setcurrval assigns the value of the remainder of the line to currval, except for the terminal spaces, hence producing %%c minus any terminal spaces.
Note that the colons are significant.

Batch script : Parse file and get particular line number substring

I have a requirement for processing some files and renaming it. I am planning to use a batch process to execute it.
The requirement is as follows
Read the files in a particular directory
Open all the TXT files in it.
Navigate to a particular line in that file.
Parse that line to get a particular string.
Use that particular string to rename the filename (or maybe copy to
another folder )
The line number that I need to extract is 12 and
the line has the following string Product Number # 773339.
What I want to extract from that line is 773339 and rename or create a new file as 773339.TXT with the same contents as original.
Below is the steps that I got so far, but not able to extract the substring yet
#ECHO off
SETLOCAL enabledelayedexpansion
FOR %%f IN (C:\Users\test\Desktop\AR\*.txt) DO (
For /F "Tokens=1-2* Delims=:" %%a IN ('FindStr/N "^" "%%f" 2^>Nul') DO (
If %%a Equ 12 (
REM Get the line number here
Echo %%a
REM Get the string from that line number
ECHO %%b
)
)
PAUSE
)

Setting a Parsed value as variable in a batch script

So I'm having a difficult time with this, I am trying to parse a txt file for a specific value and add it to a variable, but just not getting it to work correctly I am using the following code
for /F "tokens=5 delims= " %%a in ('findstr /I "Short part of line 00:00" test.txt') do set "var1=%%a"
The problem is that there are other similar lines of text that vary a slight amount, I'm including a phrase that is unique when considered in whole, but I don't think the command is taking that into consideration? the file is always the same in regards to size, is there anyway to re write this to find a token on a specific line and assign to a variable instead?
EDIT:
input.txt
Process 1
Primary process Started 2:50:00
Primary Last part process 3:40:52
Primary part 3 Balanced 5:01:55
Primary CMD Backup Completed 5:04:13
Primary Process Balance Completed 5:04:13
Process 2
Secondary process Started 2:50:00
Secondary Last part process 3:42:08
Secondary part 3 Balanced 5:06:38
Secondary CMD Backup Completed 5:08:56
Secondary Process Balance Completed 5:08:56
So if I were to use the command:
for /F "tokens=5 delims= " %%a in ('findstr /I "Primary process Started" test.txt') do set "var1=%%a"
I should read back 2:50:00 instead I'm getting 5:08:56 thats the weird one, the rest are simply flipped so "Primary part 3 Balanced" will give me back the time for "Secondary part 3 Balanced"
According to your comments this should work:
#echo off
setlocal enabledelayedexpansion
for /f "tokens=*" %%f in (input.txt) do (
set line=%%f
if "!line:~0,23!"=="Primary process Started" (
for /f "tokens=4" %%t in ("%%f") do set var1=%%t
goto BREAK
)
)
:BREAK
echo %var1%
Simplifying #MichaelS code a little bit.
#echo off
for /f "tokens=1-4 delims= " %%G in (input.txt) do (
if "%%G %%H %%I"=="Primary process Started" (
set var1=%%J
GOTO BREAK
)
)
:BREAK
echo %var1%

Need help writing a batch file to read a MS Access .ldb lock file with null delimiters

I am trying to create a batch file to read a Microsoft Access .ldb lock file. The lock file contains a list of computer names and user names. I want to extract the computer names and eventually run them against an external command.
The format of the batch file is a single row with
(1) a computer name
(2) a NULL character (Hex 00)
(3) approximately 20 spaces
(4) the user name
(5) a NULL character
(6) approximately 20 spaces
repeating.
Example in Notepad++ with (NUL) representing Hex 00:
COMPUTER0123(NUL) Admin(NUL) COMPUTER0507(NUL) Admin(NUL)
I've tried several methods using FOR to read the file but can't get past the first computer name.
setlocal EnableDelayedExpansion
set file=database.ldb
for /F %%a in ('type %file%') do (
echo %%a
)
For for most of my Access databases, the user name in the file is Admin. I've been able to use FIND to tell me how many occurrences of "Admin" are in the file (plus 1).
for /f "delims=" %%n in ('find /c /v "Admin" %file%') do set "len=%%n"
set "len=!len:*:=!"
echo %len% (minus 1) computer names to process
<%file% (
for /l %%l in (1 1 !len!) do (
set "line="
set /p "line="
echo(!line!)
)
)
Iterating through the found lines doesn't work, probably because there only is one line in the file (no carriage returns).
I would like to find a solution that would work with a standard install of Windows XP.
After receiving an accepted answer, I combined that into a batch file that I'm posting below. I named the file ShowUsersInLDB.bat and put it in my SendTo folder.
#echo off
::===================================================================
:: Put this in your SendTo folder and it will let you right-click
:: on an Access .ldb/.laccdb lock file and tell you the computer
:: names that have opened the database.
::
:: After the computer names are shown, this will prompt you to
:: search for the user names associated with each computer. This
:: depends upon finding a 3rd party file named NetUsers.exe in
:: the user profile folder. Feel free to change the path if you
:: want to store the file in another location.
::
:: NetUsers.exe can be downloaded from here: http://www.optimumx.com/downloads.html#NetUsers
::
:: Notes:
:: 1) Keep in mind that sometimes after people leave the database
:: the lock file still shows their computer name. Don't jump
:: to conclusions.
:: 2) NetUsers.exe seems to report all users who have logged on
:: to the computer and not logged off, including services.
:: If you aren't familiar with your user names or your users are
:: sharing remote desktops/Citrix/Terminal Services, you may have
:: to guess who might have created the lock entry.
::
:: Installation:
:: You may find a batch file named Install_UsersInLDB.bat that will
:: copy this file to the SendTo folder and the NetUsers.exe file to
:: the user profile (or a place you define).
::
:: Ben Sacherich - March 2014
:: Please let me know if you have any ideas for improvements.
::===================================================================
setlocal
set file="%1"
:: Make sure the file has a compatible extension.
if "%~x1"==".ldb" goto :ExtensionIsValid
if "%~x1"==".laccdb" goto :ExtensionIsValid
echo.
echo "%~n1%~x1" is not the correct file type.
echo.
pause
goto :End
:ExtensionIsValid
echo The Access "%~n1%~x1" file contains
echo the following computer names:
echo.
set "compNameLine=1"
for /f %%A in ('more "%file%"') do (
if defined compNameLine (
echo %%A
set "compNameLine="
) else set "compNameLine=1"
)
echo.
echo Are you ready to look up the user names on each computer?
pause
set "compNameLine=1"
for /f %%A in ('more "%file%"') do (
if defined compNameLine (
::echo %%A
"%userprofile%\netusers" \\%%A
set "compNameLine="
) else set "compNameLine=1"
)
echo.
echo -- Validation finished at %time%
pause
:End
exit
CMD.EXE generally does not play nicely with NUL bytes. But there are a few external commands that can handle NUL bytes.
You also have to worry about the length of the "line". CMD.EXE does not like lines longer than 8191 bytes long.
I think your best bet is MORE since it converts NULs into new lines.
The following should echo your computer names.
#echo off
setlocal
set "file=database.ldb"
set "compNameLine=1"
for /f %%A in ('more "%file%"') do (
if defined compNameLine (
echo %%A
set "compNameLine="
) else set "compNameLine=1"
)

Extracting certain characters from the last line of text file using a .bat file

What I'm trying to accomplish here is to pull data from the last line of this file ftp://ftp.nhc.noaa.gov/atcf/tcweb/invest_al902012.invest. I've managed to download it and save it as a script.txt file through a .bat file. I now want to extract the latitude(13.5N) and longitude(27.2W) as well as pressure(1009) from the last line of the file and write it to a new file.I then used this code to do part of what I want:
#echo off
setlocal EnableDelayedExpansion
for /f "delims=" %%x in (script.txt) do (
set "previous=!last!"
set "last=%%x"
)
echo !previous!>> "test3.txt"
for /f "delims=*" %%x in (test3.txt) do (
set line=%%x
set chars=!line:~35,-125!
echo !chars!>> "test.txt"
)
I'm illiterate when it comes to batch coding. This is probably extremely inefficient and only extracts the latitude part of the code I want. The file will always contain the same amount of characters in the last line so I'm thinking I'm just not grasping the concept of the !line part of the code. Any help is greatly appreciated.
The file is comma delimited, so it is probably easier to let FOR /F parse the line into tokens and keep just the ones you want.
This really simple solution parses and sets values for each line, but only the last line is remembered. The performance should be fine as long as the file never becomes huge.
#echo off
for /f "tokens=7,8,10 delims=," %%A in (script.txt) do (
set lat=%%A
set long=%%B
set pres=%%C
)
echo latitude=%lat%, longitude=%long%, pressure=%pres%
If you want to strip off the spaces, then you could simply use search and replace.
echo latitude=%lat: =%, longitude=%long: =%, pressure=%pres: =%
I do not simply include space as a delimiter in the FOR /F statement because that can throw off the token counting when a value is sometimes blank and sometimes not.

Resources