insert numerical sequence in large text file - editor

I need to create a file in this format :
item0000001
item0000002
item0000003
item0000004
item0000005
I was doing this with UltraEdit which has column mode including insert number ( start + increment including leading zeros ).
Unfortunately, UltraEdit bombs out above 1 million rows.
Does anyone know of a text editor with large file capacity that has a similar operation?

BaltoStar has not written which version of UltraEdit was used and how exactly he has tried to create the file.
However, here is an UltraEdit script which can be used to create a file with lines containing an incrementing number with leading zeros according to last number.
To use that script with UltraEdit v14.20 or UEStudio v9.00 or any later version, copy the code block of the script and paste it into a new ASCII file with DOS line terminations in UltraEdit/UEStudio. Save the file for example as CreateLinesWithIncrementingNumber.js into your preferred directory of UE/UES scripts.
Now run the script by clicking on menu item Run Active Script in menu Scripting.
The script prompts the user for first and last value of the incrementing number, and for strings left and right of the incrementing number which can be both also empty strings.
Then lean back and see how the script writes the lines with the incrementing number into a new file in blocks. I created a file with more than 150 MB with an incrementing number from 0 to 5.000.000 within a few seconds using this UltraEdit script.
if (typeof(UltraEdit.clipboardContent) == "string")
{
// Write in blocks of not more than 4 MB into the file. Do not increase
// this value too much as during the script execution much more free
// RAM in a continous block is necessary than the value used here for
// joining the lines in user clipboard 9. A too large value results
// in a memory exception during script execution and the user of the
// script also does not see for several seconds what is going on.
var nBlockSize = 4194304;
// Create a new file and make sure it uses DOS/Windows line terminations
// independent on the user configuration for line endings of new files.
UltraEdit.newFile();
UltraEdit.activeDocument.unixMacToDos();
var sLineTerm = "\r\n"; // Type of line termination is DOS/Windows.
// Ask user of script for the first value to write into the file.
do
{
var nFirstNumber = UltraEdit.getValue("Please enter first value of incrementing number:",1);
if (nFirstNumber < 0)
{
UltraEdit.messageBox("Sorry, but first value cannot be negative.");
}
}
while (nFirstNumber < 0);
// Ask user of script for the last value to write into the file.
do
{
var nLastNumber = UltraEdit.getValue("Please enter last value of incrementing number:",1);
if (nFirstNumber >= nLastNumber)
{
UltraEdit.messageBox("Sorry, but last value must be greater than "+nFirstNumber.toString(10)+".");
}
}
while (nFirstNumber >= nLastNumber);
var sBeforeNumber = UltraEdit.getString("Please enter string left of the incrementing number:",1);
var sAfterNumber = UltraEdit.getString("Please enter string right of the incrementing number:",1);
// http://stackoverflow.com/questions/16378849/ultraedit-how-do-i-pad-out-a-string-with-leading-blanks
// Convert the highest number to a decimal string and get a copy
// of this string with every character replaced by character '0'.
// With last number being 39428 the created string is "00000".
var sLeadingZeros = nLastNumber.toString(10).replace(/./g,"0");
// Instead of writing the string with the incrementing number line
// by line to file which would be very slow and which would create
// lots of undo records, the lines are collected first in an array of
// strings whereby the number of strings in the array is determined
// by value of variable nBlockSize. The lines in the array are
// concatenated into user clipboard 9 and written as block to the
// file using paste command. That is much faster and produces just
// a few undo records even on very large files.
UltraEdit.selectClipboard(9);
UltraEdit.clearClipboard();
// Calculate number of lines per block which depends on the
// lengths of the 4 strings which build a line in the file.
var nLineLength = sBeforeNumber.length + sLeadingZeros.length +
sAfterNumber.length + sLineTerm.length;
var nRemainder = nBlockSize % nLineLength;
var nLinesPerBlock = (nBlockSize - nRemainder) / nLineLength;
var asLines = [];
var nCurrentNumber = nFirstNumber;
while (nLastNumber >= nCurrentNumber)
{
// Convert integer number to decimal string.
var sNumber = nCurrentNumber.toString(10);
// Has the decimal string of the current number less
// characters than the decimal string of the last number?
if (sNumber.length < sLeadingZeros.length)
{
// Build decimal string new with X zeros from the alignment string
// and concatenate this leading zero string with the number string.
sNumber = sLeadingZeros.substr(0,sLeadingZeros.length-sNumber.length) + sNumber;
}
asLines.push(sBeforeNumber + sNumber + sAfterNumber);
if (asLines.length >= nLinesPerBlock)
{
asLines.push(""); // Results in a line termination at block end.
UltraEdit.clipboardContent = asLines.join(sLineTerm);
UltraEdit.activeDocument.paste();
UltraEdit.clearClipboard();
asLines = [];
}
nCurrentNumber++;
}
// Output also the last block.
if (asLines.length)
{
asLines.push("");
UltraEdit.clipboardContent = asLines.join(sLineTerm);
UltraEdit.activeDocument.paste();
UltraEdit.clearClipboard();
}
// Reselect the system clipboard and move caret to top of new file.
UltraEdit.selectClipboard(0);
UltraEdit.activeDocument.top();
}
else if(UltraEdit.messageBox)
{
UltraEdit.messageBox("Sorry, but you need a newer version of UltraEdit/UEStudio for this script.");
}
else
{
UltraEdit.newFile();
UltraEdit.activeDocument.write("Sorry, but you need a newer version of UltraEdit/UEStudio for this script.");
}

Related

How to update another byte of a CAN message while changing a byte in real-time?

I'm new to CANoe and CAPL scripting. I need to update a byte of CAN message based on a change in another byte through a panel. I added a CAPL script for the automatic change/update but when I send/update the byte through the panel it sends the default values for the other bytes. It gets corrected in the next sample.
includes
{
}
variables
{
message M2P_0x101 msg1;
msTimer timer10ms;
byte counter = 1;
}
on message M2P_0x101
{
if(counterAlive == 15)
{
counterAlive = 0;
}
else
{
counter++;
}
msg1.byte(0) = this.byte(0);
msg1.byte(1) = this.byte(1);
msg1.byte(2) = this.byte(2);
msg1.byte(3) = this.byte(3); // Changing through the Panel
msg1.byte(4) = this.byte(3)+1;
msg1.byte(5) = this.byte(5);
msg1.byte(6) = counter;
msg1.byte(7) = this.byte(7);
}
on timer timer10ms
{
output(msg1);
}
on start
{
setTimerCyclic(timer10ms, 10);
}
4th & 6th byte has default value because 3rd byte was changed from the panel.
I expected the 4th byte to be 1(value of 3rd byte + 1) and the 6th byte to be 4(counter value).
How to solve it? As far as I understand, it seems that the panel inputs/changes are sent to the CAN bus first and then CAPL node reads it and makes changes. This could be the reason for overwriting the other bytes of the message with default values.
Is there a way to include the changes made by CAPL before the panel sends the message to the CAN bus? A combination of Panel & CAPL node?

Unexpected error "incorrect use of identifier" when running DXL code

I have a script which calls functions from this script:
https://www.baselinesinc.com/dxl-repository/?filerepoaction=showpost&filepost_id=13
which is distributed under the LGPL. I have found several references to this script online, indicating that it works. For my own purposes, I removed the part of the code which executes on load (some 20 lines at the bottom of the file) and converted the file to an .inc, to serve as a library. I haven't done any other changes.
My own script has worked for a few months at this point. But now, all of a sudden, when I try to run it, I get these errors:
-E- DXL: <DOORSImporter\Import_From_Excel.inc:291> incorrect use of identifier (rtfValue)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:292> incorrect use of identifier (nonRTFValue)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:346> incorrect use of identifier (temp)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:464> incorrect use of identifier (columnHeading)
-E- DXL: <DOORSImporter\Import_From_Excel.inc:1127> incorrect use of identifier (cellValue)
-I- DXL: All done. Errors reported: 5. Warnings reported: 0.
Below is a snippet of code which shows this error. I marked lines 291 and 292
bool richTextTruncated(int rowNumber, int columnNumber) {
Buffer rtfValue = create
Buffer nonRTFValue = create
rtfValue = ""
nonRTFValue = ""
rtfValue = copyRichTextFromCell(""intToCol(columnNumber) "" rowNumber"") // get the rich text
rtfValue = trimWhitespace(stripRichText(tempStringOf(rtfValue))) // strip off RTF formatting. <----- 291
nonRTFValue = trimWhitespace(getCellValue(rowNumber, columnNumber)) //<----- 292
if(tempStringOf(rtfValue) != tempStringOf(nonRTFValue)) { // if the two strings aren't equal
delete(rtfValue)
delete(nonRTFValue)
return true // return that text was truncated
}
delete(rtfValue)
delete(nonRTFValue)
return false
}
The function trimWhitespace itself is defined as
string trimWhitespace(string s) so it's OK to assign its output to a buffer.
So this is all very confusing because, like I said, this has worked and the code has not been changed (last check in of the file DOORSImporter\Import_From_Excel.inc is from last November, whereas I have used the script even in January.
Our IT department might have updated DOORS recently, I don't know. I checked the latest version of the dxl reference manual for any recent changes which might have caused this, but I couldn't find any change to the way buffers are handled.
To be fair, I have found working with buffers quite confusing. In my own code, I ended up discarding them altogether. However, the script Import_From_Excel has worked unchanged for a long time now.
Any support with this will be greatly appreciated.
********** Edit 27.02.2020 **************
The function copyRichTextFromCell is defined as below. Its sourced is from the link below.
https://www.baselinesinc.com/dxl-repository/?filerepoaction=showpost&filepost_id=10
string copyRichTextFromCell(string loc) {
if(!getCell(loc)) {
return("error");
}
if(!checkResult(oleMethod(objCell, cMethodSelect))) { // selects the cell
return("error");
}
if(!checkResult(oleMethod(objCell, cMethodCopy))) { // copies the contents of the cell to the clipboard
return("error");
}
return stringOf(richClip)
}
********** Edit 28.02.2020 **************
The function trimWhitespace is as below.
string trimWhitespace(string s) {
int first = 0;
int last = length(s) - 1;
while(last > 0 && (isspace(s[last]) || s[last] == '\n' || s[last] == '\r' || s[last] == '\t')) {
last--;
}
while(isspace(s[first]) && first < last) {
first++;
}
if(s[first:last] == " ") {
return("");
}
return(s[first:last]);
}
While playing around, I discovered that if I modify as below, one of the errors disappears. It is beyond me as to why that might happen, because the code is equivalent, as far as I can tell...
bool richTextTruncated(int rowNumber, int columnNumber) {
Buffer rtfValue = create
Buffer nonRTFValue = create
string fooString
rtfValue = ""
nonRTFValue = ""
rtfValue = copyRichTextFromCell(""intToCol(columnNumber) "" rowNumber"") // get the rich text
fooString = trimWhitespace(stripRichText(tempStringOf(rtfValue)))
rtfValue = fooString // strip off RTF formatting.
fooString = trimWhitespace(getCellValue(rowNumber, columnNumber))
nonRTFValue = fooString
if(tempStringOf(rtfValue) != tempStringOf(nonRTFValue)) { // if the two strings aren't equal
delete(rtfValue)
delete(nonRTFValue)
return true // return that text was truncated
}
delete(rtfValue)
delete(nonRTFValue)
return false
}
So the lines with fooString do not throw an error anymore. In the initial version, these lines were just e.g. rtfValue = trimWhitespace(stripRichText(tempStringOf(rtfValue))) and this would throw an error.

Avoiding excessive memory allocation in golang when using an io.Writer

I am working on a command line tool in Go called redis-mass that converts a bunch of redis commands into redis protocol format.
The first step was to port the node.js version, almost literally to Go. I used ioutil.ReadFile(inputFileName) to get a string version of the file and then returned an encoded string as output.
When I ran this on a file with 2,000,000 redis commands, it took about 8 seconds, compared to about 16 seconds with the node version. I guessed that the reason it was only twice as fast was because it was reading the whole file into memory first, so I changed my encoding function to accept a pair (raw io.Reader, enc io.Writer), and it looks like this:
func EncodeStream(raw io.Reader, enc io.Writer) {
var args []string
var length int
scanner := bufio.NewScanner(raw)
for scanner.Scan() {
command := strings.TrimSpace(scanner.Text())
args = parse(command)
length = len(args)
if length > 0 {
io.WriteString(enc, fmt.Sprintf("*%d\r\n", length))
for _, arg := range args {
io.WriteString(enc, fmt.Sprintf("$%d\r\n%s\r\n", len(arg), arg))
}
}
}
}
However, this took 12 seconds on the 2 million line file, so I used github.com/pkg/profile to see how it was using memory, and it looks like the number of memory allocations is huge:
# Alloc = 3162912
# TotalAlloc = 1248612816
# Mallocs = 46001048
# HeapAlloc = 3162912
Can I constrain the io.Writer to use a fixed sized buffer and avoid all those allocations?
More generally, how can I avoid excessive allocations in this method? Here's the full source for more context
Reduce allocations by working with []byte instead of strings. fmt.Printf directly to the output instead of fmt.Sprintf and io.WriteString.
func EncodeStream(raw io.Reader, enc io.Writer) {
var args []string
var length int
scanner := bufio.NewScanner(raw)
for scanner.Scan() {
command := bytes.TrimSpace(scanner.Bytes())
args = parse(command)
length = len(args)
if length > 0 {
fmt.Printf(enc, "*%d\r\n", length))
for _, arg := range args {
fmt.Printf(enc, "$%d\r\n%s\r\n", len(arg), arg))
}
}
}
}

Zebra Printer - Not Printing PNG Stream *Provided my own answer*

I think I'm very close to getting this to print. However it still isn't. There is no exception thrown and it does seem to be hitting the zebra printer, but nothing. Its a long shot as I think most people are in the same position I am and know little about it. Any help anyone can give no matter how small will be welcomed, I'm losing the will to live
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var stream = new MemoryStream())
{
if (responseStream == null)
{
return;
}
responseStream.CopyTo(stream);
stream.Position = 0;
using (var zipout = ZipFile.Read(stream))
{
using (var ms = new MemoryStream())
{
foreach (var e in zipout.Where(e => e.FileName.Contains(".png")))
{
e.Extract(ms);
}
if (ms.Length <= 0)
{
return;
}
var binaryData = ms.ToArray();
byte[] compressedFileData;
// Compress the data using the LZ77 algorithm.
using (var outStream = new MemoryStream())
{
using (var compress = new DeflateStream(outStream, CompressionMode.Compress, true))
{
compress.Write(binaryData, 0, binaryData.Length);
compress.Flush();
compress.Close();
}
compressedFileData = outStream.ToArray();
}
// Encode the compressed data using the MIME Base64 algorithm.
var base64 = Convert.ToBase64String(compressedFileData);
// Calculate a CRC across the encoded data.
var crc = Calc(Convert.FromBase64String(base64));
// Add a unique header to differentiate the new format from the existing ASCII hexadecimal encoding.
var finalData = string.Format(":Z64:{0}:{1}", base64, crc);
var zplToSend = "~DYR:LOGO,P,P," + finalData.Length + ",," + finalData;
const string PrintImage = "^XA^FO0,0^IMR:LOGO.PNG^FS^XZ";
try
{
var client = new System.Net.Sockets.TcpClient();
client.Connect(IpAddress, Port);
var writer = new StreamWriter(client.GetStream(), Encoding.UTF8);
writer.Write(zplToSend);
writer.Flush();
writer.Write(PrintImage);
writer.Close();
client.Close();
}
catch (Exception ex)
{
// Catch Exception
}
}
}
}
}
}
private static ushort Calc(byte[] data)
{
ushort wCrc = 0;
for (var i = 0; i < data.Length; i++)
{
wCrc ^= (ushort)(data[i] << 8);
for (var j = 0; j < 8; j++)
{
if ((wCrc & 0x8000) != 0)
{
wCrc = (ushort)((wCrc << 1) ^ 0x1021);
}
else
{
wCrc <<= 1;
}
}
}
return wCrc;
}
The following code is working for me. The issue was the commands, these are very very important! Overview of the command I have used below, more can be found here
PrintImage
^XA
Start Format Description The ^XA command is used at the beginning of ZPL II code. It is the opening bracket and indicates the start of a new label format. This command is substituted with a single ASCII control character STX (control-B, hexadecimal 02). Format ^XA Comments Valid ZPL II format requires that label formats should start with the ^XA command and end with the ^XZ command.
^FO
Field Origin Description The ^FO command sets a field origin, relative to the label home (^LH) position. ^FO sets the upper-left corner of the field area by defining points along the x-axis and y-axis independent of the rotation. Format ^FOx,y,z
x = x-axis location (in dots) Accepted Values: 0 to 32000 Default
Value: 0
y = y-axis location (in dots) Accepted Values: 0 to 32000
Default Value: 0
z = justification The z parameter is only
supported in firmware versions V60.14.x, V50.14.x, or later. Accepted
Values: 0 = left justification 1 = right justification 2 = auto
justification (script dependent) Default Value: last accepted ^FW
value or ^FW default
^IM
Image Move Description The ^IM command performs a direct move of an image from storage area into the bitmap. The command is identical to the ^XG command (Recall Graphic), except there are no sizing parameters. Format ^IMd:o.x
d = location of stored object Accepted Values: R:, E:, B:, and A: Default Value: search priority
o = object name Accepted Values: 1 to 8 alphanumeric characters Default Value: if a name is not specified, UNKNOWN is used
x = extension Fixed Value: .GRF, .PNG
^FS
Field Separator Description The ^FS command denotes the end of the field definition. Alternatively, ^FS command can also be issued as a single ASCII control code SI (Control-O, hexadecimal 0F). Format ^FS
^XZ
End Format Description The ^XZ command is the ending (closing) bracket. It indicates the end of a label format. When this command is received, a label prints. This command can also be issued as a single ASCII control character ETX (Control-C, hexadecimal 03). Format ^XZ Comments Label formats must start with the ^XA command and end with the ^XZ command to be in valid ZPL II format.
zplToSend
^MN
Media Tracking Description This command specifies the media type being used and the black mark offset in dots. This bulleted list shows the types of media associated with this command:
Continuous Media – this media has no physical characteristic (such as a web, notch, perforation, black mark) to separate labels. Label length is determined by the ^LL command.
Continuous Media, variable length – same as Continuous Media, but if portions of the printed label fall outside of the defined label length, the label size will automatically be extended to contain them. This label length extension applies only to the current label. Note that ^MNV still requires the use of the ^LL command to define the initial desired label length.
Non-continuous Media – this media has some type of physical characteristic (such as web, notch, perforation, black mark) to separate the labels.
Format ^MNa,b
a = media being used Accepted Values: N = continuous media Y = non-continuous media web sensing d, e W = non-continuous media web sensing d, e M = non-continuous media mark sensing A = auto-detects the type of media during calibration d, f V = continuous media, variable length g Default Value: a value must be entered or the command is ignored
b = black mark offset in dots This sets the expected location of the media mark relative to the point of separation between documents. If set to 0, the media mark is expected to be found at the point of separation. (i.e., the perforation, cut point, etc.) All values are listed in dots. This parameter is ignored unless the a parameter is set to M. If this parameter is missing, the default value is used. Accepted Values: -80 to 283 for direct-thermal only printers -240 to 566 for 600 dpi printers -75 to 283 for KR403 printers -120 to 283 for all other printers Default Value: 0
~DY
Download Objects Description The ~DY command downloads to the printer graphic objects or fonts in any supported format. This command can be used in place of ~DG for more saving and loading options. ~DY is the preferred command to download TrueType fonts on printers with firmware later than X.13. It is faster than ~DU. The ~DY command also supports downloading wireless certificate files. Format ~DYd:f,b,x,t,w,data
Note
When using certificate files, your printer supports:
- Using Privacy Enhanced Mail (PEM) formatted certificate files.
- Using the client certificate and private key as two files, each downloaded separately.
- Using exportable PAC files for EAP-FAST.
- Zebra recommends using Linear sty
d = file location .NRD and .PAC files reside on E: in firmware versions V60.15.x, V50.15.x, or later. Accepted Values: R:, E:, B:, and A: Default Value: R:
f = file name Accepted Values: 1 to 8 alphanumeric characters Default Value: if a name is not specified, UNKNOWN is used
b = format downloaded in data field .TTE and .TTF are only supported in firmware versions V60.14.x, V50.14.x, or later. Accepted Values: A = uncompressed (ZB64, ASCII) B = uncompressed (.TTE, .TTF, binary) C = AR-compressed (used only by Zebra’s BAR-ONE® v5) P = portable network graphic (.PNG) - ZB64 encoded Default Value: a value must be specified
clearDownLabel
^ID
Description The ^ID command deletes objects, graphics, fonts, and stored formats from storage areas. Objects can be deleted selectively or in groups. This command can be used within a printing format to delete objects before saving new ones, or in a stand-alone format to delete objects.
The image name and extension support the use of the asterisk (*) as a wild card. This allows you to easily delete a selected groups of objects. Format ^IDd:o.x
d = location of stored object Accepted Values: R:, E:, B:, and A: Default Value: R:
o = object name Accepted Values: any 1 to 8 character name Default Value: if a name is not specified, UNKNOWN is used
x = extension Accepted Values: any extension conforming to Zebra conventions
Default Value: .GRF
const string PrintImage = "^XA^FO0,0,0^IME:LOGO.PNG^FS^XZ";
var zplImageData = string.Empty;
using (var response = request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var stream = new MemoryStream())
{
if (responseStream == null)
{
return;
}
responseStream.CopyTo(stream);
stream.Position = 0;
using (var zipout = ZipFile.Read(stream))
{
using (var ms = new MemoryStream())
{
foreach (var e in zipout.Where(e => e.FileName.Contains(".png")))
{
e.Extract(ms);
}
if (ms.Length <= 0)
{
return;
}
var binaryData = ms.ToArray();
foreach (var b in binaryData)
{
var hexRep = string.Format("{0:X}", b);
if (hexRep.Length == 1)
{
hexRep = "0" + hexRep;
}
zplImageData += hexRep;
}
var zplToSend = "^XA" + "^FO0,0,0" + "^MNN" + "~DYE:LOGO,P,P," + binaryData.Length + ",," + zplImageData + "^XZ";
var label = GenerateStreamFromString(zplToSend);
var client = new System.Net.Sockets.TcpClient();
client.Connect(IpAddress, Port);
label.CopyTo(client.GetStream());
label.Flush();
client.Close();
var cmd = GenerateStreamFromString(PrintImage);
var client2 = new System.Net.Sockets.TcpClient();
client2.Connect(IpAddress, Port);
cmd.CopyTo(client2.GetStream());
cmd.Flush();
client2.Close();var clearDownLabel = GenerateStreamFromString("^XA^IDR:LOGO.PNG^FS^XZ");
var client3 = new System.Net.Sockets.TcpClient();
client3.Connect(IpAddress, Port);
clearDownLabel.CopyTo(client3.GetStream());
clearDownLabel.Flush();
client3.Close();
}
}
}
}
}
}
Easy once you know how.
Zebra ZPL logo example in base64
Python3
import crcmod
import base64
crc16 = crcmod.predefined.mkCrcFun('xmodem')
s = hex(crc16(ZPL_LOGO.encode()))[2:]
print (f"crc16: {s}")
Poorly documented may I say the least

How can I efficiently parse formatted text from a file in Qt?

I would like to get efficient way of working with Strings in Qt. Since I am new in Qt environment.
So What I am doing:
I am loading a text file, and getting each lines.
Each line has text with comma separated.
Line schema:
Fname{limit:list:option}, Lname{limit:list:option} ... etc.
Example:
John{0:0:0}, Lname{0:0:0}
Notes:limit can be 1 or 0 and the same as others.
So I would like to get Fname and get limit,list,option values from {}.
I am thinking to write a code with find { and takes what is inside, by reading symbol by symbol.
What is the efficient way to parse that?
Thanks.
The following snippet will give you Fname and limit,list,option from the first set of brackets. It could be easily updated if you are interested in the Lname set as well.
QFile file("input.txt");
if (!file.open(QIODevice::ReadOnly | QIODevice::Text))
qDebug() << "Failed to open input file.";
QRegularExpression re("(?<name>\\w+)\\{(?<limit>[0-1]):(?<list>[0-1]):(?<option>[0-1])}");
while (!file.atEnd())
{
QString line = file.readLine();
QRegularExpressionMatch match = re.match(line);
QString name = match.captured("name");
int limit = match.captured("limit").toInt();
int list = match.captured("list").toInt();
int option = match.captured("option").toInt();
// Do something with values ...
}

Resources