Is there any way to merge fields without adding the doneMatch into the field? - azure-devops-extensions

We need to merge two fields into one. In the config, there is a "doneMatch" special string, and this seems to get appended to the merged field. Why is this needed, and is there a way to not have it also appended to the target field?
For example, I have:
src.fieldA = "City"
src.fieldB = "State"
I want to merge these 2 fields into target.fieldA as "City: State". However, what I end up with is "City: State##DONE##" I can change the config file so that it uses a different doneMatch but it can't be null or empty.. so if I changed it to ";", then the resulting field would be "City: State;". I have to have an end done char/string for some reason. What is this used for? If I am synching the fields with newer updates, is it going to detect the previous ##DONE## in the target.fieldA and think it's already done the merge, so would not make any new changes?
Can someone send me more info on this feature?

I have updated the code for v9.0.1 that changes the way that the FieldMerge works. It no longer uses doneMatch and instead requires that all 3 fields are different and then skips if it has already been done.
if (source.Fields.Contains(config.sourceField1) && source.Fields.Contains(config.sourceField2))
{
var val1 = source.Fields[config.sourceField1].Value != null ? source.Fields[config.sourceField1].Value.ToString() : string.Empty;
var val2 = source.Fields[config.sourceField2].Value != null ? source.Fields[config.sourceField2].Value.ToString() : string.Empty;
var valT = target.Fields[config.targetField].Value != null ? target.Fields[config.targetField].Value.ToString() : string.Empty;
var newValT = string.Format(config.formatExpression, val1, val2);
if (valT.Equals(newValT))
{
Trace.WriteLine(string.Format(" [SKIP] field already merged {0}:{1}+{2} to {3}:{4}", source.Id, config.sourceField1, config.sourceField2, target.Id, config.targetField));
} else
{
target.Fields[config.targetField].Value = string.Format(config.formatExpression, val1, val2) + config.doneMatch;
Trace.WriteLine(string.Format(" [UPDATE] field merged {0}:{1}+{2} to {3}:{4}", source.Id, config.sourceField1, config.sourceField2, target.Id, config.targetField));
}
}
https://github.com/nkdAgility/azure-devops-migration-tools/pull/529

Test the field merge with Azure Devops Migration tools, I could also reproduce this situation. The doneMatch field is required in the configuration.json file(##Done## by default).
There seems to be no way to avoid adding donematch to the target field. Since I am not the developer of this tool, I am not sure about the function of this field.
I would like to share a workaround to solve this issue.
Workaround:
You could try to set the " " to the doneMatch field. ( "doneMatch": " ")
For example:
"FieldMaps": [
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldMergeMapConfig",
"WorkItemTypeName": "*",
"sourceField1": "System.Description",
"sourceField2": "Microsoft.VSTS.Common.AcceptanceCriteria",
"targetField": "System.Description",
"formatExpression": "{0} {1}",
"doneMatch": " "
}
Since the configuration file is a Json file, you could use " " to represent spaces.
Result:
is it going to detect the previous ##DONE## in the target.fieldA and
think it's already done the merge, so would not make any new changes
Based on my test, the ##Done## in the target field will not affect other operations. You can still operate on this field.
Update:
The above method could only work on field type: Text (multiple lines). If the field is other types, this method doesn't work.
You could create a new field(Text (multiple lines)). Then you could set the target field as the new field.
e.g.
"FieldMaps": [
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldMergeMapConfig",
"WorkItemTypeName": "*",
"sourceField1": "Custom.test1",
"sourceField2": "Custom.test2",
"targetField": "Custom.test3",
"formatExpression": "{0} {1}",
"doneMatch": " "
}

Related

Lucene search to match exact element in Umbraco array

I'm trying to search Umbraco (v10) tags, which are stored as an array. Regardless of whether they are stored in CSV or JSON, I can't force an exact match.
For example, one article has tags containing "foo bar", while a second article has tags containing "foo".
If I use Lucene to search for foo, I get both articles returned. I do not want to match "foo bar".
This is my code:
IBooleanOperation query = _externalIndex.Searcher
.CreateQuery("content", BooleanOperation.And)
.NodeTypeAlias(modelTypeAlias)
.And()
//.Field("tags", tag);
.NativeQuery($"tags:\"{tag}\"");
I have tried using NativeQuery, wrapping my tag in quotes, using regex to match start/end of the string, using parenthesis to indicate a set of matches.
I cannot identify a way to isolate an exact match.
While your question is slightly different the this one, the answer is basically the same.
If you want the index to be based on the whole field rather then the individual words then you need to make sure the field is indexed as a StringField rather then a TextField.
String fields are not tokenized.
RonC's answer pointed me in the right direction, but it was more complicated than that.
I needed to split the JSON array of tags into individual elements, using the TransformingIndexValues event handler, and store them in a new field tagsSplit:
private void TransformTagsValues(object? sender, IndexingItemEventArgs e)
{
if (e.ValueSet.Category != "content")
{
return;
}
Dictionary<string, List<object>> updatedValues = e.ValueSet.Values.ToDictionary(x => x.Key, x => x.Value.ToList());
if (!updatedValues.ContainsKey("tags"))
{
return;
}
JArray tagsArray = JArray.Parse(updatedValues["tags"].Single().ToString()!);
updatedValues["tagsSplit"] = tagsArray.Select(token => (object)token.Value<string>()!).ToList();
e.SetValues(updatedValues.ToDictionary(x => x.Key, x => (IEnumerable<object>)x.Value));
}
Then I needed to configure the indexer to treat the field values as Raw (which internally uses StringField):
public void Configure(string name, LuceneDirectoryIndexOptions options)
{
if (name.Equals(UmbracoIndexes.ExternalIndexName))
{
options.FieldDefinitions.AddOrUpdate(new FieldDefinition("tagsSplit", FieldDefinitionTypes.Raw));
}
}
Now finally if I query using field:
IBooleanOperation query = _externalIndex.Searcher
.CreateQuery("content", BooleanOperation.And)
.NodeTypeAlias(modelTypeAlias)
.And()
.Field("tagsSplit", tag);
It gives me an exact match.
It is also case-sensitive, but since I'm searching by existing tags and not free-text, that is fine for my use-case.

Custom wireshark disector shows value but fieldname is not visible using lua

I am testing some network packets of my Organisation's product. We already have custom plugins. I am trying to add some some more fields into those existing plugins (like conversion of 2 byte code to a string and assign it to a field)
Thankyou in advance for reading my query.
--edit
Wireshark version : 2.4.5 (organization's plugins dont work on latest wireshark application)
--edit
Problem statement:
I am able to add field and show value, but fieldname is not displayed as defined.
I cannot share the entire .lua file but i will try to explain What i did:
Below is the image where I have a field aprint.type. this is a two byte field. In .lua file, for display purpose it is appended with corresponding description using a custom function int_to_enum.
I want to add one more proto field aprint.typetext which will show the text.
What I did:
Added a protofield f_apr_msg_type_txt = ProtoField.string("aprint.typetxt","aprint_type_text") (Tried f_apr_msg_type_txt = ProtoField.string("aprint.typetxt","aprint_type_text",FT_STRING) also)
Below the code where subtree aprint.type is shown, added my required field as subtree:add(f_apr_msg_type_txt, msg_type_string) (Below is image of code extract)
I am able to see the text but field Name is shown as Wireshark Lua text (_ws.lua.text)
Normally displaying strings based on numeric values is accomplished by a value string lookup, so you'd have something like so:
local aprint_type_vals = {
[1] = "Foo",
[2] = "Bar",
[9] = "State alarm"
}
f_apr_msg_type = ProtoField.uint16("aprint.type", "Type", base.DEC, aprint_type_vals)
f_apr_msg_type_txt = ProtoField.string("aprint.typetxt","aprint_type_text", base.ASCII)
... then
local msg_type = tvb(offset, 2):le_uint()
subtree:add_le(f_apr_msg_type, tvb(offset, 2))
subtree:add(f_apr_msg_type_txt, tvb(offset, 2), (aprint_type_vals[msg_type] or "Unknown"))
--[[
Alternatively:
subtree:add(f_apr_msg_type_txt, tvb(offset, 2)):set_text("aprint_type_text: " .. (aprint_type_vals[msg_type] or "Unknown"))
--]]
I'm also not sure why you need the extra field with only the text when the text is already displayed with the existing field, but that's basically how you'd do it.

DOORS DXL Recreating the Main Column

I would like to create an DXL attribute in DOORS that contains the same information as the main column.
It is important to maintain the same heading font style in the attribute as in the main column as this is used for automatic creation of "Table of content" in the Word document after DOORS Publish.
I found the below dxl-script on the internet, but getCanvas does not seem to work.
All text are passed fine to my new attribute, but the heading have the same font style as normal text.
if (obj."Object Heading" "" != "")
{
font(getCanvas, level(obj), HeadingsFont)
displayRich(number(obj) " " obj."Object Heading" "")
}
if (obj."Object Text" "" != "")
{
font(getCanvas, level(obj), TextFont)
displayRich(richTextWithOle(obj."Object Text"))
}
Can anyone help?
KR
Klaus
For me, the code actually does work in a Layout-DXL column (which is not a DXL attribute).
if (obj."Object Heading" "" != "") {
DBE dbCanvas = getCanvas()
font(dbCanvas, level(obj), HeadingsFont)
displayRich(number(obj) " " obj."Object Heading" "")
}
My DOORS version is 9.6, although the methods don't seem to be that new, so DOORS version does not seem to be the issue.
If nothing else helps regarding the DXL code, I would suggest that you have a look at your target word document. There, you should be able to control anything you paste into the document via VBA code in a post-processing step. Although I didn't really get, why you are avoiding using the main column for your source content. Are you trying to show content of a linked or referenced module?
Thanks a lot for the answer. It helped me somehow.
My problem was that I was trying to enter the dxl-code in an attribute. I followed your suggestion and made an Layout-DXL Column instead and it worked almost immediately :-)
I ended up with the dxl-code as shown below.
if (obj."Object Heading" "" != "")
{
DBE dbCanvas = getCanvas()
if( dbCanvas != null )
font(dbCanvas, level(obj), HeadingsFont)
displayRich(number(obj) " " obj."Object Heading" "")
}
else
{ // insert rest of text
if ( probeAttr_(obj, "Requirement") == "Requirement" )
{ // insert requirement text from DT module
displayRich(richText(obj."DXL to DT - ID & Object Text"))
}
else
{ // insert rest of text from this module
displayRich(richTextWithOle(obj."Object Text"))
}
}
I would like to have a publish of the Test Procedure where every Test Case starts with the requirement text followed by the test steps necessary to perform the test as shown in Fig 1.
Test Procedure Publish view
The view in DOORS now looks like I want, but I get error when publishing in DOORS.
DOORS publish error
I therefore protected the line "font(dbCanvas, level(obj), HeadingsFont)", but now I get no headings in the Word document and Table of Content is empty.
Word snip
Is there a solution to this?
KR
Klaus

Jmeter ForEach Controller failing to write variables to file in order retrieved

Jmeter ForEach Controller failing to write variables in original order correctly
I am executing a http request retrieving a json payload with an array of employees. For each record (employee) I need to parse the record for specific fields e.g. firstName, lastName, PersonId and write to a single csv file, incrementing a new row per record.
Unfortunately, the file created has two issues. The PersonId never gets written and secondly the sequence of the values is not consistent with the returned original values. Sometimes I get the same record for lastName with the wrong firstName and vice versa. Not sure if the two issues are related, I suspect my regular expression extract is wrong for a number.
Jmeter setup. (5.2.1)
jmeter setUp
Thread group
+ HTTP Request
++ JSON JMESPath Extractor
+ ForEach Controller
++ Regular Expression Extractor: PersonId
++ Regular Expression Extractor: firstName
++ Regular Expression Extractor: lastName
++ BeanShell PostProcessor
getWorker returns the following payload
jsonPayload
JSON JMESPath Extractor to handle the payload.
{
"items" : [
{
"PersonId" : 398378,
"firstName" : "Sam",
"lastName" : "Shed"
},
{
"PersonId" : 398379,
"firstName" : "Bob",
"lastName" : "House"
}
],
"count" : 2,
"hasMore" : true,
"limit" : 2,
"offset" : 0,
"links" : [
{
"rel" : "self",
"href" : "https://a.site.on.the.internet.com/employees",
"name" : "employees",
"kind" : "collection"
}
]
}
JSON JMESPath Extractor Configuration
Name of created variables: items
JMESPath expressions: items
Match No. -1
Default Values: Not Found
ForEach Controller
ForEach Controller Configuration
Input variable prefix: items
Start Index: Empty
End Index: Empty
Output variable name: items
Add "_"? Checked
Each of the Regular Expression Extracts follow the same pattern as below.
Extract PersonId with Regular Expression
Apply to: Main Sample Only
Field to check: Body
Name of created variable: PersonId
Regular Expression: "PersonId":"(.+?)"
Template: $1$
Match No. Empty
Default Value: PersonId
The final step in the thread is where I write out the parsed results.
BeanShell PostProcessor
PersonNumber = vars.get("PersonNumber");
DisplayName = vars.get("DisplayName");
f = new FileOutputStream("/Applications/apache-jmeter-5.2.1/bin/scripts/getWorker/responses/myText.csv", true);
p = new PrintStream(f);
this.interpreter.setOut(p);
print(PersonId+", "+ PersonNumber+ ", " + DisplayName);
f.close();
I am new to this and looking either for someone to tell me where I screwed up or direct me to a place I can read up on the appropriate topics. (Both are fine). Thank you.
For Each Controller doesn't know the structure of items variable since it is in JSON format. It is capable of just understanding an array and traverses through them. I would suggest to move away from For Each Controller in your case and use the JSON extractor itself for all the values like below
Person ID
First Name
Last Name
Beanshell Sampler Code
import java.io.FileWriter; // Import the FileWriter class
int matchNr = Integer.parseInt(vars.get("personId_C_matchNr"));
log.info("Match number is "+matchNr);
f = new FileOutputStream("myText.csv", true);
p = new PrintStream(f);
for (int i=1; i<=matchNr; i++){
PersonId = vars.get("personId_C_"+i);
FirstName = vars.get("firstName_C_"+i);
LastName = vars.get("lastName_C_"+i);
log.info("Iteration is "+i);
log.info("Person ID is "+PersonId);
log.info("First Name is "+FirstName);
log.info("Last Name is "+LastName);
p.println(PersonId+", "+FirstName+", "+LastName);
}
p.close();
f.close();
Output File
HOW THE ABOVE ACTUALLY WORKS
When you extract values using the matchNr, it goes in a sequential order in which the response has arrived. For example, in your case, Sam & Shed appear as first occurrences and Bob & House appear as subsequent occurrences. Hence JMeter captures them with the corresponding match and stores them as 1st First Name = Sam, 2nd First Name = Bob and so on.
GENERIC STUFF
The regex expression for capturing Person ID which you have used seems to be inaccurate. The appropriate one would be
"PersonId" :(.+?),
and not
"PersonId":"(.+?)"
Move to JSR223 processors instead of Beanshell as they are more performant. Source: Which one is efficient : Java Request, JSR223 or BeanShell Sampler for my script. The migration is pretty simple. Just copy the code that you have in Beanshell and paste it in JSR223.
Close any stream or writer that is open appropriately else it might cause issues when other users are trying to write to the file during load test
In case you are planning to use this file as a subsequent input within JMeter, please note that there is a space between comma and the next element. For example, it is "Sam, Shed" and not "Sam,Shed".JMeter by default does not trim any spaces and will use the value just like that. Hence you might want to take a judicious call regarding that space
Hope this helps!
Since JMeter 3.1 you shouldn't be using Beanshell, go for JSR223 Test Elements and Groovy language for scripting.
Given Groovy has built-in JSON support you shouldn't need any extractors, you can write the data into a file in a single shot like:
new groovy.json.JsonSlurper().parse(prev.getResponseData()).items.each { item ->
new File('myText.csv') << item.get('PersonId') << ',' << item.get('firstName') << ',' << item.get('lastName') << System.getProperty('line.separator')
}
More information: Apache Groovy - Why and How You Should Use It

How do you get strip RTF formatting and get actual string value using DXL in DOORS?

I am trying to get the values in "ID" column of DOORS and I am currently doing this
string ostr=richtext_identifier(o)
When I try to print ostr, in some modules I get just the ID(which is what I want). But in other modules I will get values like "{\rtf1\ansi\ansicpg1256\deff0\nouicompat{\fonttbl{\f0\fnil\fcharset0 Times New Roman;}{\f1\froman\fcharset0 Times New Roman;}} {*\generator Riched20 10.0.17134}\viewkind4\uc1 \pard\f0\fs20\lang1033 SS_\f1\fs24 100\par } " This is the RTF value and I am wondering what the best way is to strip this formatting and get just the value.
Perhaps there is another way to go about this that I am not thinking of as well. Any help would be appreciated.
So the ID column of DOORS is actually a composite- DOORS builds it out of the Module level attribute 'Prefix' and the Object level attribute 'Absolute Number'.
If you wish to grab this value in the future, I would do the following (using your variables)
string ostr = ( module ( o ) )."Prefix" o."Absolute Number" ""
This is opposed to the following, which (despite seeming to be a valid attribute in the insert column dialog) WILL NOT WORK.
string ostr = o."Object Identifier" ""
Hope this helps!
Comment response: You should not need the module name for the code to work. I tested the following successfully on DOORS 9.6.1.10:
Object o = current
string ostr = ( module ( o ) )."Prefix" o."Absolute Number" ""
print ostr
Another solution is to use the identifier function, which takes an Object as input parameter, and returns the identifier as a plain (not RTF) string:
Declaration
string identifier(Object o)
Operation
Returns the identifier, which is a combination of absolute number and module prefix, of object o as a string.
The optimal solution somewhat depends on your underlying requirement for retrieving the object ID.

Resources