replace all double quotes with nothing in csv file in BIML script - visual-studio-2019

I am importing flatfile connections using BIML.
" is used around text and ; is used as delimiter.
However, in some of the files I see this:
;"this is valid text""";
There are double double quotes with nothing between them. If I edit the file and search & replace all double double quotes with nothing, the import runs well. So, is it in BIML possible to do this action automagically? Search al instances of "" and replace these with ?
<#
string[] myFiles = Directory.GetFiles(path, extension);
string[] myColumns;
// Loop trough the files
int TableCount = 0;
foreach (string filePath in myFiles)
{
TableCount++;
fileName = Path.GetFileNameWithoutExtension(filePath);
#>
<Package Name="stg_<#=prefix#>_<#=TableCount.ToString()#>_<#=fileName#>" ConstraintMode="Linear" AutoCreateConfigurationsType="None" ProtectionLevel="<#=protectionlevel#>" PackagePassword="<#=packagepassword#>">
<Variables>
<Variable Name="CountStage" DataType="Int32" Namespace="User">0</Variable>
</Variables>
<Tasks>
<ExecuteSQL ConnectionName="STG_<#=application#>" Name="SQL-Truncate <#=fileName#>">
<DirectInput>TRUNCATE TABLE <#=dest_schema#>.<#=fileName#></DirectInput>
</ExecuteSQL>
<Dataflow Name="DFT-Transport CSV_<#=fileName#>">
<Transformations>
<FlatFileSource Name="SRC_FF-<#=fileName#> " ConnectionName="FF_CSV-<#=Path.GetFileNameWithoutExtension(filePath)#>">
</FlatFileSource>
<OleDbDestination ConnectionName="STG_<#=application#>" Name="OLE_DST-<#=fileName#>" >
<ExternalTableOutput Table="<#=dest_schema#>.<#=fileName#>"/>
</OleDbDestination>
</Transformations>
</Dataflow>
</Tasks>
</Package>
<# } #>

Turns out I was looking completely at the wrong place for this.
Went to the part where the file is read and added .Replace("\"\"","")
myColumns = myFile.ReadLine().Replace("""","").Replace(separator,"").Split(delimiter);

Related

can't extract module's Version DXL IBM DOORS

I'm working on a DXL program in Doors which supposed to output to a csv file all of source module, target, linkset and version of each (source/target) modules. I've succeed to output "source module, target, linkset but I couldn't extract the version of modules. Does anyone know how to do it ?
Here is my code bellow:
The following uses an output file which is in C:\Temp, but you can easily change that.
it works by matching modules using a grep string, again which you can change to suit.
I have different module types, and therefore I can just get the results on the prefix 'STR' in this case. I decided to have multiple scripts for different prefix modules, rather than dynamically pass the STR keyword.
You need to enter your folder and subfolders for the "modPathname"
then adjust the Regexp "GrepString" to suit your naming convention
alternatively you can bypass all this by hardcoding the values.
string filename = "C:\\TEMP\\STR_Baseline_Info.txt"
Stream outFile = write filename
string Modtype = "STR"
void PrintAndOutput ( string s)
{
// print (s) // Enable if you want output
outFile << s
}
void DisplayResults( string Modtype )
{
Item modItem
Module mName
Regexp GrepString = regexp2 "^" Modtype "-[0-8][0-9][0-9]"
Folder modPathname = folder "/EnterYourFolderHere/AnySubFolders/" Modtype ""
string fullModuleName , CommentStr , moduleName
Baseline b
PrintAndOutput "------------------------------------------------------------------------------\n"
CommentStr = "This File is automatically produced via the " Modtype " Baseline Info.dxl script. \n" Modtype " Versions as of : " dateAndTime(today) "\n"
PrintAndOutput CommentStr
PrintAndOutput "-------------------------------------------------------------------------------\n\n"
for modItem in modPathname do
{
if (type (modItem ) == "Formal")
{
moduleName = (name modItem)
if (GrepString moduleName)
{
fullModuleName = (fullName(modItem))
mName = read(fullName modItem , false)
b= getMostRecentBaseline (mName)
if (b != null )
{
PrintAndOutput moduleName " -\tBaseline : "(major b)"."(minor b)(suffix b) " \t " (dateOf b) "\n"
}
else
{
PrintAndOutput moduleName " \t### No Baseline Information ### \t" " \n"
}
}
}
}
}
DisplayResults ( Modtype)

Using Python 3.6 to parse XML how can I determine if an XML tag contains no data

I am trying to learn Python by writing a script that will extract data from multiple records in an XML file. I have been able to find the answers to most of my questions by searching on the web, but I have not found a way to determine if an XML tag contains no data before the getElementsByTagName("tagname")[0].firstChild.data method is used and an AttributeError is thrown when no data is present. I realize that I could write my code with a try and handle the AttributeError but I would rather know that the tag is empty before I try to extract the data an not have to handle the exception.
Here is an example of an XML file that contains two records one with data in the tags and one with an empty tag.
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
<records>
<rec>
<name>ZYSRQPO</name>
<state>Washington</state>
<country>United States</country>
</rec>
<rec>
<name>ZYXWVUT</name>
<state></state>
<country>Mexico</country>
</rec>
</records>
Here is a sample of the code that I might use to extract the data:
from xml.dom import minidom
import sys
mydoc = minidom.parse('mydataFile.xml')
records = mydoc.getElementsByTagName("rec")
for rec in records:
try:
name = rec.getElementsByTagName("name")[0].firstChild.data
state = rec.getElementsByTagName("state")[0].firstChild.data
country = rec.getElementsByTagName("country")[0].firstChild.data
print('{}\t{}\t{}'.format(name, state, country))
except (AttributeError):
print('AttributeError encountered in record {}'.format(name), file=sys.stderr)
continue
When processing this file no information for the record named ZYXWVUT will be printed except that an exception was encountered. I would like to be able to have a null value for the state name used and the rest of the information printed about this record. Is there a method that can be used to do what I want, so that I could use an if statement to determine whether the tag contained no data before using getElementsByTagName and encountering an error when no data is found?
from xml.dom import minidom
import sys
mydoc = minidom.parse('mydataFile.xml')
records = mydoc.getElementsByTagName("rec")
for rec in records:
name = rec.getElementsByTagName("name")[0].firstChild.data
state = None if len(rec.getElementsByTagName("state")[0].childNodes) == 0 else rec.getElementsByTagName("state")[0].firstChild.data
country = rec.getElementsByTagName("country")[0].firstChild.data
print('{}\t{}\t{}'.format(name, state, country))
Or if there is any chance, that name and country is empty too:
from xml.dom import minidom
import sys
def get_node_data(node):
if len(node.childNodes) == 0:
result = None
else:
result = node.firstChild.data
return result
mydoc = minidom.parse('mydataFile.xml')
records = mydoc.getElementsByTagName("rec")
for rec in records:
name = get_node_data(rec.getElementsByTagName("name")[0])
state = get_node_data(rec.getElementsByTagName("state")[0])
country = get_node_data(rec.getElementsByTagName("country")[0])
print('{}\t{}\t{}'.format(name, state, country))
I tried reedcourty's second suggestion and found that it worked great. But I decided that I really did not want none to be returned if the element was empty. Here is what I came up with:
from xml.dom import minidom
import sys
def get_node_data(node):
if len(node.childNodes) == 0:
result = '*->No ' + node.nodeName + '<-*'
else:
result = node.firstChild.data
return result
mydoc = minidom.parse(dataFileSpec)
records = mydoc.getElementsByTagName("rec")
for rec in records:
name = get_node_data(rec.getElementsByTagName("name")[0])
state = get_node_data(rec.getElementsByTagName("state")[0])
country = get_node_data(rec.getElementsByTagName("country")[0])
print('{}\t{}\t{}'.format(name, state, country))
When this is run against this XML:
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
<records>
<rec>
<name>ZYSRQPO</name>
<country>United States</country>
<state>Washington</state>
</rec>
<rec>
<name></name>
<country>United States</country>
<state>Washington</state>
</rec>
<rec>
<name>ZYXWVUT</name>
<country>Mexico</country>
<state></state>
</rec>
<rec>
<name>ZYNMLKJ</name>
<country></country>
<state>Washington</state>
</rec>
</records>
It produces this output:
ZYSRQPO Washington United States
*->No name<-* Washington United States
ZYXWVUT *->No state<-* Mexico
ZYNMLKJ Washington *->No country<-*

Tag search in HTML using js in ant

I am using the below mentioned code to get the content of a specific tag, but when I am trying to execute it I am getting some extra data along with it, I don't understand why is it happening. Lets say if I search for title tag then I am getting " [echo] Title : <title>Unit Test Results</title>,Unit Test Results" this as result, but the problem is title only contains "<title>Unit Test Results</title>" why this extra ",Unit Test Results" thing is coming.
<project name="extractElement" default="test">
<!--Extract element from html file-->
<scriptdef name="findelement" language="javascript">
<attribute name="tag" />
<attribute name="file" />
<attribute name="property" />
<![CDATA[
var tag = attributes.get("tag");
var file = attributes.get("file");
var regex = "<" + tag + "[^>]*>(.*?)</" + tag + ">";
var patt = new RegExp(regex,"g");
project.setProperty(attributes.get("property"), patt.exec(file));
]]>
</scriptdef>
<!--Only available target...-->
<target name="test">
<loadfile srcFile="E:\backup\latest report\Report-20160523_2036.html" property="html.file"/>
<findelement tag="title" file="${html.file}" property="element"/>
<echo message="Title : ${element}"/>
</target>
The return value of RegExp.exec() is an array. From the Mozilla documentation on RegExp.prototype.exec():
The returned array has the matched text as the first item, and then
one item for each capturing parenthesis that matched containing the
text that was captured.
If you add the following code to your JavaScript...
var patt = new RegExp(regex,"g");
var execResult = patt.exec(file);
print("execResult: " + execResult);
print("execResult.length: " + execResult.length);
print("execResult[0]: " + execResult[0]);
print("execResult[1]: " + execResult[1]);
...you'll get the following output...
[findelement] execResult: <title>Unit Test Results</title>,Unit Test Results
[findelement] execResult.length: 2
[findelement] execResult[0]: <title>Unit Test Results</title>
[findelement] execResult[1]: Unit Test Results

WLST Ant Task white space within arguments

I am using the WLST Ant task which allows a list of space delimited arguments to be passed in under the arguments attribute.
The issue is when I pass a file directory which contains a space. For instance "Program Files" which becomes two arguments of Program and Files.
Is there any suggestions to get around this?
My suggestion below would only work with one value.
For example append the "Program Files" argument to the end and loop from the known end argument to the actual end of sys.argv.
IE If we want "Program Files" to be the 4th system argument then inside the WLST script we append sys.argv[4],[5]...[end].
Short answer for WLST 11.1.1.9.0: You can't get around this.
I have the same problem and debugged a bit.
My findings:
The class WLSTTask in weblogic-11.1.1.9.jar calls via command line WLSTInterpreterInvoker which parses the args:
private void parseArgs(String[] arg) {
for (int i = 0; i < arg.length; i++) {
this.arguments = (this.arguments + " " + arg[i]);
}
[...]
For reasons I don't know these args are parsed again, before the python script is invoked:
private void executePyScript() {
[...]
if (this.arguments != null) {
String[] args = StringUtils.splitCompletely(this.arguments, " ");
[...]
public static String[] splitCompletely(String paramString1, String paramString2)
{
return splitCompletely(new StringTokenizer(paramString1, paramString2));
}
private static String[] splitCompletely(StringTokenizer paramStringTokenizer) {
int i = paramStringTokenizer.countTokens();
String[] arrayOfString = new String[i];
for (int j = 0; j < i; j++) arrayOfString[j] = paramStringTokenizer.nextToken();
return arrayOfString;
}
Unfortunately the StringTokenizer method does not distinguish quoted strings and so sys.argv in Python gets separate arguments, even if you quote the parameter
There are two possible alternatives:
Replace spaces in Ant with something else (eg %20) and 'decode' them in Python.
Write a property file in Ant and read from that in Pyhton.
The code for executePyScript() in 12.2.1 has changed a lot and it seems that the problem may be gone there (I haven't checked)
if ((this.arguments.indexOf("\"") == -1) && (this.arguments.indexOf("'") == -1))
args = StringUtils.splitCompletely(this.arguments, " ");
else {
args = splitQuotedString(this.arguments);
}

Count images in Folder using ASP.net

I want to count the number of images a folder folder, but it produces this error :
Could not find a part of the path 'c:\Content\slideshow\images\image\'.
All of the images are in a folder in the project. Located a Content/slideshow/images/image
This is my code:
<%
string dir = #"/Content/slideshow/images/image";
string[] files;
int numFiles;
files = System.IO.Directory.GetFiles(dir);
numFiles = files.Length;
Response.Write("num ber of images :" + numFiles);
%>
"Could not find a part of the path
'c:\Content\slideshow\images\image\'"
Means very simply that the folder does not exist. If you want to user a relative path you can do the following.
Server.MapPath("~/{RelativePathHere})
Edit : In response to your comment. You will need to loop through the file and check for file extension of each (Keeping your own count)
Use HttpContext.Current.Server.MapPath to map the virtual path to physical path and then pass it to Directory.GetFiles method
To call Directory.GetFiles() you need to pass the full path to the images directory.
string dirPath = #"~/Content/slideshow/images/image";
string dirFullPath = Server.MapPath(dirPath);
string[] files;
int numFiles;
files = System.IO.Directory.GetFiles(dirFullPath);
numFiles = files.Length;
Response.Write("number of images: " + numFiles);
Server.MapPath returns the entire physical file path associated to the dirPath virtual path.
You need to pass this as a relative path using Server.MapPath
Then I would suggest using DirectoryInfo.GetFiles instead of Directory.GetFiles and filter on the image types that you want, so you don't count non-image files. This will yield a FileInfo[].
<%
string dir = Server.MapPath(#"/Content/slideshow/images/image");
FileInfo[] files;
int numFiles;
files = (new System.IO.DirectoryInfo(dir)).GetFiles("filePattern");
numFiles = files.Length;
Response.Write("num ber of images :" + numFiles);
%>
If you have multiple file types that you want to count the best way to do this is just to remove the pattern then filter the results.
var extensions = new String[] {"jpg", "png", "gif"};
files = (new System.IO.DirectInfo(dir)).GetFiles();
foreach(var extension in extensions)
{
numFiles += files.AsEnumerable.Where(f => f.Extension.Equals(extension));
}
int numberOfFiles ;
string path = "C:/PIC";
numberOfFiles = System.IO.Directory.GetFiles(path).Length;
Check Now

Resources