This is a question about Nifi.
I made Nifi pipeline to convert flowfile with xml format to csv format.
Now, I would like to concatenate or union the converted csv flowfile to existing tables by filename (which stands for table name as well).
Simply put, my processor flow is following.
GetFile (from a particular directory) -> 2. Convert xml to csv -> 3.Update the flowfile with table name
-> 4. PutFile (to a different directory)
But, at the end of the flow, PutFile processor throws an error, saying "file with the same name already exists".
I have no ideas how flowfile can be added to existing csv table.
Any advice, tips, ideas are appreciated.
Thank you in advance.
there is no support to append file however you could use ExecuteGroovyScript to do it:
def ff=session.get()
if(!ff)return
ff.read().withStream{s->
String path = "./out_folder/${ff.filename}"
//sync on file path to avoid conflict on same file writing (hope)
synchronized(path){
new File( path ).append(s)
}
}
REL_SUCCESS << ff
if you need to work with text (reader) content rather then byte (stream) content
the following example shows how to exclude 1 header line from flow file if destination file already exists
def ff=session.get()
if(!ff)return
ff.read().withReader("UTF-8"){r->
String path = "./.data/${ff.filename}"
//sync on file path to avoid conflict on same file writing (hope)
synchronized(path){
def fout = new File( path )
if(fout.exists())r.readLine() //skip 1 line (header) only if out file already exists
fout.append(r) //append to the file the rest of reader content
}
}
REL_SUCCESS << ff
In my Rails app, I managed to stream large CSV files directly from Postgres based on solutions mentioned in this SO post. My working code looks somewhat like so:
query = <A Long SQL Query String>
response.headers["Cache-Control"] = "no-cache"
response.headers["Content-Type"] = "text/csv; charset=utf-8"
response.headers["Content-Disposition"] =
%(attachment; filename="#{csv_filename}")
response.headers["Last-Modified"] = Time.now.ctime.to_s
conn = ActiveRecord::Base.connection.raw_connection
conn.copy_data("COPY (#{query}) TO STDOUT WITH (FORMAT CSV, HEADER TRUE, FORCE_QUOTE *, ESCAPE E'\\\\');") do
while row = conn.get_copy_data
response.stream.write row
end
end
response.stream.close
end
Some of the columns (VARCHAR) being queried have values as either English or Chinese strings. The CSV file resulting from the above code doesn’t show the Chinese characters as is. Instead, I get something like this:
大大 文文
Am I supposed to change the way I’m using the copy_data function, or is there something I could do to the CSV file to solve this? I’ve tried saving the file as UTF-8 .txt file, as well as trying the convert_to function mentioned in the copy_data documentation, but to no avail.
This depends of the original encoding included in the CSV file.
Do this on Linux :
file -i you_file
Are you sure it's not UTF-16 or GB 18030 ?
And also in what kind of encoding is setup your database ?
do a \l in psql to see this.
So it boiled down to my MS Excel not being able to render the Chinese chars correctly. On MacOS, opening the same .csv file using the Numbers app (or even Atom, for that matter) resolved this issue for me.
If I copy something from textedit or web and paste it to localizable file it shows this compilation error. If I type those in localizable file it does not show any error. I am assuring you that I using the correct format and ';' in the file.
"New" = "New";
"In Progress" = "In Progress";
"Waiting" = "Waiting";
"Closed" = "Closed";
Use plutil from the Terminal:
you have to run it for each version of the localizable file. E.g
cd into your project root
cd eb.lproj - you can replace this with
any localisation you are working with.
plutil -lint Localizable.strings
When you run step 3, you will either be shown an error, telling you what is wrong with your file. Or you will be told the file is OK
Note that plutil output is bad, it seems it always says "Unexpected character at line 1" but above that output, it prints other stuff like missing semicolon on line 121, and that is the real error
For me, it was missing semi-colons. If you use a tool to generate .strings file, make sure there are no un-escaped quotes that may 'eat' the delimiting semi-colons.
pl < Localizable.strings
is better than plutil -lint Localizable.strings
Log will show something like this
2019-08-14 16:39:34.066 pl[21007:428513] CFPropertyListCreateFromXMLData(): Old-style plist parser: missing semicolon in dictionary on line 427. Parsing will be abandoned. Break on _CFPropertyListMissingSemicolon to debug.
2019-08-14 16:39:34.068 pl[21007:428513] CFPropertyListCreateFromXMLData(): Old-style plist parser: missing semicolon in dictionary on line 427. Parsing will be abandoned. Break on _CFPropertyListMissingSemicolon to debug.
2019-08-14 16:39:34.071 pl[21007:428513] *** Exception parsing ASCII property list: NSParseErrorException Error Domain=NSCocoaErrorDomain Code=3840 "Unexpected character / at line 1" UserInfo={NSDebugDescription=Unexpected character / at line 1, kCFPropertyListOldStyleParsingError=Error Domain=NSCocoaErrorDomain Code=3840 "Missing ';' on line 427" UserInfo={NSDebugDescription=Missing ';' on line 427}}
Fastest way to detect the line with the issued string is to :
right click the strings file and
then Open as/ASCII property list.
Xcode will immediately tell you in what line there's an error.
I know this question was asked long ago but my scenario and solution is little bit different.
Today I faced same issue but when I tried to check the issue using
plutil -lint Localizable.strings
I got OK status which means everything is fine, then I tried to find issue using
pl < Localizable.strings
But again I got file text printed with no error mentioned, then I tried a trick and it worked for me.
Right click on the Localizable.strings file
Then select option Open As option
Then select option ASCII Property List
That's it, XCode shows me the issue with line number and the issue was I had this DéjàVerified text as key on specified line, this helps me to identify and solve the issue, I hope it will save someone's time.
Cheers!
There can be multiple reasons for this:
Semicolon is missing at the end.
Multiple semicolons at the end.
" within the message which should be escaped by \".
Extra character after semicolon.
Invalid white space in the file.
Other invalid characters in the file.
Merge conflict characters in the file!
<<<<<<< HEAD, ======= and >>>>>>>.
Please note that plutil -lint Localizable.strings returned OK for point-2 & 7!
In my case, I was missing "=" between a string pair. Even plutil did not help me to spot the error line. I manually checked each string pair. :/
Your syntax seems to be fine, the only thing that I can see can "break" your file and cause this error is the quote character. Make sure to use the reqular one " and not in any other form like ″ for example.
Also make sure the strings file name is always Localizable.strings
I Had the same issue and i resolved it by commenting or removed unused strings in my Localizable.String file :)
I once had a similar error and it turned out that there was an URL in the middle of the file, like this:
// Some Comment 1
"Some key 1" = "Some value 1";
http://...whatever...
// Some Comment 2
"Some key 2" = "Some value 2";
When calling plutil -lint on that file the output was:
Unexpected character / at line 1
Well, the first character indeed was / as the file started with a comment but the problem resolved after removing the URL; or turning it into a comment which it actually should have been. Note that the URL was nowhere near the beginning of the strings file, it was about in the middle of a 6000 lines string file. I was only able to find it by browsing through commit history and always look at the changes.
if missing ";" at end of the all lines in Localizable.string file, this error can occur.
eg :-
"header_text" = "Current Language";
"change_language" = "Change Language";
"header_text" = "වත්මන් භාෂාව";
"change_language" = "භාෂාව වෙනස් කරන්න";
This may be because the translation file format is wrong.
You can download a mac software called Localizable.
This is the download link: https://apps.apple.com/cn/app/localizable-%E7%BF%BB%E8%AF%91%E6%96%87%E4%BB%B6%E5%B7%A5%E5%85%B7/id1268616588?mt=12,
you only need to drag Localizable.strings to the software and it will It is useful to tell you which line in the file may have a problem. It saved me a lot of time. Now I share it with you.
I hope it will be helpful to you.
It seems your info.plist is not in correct form . check it properly. I also had the same issue . I resolved it by modifying my info.plist.
I just had this experience:
external translator doing the work inside Visual Code or other text editors
Files not working and getting an error like this one: ( testing with plutil -lint )
Localizable.strings: Unexpected character " at line 1
CardRatingView.strings: Unexpected character / at line 2
I just created a new file within XCode and copy pasted all the file content and suddenly everything was working properly.
I guess something can go wrong / corrupting the file itself while working with other text editors.
If showing something like Unexpected character " at line 1, and it is the first string like "app_name"="Any Name"
Check that the file is UTF16
I ran into this issue, all my formatting was correct. Checking for illegal characters using plutil -lint Localizable.strings and using ruby libraries like "utf8_utils" also didn't work at finding the illegal characters. BUT when I pasted the Localizable.strings contents into the Terminal app while running irb, it did show me the weird characters.
"PercentComplete" = "%d procent gennemført";
Pasted into irb:
"PercentComplete"\U+FFC2\U+FFA0= "%d procent gennemf\U+FFC3\U+FFB8rt";
Then all I had to do was a regex replace to fix those weird white space characters: \U+FFC2\U+FFA0
Thanks to the plutil suggestion I understood that to make it work you have to delete also any \ or * as are not read as comments and, important, add a ; to the end of the file. Xcode 11.5.
If pl and plutil show no problems, check the file's encoding. I had a similar problem twice and in my case it was due to incorrect encoding, though I have no idea how it has been changed (I literally added a single line in the middle of the file in X-Code). Converting from UTF-16LE to UTF-16BE in some editor (I used Android Studio) fixed the problem.
For me I had an NSLocalizedString in my code that contained a string interpolation e.g. NSLocalizedString("\(product.price ?? "")per_month"). When I exported localisations this got added to my strings file, which was then in the wrong format. It threw me off because my strings file in Xcode looked fine, but actually the file gets updated as part of the export localisations process, and errors were creeping in there.
If anyone things they might be having the same issue try calling genstrings separately and seeing if the newly generated file is in the correct format. Make sure you save your strings first as this will overwrite your strings file : find ./ -name "*.swift" -print0 | xargs -0 genstrings -SwiftUI -o en.lproj
This tool can help solve this problem, just select your localizable.strings file, it will help you find out which line format is wrong, it can save a lot of time
https://localizable.appdevtool.io/
In my case, I had one line with using ” instead of " and that breaks the file. My code editor did not detect this difference.
I was having the similar issue where i didn't escape the string value with backslash \ for one of my string's value.
Before:
"INVALID_NUMBER" = "It seems you're entering invalid number. Number should starts with "0" or "7"";
Updated:
"INVALID_NUMBER" = "It seems you're entering invalid number. Number should starts with \"0\" or \"7\"";
Backslashes are required when you want to display the quotation marks "
Please, have a quick look at here for How to include Quotation mark in strings
It seems like SVN is having some issue with this file. As it consider it to be a binary file. It is inserting a lot of non printable characters between each characters. I still couldn't find a proper solution. Just changing the Localizable.string files from production PC for avoiding any issue with it.
Update: Updating the SVN client (smartSVN) to the latest version solved the issue. It seems one of my colleague was using a older version. When he commited the change to localizable file it caused the error.
I run my .fsx file like
>fsi A.fsx
In this file I read csv with CsvProvider that has to have path to csv data.
type Data = CsvProvider<"my_data.txt", ";", Schema
I need to pass file name as command line argument and it is possible
>fsi A.fsx my_data.txt
I can read it like
let originalPath = fsi.CommandLineArgs.ElementAt(1)
Problem is, that file name used in CsvProvider constructor needs to be constant and command line argument is not. How I can initialize CsvProvider from command line argument?
The value inside the angle brackes <"my_data.txt"...> specifies an example format file and is checked at compile time, hence the need for it to be a constant string. Assuming your .fsx script merely wants to load a different CSV file of the same general format, you would use
let contents = Data.Load(originalPath)
I read an Excel 2003 file with a text editor to see some markup language.
When I open the file in Excel it displays incorrect characters. On inspection of the file I see that the encoding is Windows 1252 or some such. If I manually replace this with UTF-8, my file opens fine. Ok, so far so good, I can correct the thing manually.
Now the trick is that this file is generated automatically, that I need to process it automatically (no human interaction) with limited tools on my desktop (no perl or other scripting language).
Is there any simple way to open this XL file in VBA with the correct encoding (and ignore the encoding specified in the file)?
Note, Workbook.ReloadAs does not function for me, it bails out on error (and requires manual action as the file is already open).
Or is the only way to correct the file to go through some hoops? Either: text in, check line for encoding string, replace if required, write each line to new file...; or export to csv, then import from csv again with specific encoding, save as xls?
Any hints appreciated.
EDIT:
ADODB did not work for me (XL says user defined type, not defined).
I solved my problem with a workaround:
name2 = Replace(name, ".xls", ".txt")
Set wb = Workbooks.Open(name, True, True) ' open read-only
Set ws = wb.Worksheets(1)
ws.SaveAs FileName:=name2, FileFormat:=xlCSV
wb.Close False ' close workbook without saving changes
Set wb = Nothing ' free memory
Workbooks.OpenText FileName:=name2, _
Origin:=65001, _
DataType:=xlDelimited, _
Comma:=True
Well I think you can do it from another workbook. Add a reference to AcitiveX Data Objects, then add this sub:
Sub Encode(ByVal sPath$, Optional SetChar$ = "UTF-8")
Dim stream As ADODB.stream
Set stream = New ADODB.stream
With stream
.Open
.LoadFromFile sPath ' Loads a File
.Charset = SetChar ' sets stream encoding (UTF-8)
.SaveToFile sPath, adSaveCreateOverWrite
.Close
End With
Set stream = Nothing
Workbooks.Open sPath
End Sub
Then call this sub with the path to file with the off encoding.