I have an XML file in my app resources folder. I am trying to update that file with new dictionaries dynamically. In other words I am trying to edit an existing XML file to add new keys and values to it.
First of all can we edit a static XML file and add new dictionary with keys and values to it. What is the best way to do this.
In general, you can read an XML file into a document object (choose your language), use methods to modify it (add your new dictionary), and (re-)write it back out to either the original XML file, or a new one.
That's straightforward ... just roll up the ol' sleeves and code it up.
The real problem comes in with formatting in the XML file before and after said additions.
If you are going to 'unix diff' the XML file before and after, then order is important. Some standard XML processors do better with order than others.
If the order changes behind the scenes, and is gratuitously propagated into your output file, you lose standard diffing advantages, such as some gui differs, and some scm diffs (svn, cvs, etc.).
For example, browse to:
Order of XML attributes after DOM processing
They discuss that DOM loses order where SAX does not.
You can also write a custom XML 'diff'er (there may be such off-the-shelf ... for example check out 'http://diffxml.sourceforge.net/') that compares 2 XML documents tag-by-tag, attribute-by-attribute, etc.
Perhaps some standard XML-related tool such as XSLT will allow you to keep the formatting constant without changing tag or attribute order. You'd have to research that.
BTW, a related problem is the config (.ini) file problem ... many common processors flippantly announce that the write-order may not agree with the read-order.
Related
I'm writing an ID3 parser and editor. It does already support ID3v1, v2.1-2.3. Are there any other widely used ID3 versions or extensions? For example, I've read about Enhanced ID3v1 tag (which goes before ID3v1) and starts with "TAG+", but I've never seen it inside MP3 files. Should I implement support for it anyway?
"ID3v2.1" never existed.
Yes, Enhanced TAG identifies by TAG+, which extends IDv1.
For a list of all metadata systems to be expected in MP3 files see https://stackoverflow.com/a/62366354 - top priority should have ID3v2.4 as you will encounter those most aside from ID3v2.3. Then go for informal and/or legacy ones because those can still be encountered (just because files become old doesn't mean they cease to exist).
Keep the following things in mind when parsing files:
A file can have both: IDv1 and IDv2 tags.
A file can have multiple IDv2 tags (i.e. IDv2.3 and IDv2.4). Although it shouldn't occur it should pose no problem to your parser to also accept multiple tags of the same version.
ID3v2 is not limited to MP3 files (but IDv1 and all its informal extensions are).
Consider the following parsing order in an MP3 file:
Check for ID3v1 at the end of the file.
Check for ID3v1.2 in front of ID3v1.
Check for Enhanced TAG in front of ID3v1.
Check for multiple ID3v2 at the start of file and, as for ID3v2.4, a footer at the end of the file in front of all ID3v1-like tags.
Given:
Some kind of DSL parsed with Xtext parser and then edited by user in TMF-based editor.
When user open file for editing I want first get access to the parse tree of just opened file, modify loaded file content in a some way and then provide to user modified source for editing.
When user wish to save file I again want to preprocess text representation based on actual parse tree and save such altered version.
Is there any Xtext/EMF API to implement such pre-/post- processing?
The goal is to add some content not presented in the physical file, allow user to edit this content and remove it before saving to file. This extra content should be stored separately from DSL source file.
If I understand your question correctly, you want to display additional information in the text editor itself (and not add additional information only to the EMF model, not to the text, for which IDerivedStateComputer could be used).
If the user is not supposed to edit the additional text, the "Code Mining" feature might be useful: https://www.eclipse.org/Xtext/documentation/310_eclipse_support.html#code-mining and https://blogs.itemis.com/en/code-mining-support-in-xtext
To answer the question itself:
Is there any Xtext/EMF API to implement such pre-/post- processing?
No, I am pretty sure there is no such Xtext API for pre-/post-processing files based on their own parse tree (EMF is irrelevant as you want to change the physical content). You could try to mess around with the XtextDocumentProvider (i.e. create your own subclass and register it in the UI module), but this is very likely to break the UI because the line numbers and offsets won't match.
You might have more luck implementing a custom Eclipse action that is executed on the original file and creates a temporary modified copy based on the parsed original file and then opens an editor for the temporary file. Then you could implement a IXtextBuilderParticipant that writes the result back to the original file on save (you have to register it using the org.eclipse.xtext.builder.participant extension point).
Another idea would be not to use an Eclipse action but a tabbed editor using MultiPageEditorPart, with the original as one of three tabs (the composite file and the 'additional info' file being the other two).
The goal is to add some content not presented in the physical file, allow user to edit this content and remove it before saving to file. This extra content should be stored separately from DSL source file.
Couldn't you present this information in another view similar to the 'Properties' view of EMF ? e.g. the user opens file, the Xtext editor opens as well as the 'Properties' view, which presents a way to edit these "extra" information. Upon save of either view, the Xtext save is called and your extra properties are serialized in their own model.
I need to process a (GCS) bucket of files, where each file is compressed and contains a single multi-line JSON record. Also, the name of the file being processed is significant and I need to know it within my transform.
Starting with examples in the docs, TextIO looks pretty close, but it looks like its designed to process each file line-by-line and does not allow me to read the entire file at once. Also, I don't see any way to get the filename that's being processed?
PCollectionTuple results = p.apply(TextIO.Read
.from("gs://bucket/a/*.gz")
.withCompressionType(TextIO.CompressionType.GZIP)
.withCoder(MyJsonCoder.of()))
Looks like I need to write a custom IO reader, or some such? Any tips for best place to start?
You are correct that right now none of the existing classes do exactly what you want. There are 2 reasonable approaches:
Match the filepattern yourself (using IOChannelUtils and IOChannelFactory) and wrap the resulting files into a PCollection<String> where the String will be a filename, using Create.of(filenames). Then apply a ParDo with a function which reads the given filename.
Write your own subclass of Source (there's also FileBasedSource, but it's not quite right for your use case). It would be configured by the filepattern, and splitIntoBundles would match the filepattern and expand into individual sources each corresponding to one file.
I would recommend the first approach because it seems like less code and your use case does not require the full power of Source.
I am using tshark to filter some packets based on Display/Read filters from one file into another.
I want to have one final output file out.pcap after executing multiple read filters over number of files and combine all into out.pcap.
I was trying to use mergecap but it does not allow to append (combine) two file and store in one of them without overwriting.
Is there any way to do this, as I don't want to keep creating temporary files and merge all them together at the end.
This is not possible that I know of with existing tools, although given the way the capture file format is layed out, it should be possible to write a new tool (or extend mergecap) to do this.
We have two .po files, each from different branches of a piece of software.
We need to combine these into a single .po file.
There are duplicates between the two files, and the ideal handling would be for one file's strings to be favoured (consistently).
We have a SUSE system so the --output-file doesn't seem to have the behaviour of ignoring/merging duplicates which the Sun version has according to a man page I found from a web search. (We do not have a Sun machine handy!)
What you are looking for is the msgcat util, it concatenates and merges the specified PO dictionaries.
This is part of gettext utils, for more information please consult gettext manual page on msgcat.
you can use poedit.
To merge your current po-file, you must to open it and click:
Catalog > Update from POT-file.
Set the filter to all files and select your second.po file
Poedit will show you new & obsolete strings
I use msgmerge:
msgmerge [old_file.po] [new_file.po] > output.po
It works for me, but be aware that it does a silly merge, it is, it discards the entries in the old_file (new file items overwrites old one items).