I am placing text files in spool direcotry.
I want to place in HDFS with the same names as source file names.
I used %{file} for prefix but it is giving full path of filename.But I dont want absolutepath,want only filename.
Regards,
KSReddy
You need to add a parameter for spooldir which adds a header which is false by default.
agentname.sources.sourcename.fileHeader=true
Related
Is it possible somehow to do a general configuration for Eleventy so that 'my_file.html' in the input folder ends up just as 'my_file.html' in the _site folder, not '/my_file/index.html'?
I know I can do it on a file by file basis with permalinks. But I'd like it configured for the site as a whole, if possible.
Unfortunately, since this behavior isn't encouraged, there isn't a simple configuration option to disable the directory output.
However, since permalink is part of the data cascade, you can set a global default, and in combination with the filePathStem computed data, you can set the output to be .html files.
To set a permalink using a global data file, add a permalink.js file to your global _data directory.
// _data/permalink.js
module.exports = '/{{ page.filePathStem }}.html'
There is also a config global data option coming soon to v1.0.0. I am not sure if it will handle the {{ page.filePathStep }} preprocessing, but if it does, that could be an option too, especially since it will keep config-related things inside the config file.
Yes, you can control where Eleventy writes things out by specifying a permalink value in the front matter. This is covered here: https://www.11ty.dev/docs/permalinks/#remapping-output-(permalink)
An example would be:
---
permalink: "/my_file.html"
---
I use flume-1.8.0.
On the document, it says that I cannot set the directory pattern.
(Regular expression (and not file system patterns) can be used for filename only.)
But I have to set the directory pattern to get log from other system which controlled by other team.
Is there some solution to set directory path like /dir/201801/0101.log, /dir/201802/0001.log, ... ?
Use something like this for the file groups with file patterns i.e use the Regex ASCII pattern see https://en.wikipedia.org/wiki/Regular_expression for more details
a1.sources.r1.filegroups.f2 = /path/to/files/with/pattern/databundle_cnt_[0-9]{4}-[0-9]{2}-[0-9]{2}.csv
In your case I will advise
a1.sources.r1.filegroups.f2 = /dir/[0-9]{6}/[0-9]{4}.log
I am looking for the Flume "Spooling Directory Source" recursive-look for the the files within subdirectories.
There are some references here https://issues.apache.org/jira/browse/FLUME-1899
however since then multiple versions have come out, is there any way we can have recursive directory lookup within subdirectories for the files in Spooling Source.
I think you can use the patch FLUME-1899-2.patch directly.
set the "recursiveDirectorySearch" as ture in your config file.
NOTE: the regex in ignorePattern of config file will also affect the recursiveDirectory folder name. so you might need to modify the code in org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java if you want to ignore the folder name.
A lot of my website content is stored in a database for this reason I created a script that goes through the content stored in the database and extracts strings that must be added to my .pot file. (which already contains other strings).
I created a function called "add_string($s)" that should add one string to the .pot file but I'm not sure how to do it.
function add_string($s)
{
// Add string to .pot file?
}
How can this be done?
Thanks
A pot file is just a text file in certain format.
You can use file_put_contents with the FILE_APPEND flag to add text to a file.
With regards to the format of the data argument of the file_put_contents call, you will have
lines referencing files starting with #:, lines referencing instructions to translators starting with #. and then lines starting with msgid with the message id and msgstr the message value. You will want to include the newline \n after each line.
See here for an example of the pot file format.
You will probably want to keep track of your strings in PHP data structure as you write them to file or before you write them to file, to avoid duplicating message values in the template.
Is it possible using .tfignore to add a wildcard to directories? I assumed it would have been a case of just adding an asterisk wildcard to the directory segment. For example:
\path\*\local.properties
However this does not work and I am unsure how I would achieve such behaviour without explicitly declaring every reference that I need excluding. .
Documentation
# begins a comment line
The * and ? wildcards are supported.
A filespec is recursive unless prefixed by the \ character.
! negates a filespec (files that match the pattern are not ignored)
Extract from the documentation.
The documentation should more correctly read:
The * and ? wildcards are supported in the leaf name only.
That is, you can use something like these to select multiple files or multiple subdirectories, respectively, in a common parent:
/path/to/my/file/foo*.txt
/path/to/my/directories/temp*
What may work in your case--to ignore the same file in multiple directories--is just this:
foo*.txt
That is, specify a path-less name or glob pattern to ignore matching files throughout your tree. Unfortunately you have only those two options, local or global; you cannot use a relative path like this--it will not match any files!
my/file/foo*.txt
The global option is a practical one because .tfignore only affects unversioned files. Once you add a file to source control, changes to that file will be properly recognized. Furthermore, if you need to add an instance of an ignored name to source control, you can always go into TFS source control explorer and manually add it.
It seems this is now supported
As you see I edited tfignore in the root folder of the project such that any new branch will ignore its .vs folder when being examined for source control changes
\*\.vs
Directory/folder name wildcarding works for me in VS2019 Professional. For example if I put this in .tfignore:
*uncheckedToTFS
The above will ignore any folder named ending with "uncheckedToTFS", regardless of where the folder is (it doesn't have to be top level folder, can be many levels deep).