Whats the most clever (or preferably the right way) to handle different file system paths?
i.e
Windows
cd("Folder\\\\file.jl") #this becomes "\\"
Unix
cd("Folder/file.jl")
The only solution that comes to mind is declairing a global variable at runtime
#windows_only global slash = "\\"
#linux_only global slash = "/"
but that looks awfully dodgy
joinpath("Folder","file.jl") should do the trick.
From the REPL ?joinpath gives:
joinpath(parts...) -> AbstractString
Join path components into a full path. If some argument is an absolute
path, then prior components are dropped.
So, if needed more than two path parts can be joined as in:
joinpath("dir1","dir2","file1") == "dir1/dir2/file1" (on a Linux machine)
Related
I want to read in a list of files (inc path) from either a spreadsheet or a text file for some downstream processing. The list has been generated as a log from another process and the path includes a 2 digit year folder followed by a project number folder as follows:
\\servername\projects\19\1901001\project files\filetobeprocessed.abc
The problem is as soon as the above string is read in, it is interpreted as
\\servername\\projects\x019\x01901001\\project files\x0ciletobeprocessed.abc
Which then means that I cannot use the path to access the file.
Assigning the path string to a variable, I have tried:
thePath = repr(pathreadfromfile)
After assigning the path string I have tried fixing the string using
thePath.replace('\x0','\\')
thePath.replace('\\x0','\\')
thePath.replace(r'\x0','\\')
Nothing seems to fix the path so that it can be used to open the file.
I can't find anything in either python or Ironpython that suggests a fix for this programatically. I know that you can fix this is the path is known within the code by using r'' to use raw text to create the path.
Any help appreciated
Obviously, the backslash \ is interpreted as an escape character.
For a really simple solution, hopefully the simplest, I would suggest using forward slash / for all your path separators instead of backslash.
If you really need the backslash somewhere further down the road, you can replace them back again.
I'm writing an Apache Beam pipeline in python and trying to load multiple text files but encounter an error when using the pattern match. When I pass in an exact filename, the pipeline runs correctly.
For example:
files = p | 'Read' >> ReadFromText('lyrics.txt')
However, when using pattern match an error occurs:
files = p | 'Read' >> ReadFromText('lyrics*')
IOError: No files found based on the file pattern
In this example, I have several files that start with "lyrics".
I've tried many different pattern types but haven't had any success with anything except passing the complete file name. Is there a different way to apply pattern match in this case?
Updated with answer
If you're on Windows don't forget to use a backslash instead of forward slash when specifying directories. For example: ReadFromText('.\lyrics*')
This looks like a bug. I've filed https://issues.apache.org/jira/browse/BEAM-7560. In the meantime, try an absolute path or ReadFromText('./lyrics*').
How can I retrieve the path to the go binary in a container that does not have which programatically?
One option would be to execute which go as follows:
bytes, err := exec.Command("which", "go").Output()
However, I would like to not depend on which being available. Does go provide any in-built mechanism to retrieve this and if not, what is the alternative apart from having the user pass in the path themselves?
From the man page of which:
Which takes one or more arguments. For each of its arguments it prints to stdout the full path of the executables that would have been executed when this argument had been entered at the shell prompt. It does this by searching for an executable or script in the directories listed in the environment variable PATH using the same algorithm as bash(1).
Go's os/exec.LookPath function is very close to this functionality:
LookPath searches for an executable named file in the directories named by the PATH environment variable. If file contains a slash, it is tried directly and the PATH is not consulted. The result may be an absolute path or a path relative to the current directory.
Use path/filepath.Abs if you need a guaranteed absolute path.
I don't expect this to be the best answer, but it is one that I just found. I was hoping for more of a go-specific one, but in the meantime type in linux is a default built-in one available in bash and sh (alpine).
You can test this yourself by running type type which yields:
type is a shell builtin
The usage in go would look like this:
b, err := exec.Command("type", "go").Output()
if err != nil {
/* 'type' is not available on the O/S */
}
goPath := strings.TrimPrefix(strings.TrimSuffix(string(b), "\n"), "go is ")
The reason the Trim functions are required is because the output would look like this:
go is /usr/local/go/bin/go\n
This isn't the nicest way of going about it, but it works.
I use flume-1.8.0.
On the document, it says that I cannot set the directory pattern.
(Regular expression (and not file system patterns) can be used for filename only.)
But I have to set the directory pattern to get log from other system which controlled by other team.
Is there some solution to set directory path like /dir/201801/0101.log, /dir/201802/0001.log, ... ?
Use something like this for the file groups with file patterns i.e use the Regex ASCII pattern see https://en.wikipedia.org/wiki/Regular_expression for more details
a1.sources.r1.filegroups.f2 = /path/to/files/with/pattern/databundle_cnt_[0-9]{4}-[0-9]{2}-[0-9]{2}.csv
In your case I will advise
a1.sources.r1.filegroups.f2 = /dir/[0-9]{6}/[0-9]{4}.log
I want to extract the directory name from a windows path. The windows path is a string, something like the following:
"c:\\some\path\name"
when I do the following:
File.dirname("c:\\some\\path\\name")
The result is
"."
If I run this on the unix path it works fine
File.dirname("/some/path/name") => "/some/path"
Do I need to somehow set the FILE::ALT_SEPARATOR? I have tried different variations of the path to no avail.
One solution I found is to replace all the backslashes with a forward slash. This works decently well. However there still must be a better solution.
File.dirname("c:\\some\\path\\name".gsub('\\', '/')).gsub('/', '\\')
=> "c:\\some\\path"
I sub the backslashes back in after the dirname call in order to keep the representation consistent.
The recommended way is to always use unix-type forward slash for path separators in Ruby code. Even if you use it on Windows OS, they will be correctly mapped internally to its backslash path separators.
If the backslash comes from the user input, then you need to detect whether the OS is such that allows a backslash in a file name (e.g., Windows does not, Unix does). Then if the backslash is not allowed, then you should convert them to forward slash during validation. In the Ruby code, keep all separators as forward slash. So, you should not be caring about backslashes when using commands such as File.dirname.