How can I rename files within a tar.gz file during extraction of the tar.gz file? - tar

Simplified problem: I have a tar.gz file: Example.tar.gz
When extracted, it produces 1 folder named 2014, and 3 files within that folder named as follows:
a1:00:test
a2:00:test
a3:00:test
Objective: When extraction is complete, I want the folder to still be named 2014, but have the files within the folder named as follows:
a1-00-test
a2-00-test
a3-00-test
These files are on an ext4 file system, but the extracted contents must be on a ntfs file system which is not possible due to the illegal file names (:).
These tar.gz files are very large (30 GB), and I have over 70 of them. It would not be possible to extract all the files on the ext4 file system and rename them due to limited storage.
tar -xzvf /media/joshua/output/Example.tar.gz -C /media/joshua/elements/
Above code works normally, but not when extracting to the ntfs file system (due to illegal file name).
tar --transform='s/:/-/'-xzvf /media/joshua/output/Example.tar.gz -C /media/joshua/elements/
Above code suggested by #Barmar changes the first : to a - but not the second :
I have tried suggestions here: how to rename files you put into a tar archive using linux 'tar' but I am new to Linux and having some trouble understanding the syntax, and applying it to my situation. Any help or general suggestions is appreciated!

Use the --transform option as described in the question you linked to:
tar --transform='s/:/-/g' -xzvf /media/joshua/output/Example.tar.gz -C /media/joshua/elements/
s/:/-/ replaces : with - in the filename.

Related

Can you use .\swagger-codegen-cli.jar generate to generate files that are not located in the same folder as swagger-codegen-cli?

Using java -jar .\swagger-codegen-cli.jar generate -l typescript-angular -i .\swagger.json, I can turn ".\swagger.json" into a typescript-angular file. However, the .json file must be in the same folder as the swagger-codegen-cli, which is impractical when you have many files to convert (You have to manually download and move every file). I've scoured the web, but haven't been able to find a practical solution. Is there a way to do this? Am I able to integrate swagger-codegen in such a way that I can "mass produce" files? I know I can change the output directory (-o) but that isn't what I'm looking for. Say I have a folder called "scripts" where I've got a lot of .json files. Am I able to call swagger-codegen-cli generate on any file inside of that folder, even if swagger-codegen-cli is located in a different folder?
The -i argument expects a fully-qualified or relative path to an OpenAPI file, or an URL where that file is located. That is, the file does not have to be in the same folder as the Swagger Codegen CLI.
Examples:
# The file is in the current working folder
-i swagger.json
-i ./swagger.json
# The file is in a subfolder of the current working folder
-i sub/folder/swagger.json
-i ./sub/folder/swagger.json
# Relative path to a file in another folder
-i ../../path/to/swagger.json
# Absolute path - Windows
-i C:/path/to/swagger.json
# Absolute path - *nix
-i ~/path/to/swagger.json
# URL
-i https://petstore.swagger.io/v2/swagger.json

Cannot open Zip file after creation with tar.exe

I'm trying to use Tar.exe in a Windows 10 command prompt to zip an entire folder and its subdirectories into a .zip file.
After reading different answers on here and this online help, so far I have the following:
tar.exe -cvzf "C:\Users\Me\Desktop\Output.zip" "C:\Users\Me\Desktop\MyFolder"
This appears to work within the command prompt (no errors and all files get listed). The .zip file gets created on the Desktop but when I try to open it by double-clicking on it, I get this error:
Windows cannot open the folder.
The Compressed (zipped) Folder 'C:\Users\Me\Desktop\MyFolder' is invalid.
I dragged the file into Notepad to see if there were any headers that might uncover the problem, but it looks like some kind of oriental affair...!
Can anyone advise what I've done wrong here please?
After much deliberation, I finally opted to use 7-Zip:
"C:\Program Files\7-Zip\7z.exe" a -tzip "C:\Users\Me\Desktop\MyFolder.zip" "C:\Users\Me\Desktop\MyFolder"
The destination machine without 7-Zip could still read the file, but more importantly, the speed of the zip creation was magnitudes faster than Windows's UI or the Tar function.

How to prevent dot underscore "._" files from being created after untar in tf.keras.utils.get_file()

tf.keras.utils.get_file(<directory_name>, <tgz path>, untar = True)
How do I prevent the untar from the above get_file() call from creating duplicate "._*" files?
I JUST figured it out for me, maybe it's the same for you. It wasn't the functions fault, but instead the files you are working with do in fact have files with "." as a prefix. For me, it was because I have a mac and in the image folders I zipped there are hidden mac files that have . as a prefix.
I just rezipped my files using this command:
tar --disable-copyfile -czf weeds_image.tgz weed_images

Pyinstaller adding data files

I'm struggling with pyinstaller. Whenever I build this specific script with a kivy GUI and a .kv file, and run the .exe after the build, I get a fatal error:
IOError: [Errno 2] No such file or directory: 'main.kv'
I've tried adding the .kv file, as well as a mdb and dsn file (for pypyodbc) using --add-data, but I get an error: unrecognized arguments: --add-data'main.kv'. (There were more --add-data arguments for the other files mentioned.)
Are there any solutions for this or maybe alternative methods?
As others (#Anson Chan, #schlimmchen) have said:
If you want to add some extra files, you should use Adding Data Files.
Two ways to implement
Command Line: add parameter to --add-data
Spec file: add parameter to datas=
Generated when running pyinstaller the first time.
Then later you can edit your *.spec file.
Then running pyinstaller will directly use your *.spec file.
Parameter Logic
Parameter in --add-data or datas=:
--add-data:
format: {source}{os_separator}{destination}
os_separator:
Windows: ;
Mac/Linux/Unix: :
source and destination
Logic:
source: path to single or multiple files, supporting glob syntax. Tells PyInstaller where to find the file(s).
destination
file or files: destination folder which will contain your source files at run time.
* NOTE: NOT the destination file name.
folder: destination folder path, which is RELATIVE to the destination root, NOT an absolute path.
Examples:
Single file: 'src/README.txt:.'
multiple files: '/mygame/sfx/*.mp3:sfx'
folder: '/mygame/data:data'
datas=
Format: list or tuple.
Examples: see the following.
added_files = [
( 'src/README.txt', '.' ),
( '/mygame/data', 'data' ),
( '/mygame/sfx/*.mp3', 'sfx' )
]
a = Analysis(...
datas = added_files,
...
)
Your case
For your (Windows OS) here is:
--add-data in command line
pyinstaller -F --add-data "main.kv;." yourtarget.py
OR:
datas= in yourtarget.spec file, see following:
a = Analysis(...
datas = ["main.kv", "."],
...
)
If you check pyinstaller -h for help, you can find --add-data option works like this [--add-data <SRC;DEST or SRC:DEST>]. So in your case try
pyinstaller -F --add-data "main.kv;main.kv" yourtarget.py
The solution is to run: pyi-makespec yourscript.py
Then edit the yourscript.spec script and add the files under datas in a= Analysis.
datas=[ ( '/pathToYourFile/main.kv', '.' )]
then run pyinstaller yourscript.spec
should be good after that.
Next -F or --onefile option is assumed when running pyinstaller.
Note that (MacOS Monterey, 12.2 here) the expected folder hierarchy w/in you .app file will be similar to this,
pyinstaller does not add files nor create necessary folders into any of the folders of this folder structure; at least not in any apparent way. You won't find them.
However, when the application runs, a temporary folder is used under /var/folders which is very different from the folder structure in point 1. above. print(os.path.dirname(__file__)) while running the application will reveal which exact temporary folder is used each time it runs. For convenience, let's call it my_app_tmp_folder i.e. your app runs under the folder /var/folder/my_app_tmp_folder
Then, pyinstaller adds data files or creates necessary directories w/in this temporary folder. In other words, when the application runs, all added files will be there and according to the specified folder structure (through --add-data option). print(os.listdir(os.path.dirname(__file__))) will show system and application needed files and folders.
Bottom line: Files specified w/ --add-data option will be visible w/in /var/folder/my_app_tmp_folder when running and not w/in the *.app folder.
Some useful links from documentation:
https://pyinstaller.readthedocs.io/en/stable/runtime-information.html#using-file
https://pyinstaller.readthedocs.io/en/stable/spec-files.html#adding-files-to-the-bundle
https://pyinstaller.readthedocs.io/en/stable/operating-mode.html#bundling-to-one-file
My application had this issue and a subsequent issue that is likely, if not inevitable.
1. --add-data for a kv file
Use --add-data as in the answer by crifan.
2. Kivy still can't find the file
Once PyInstaller has the kv file in the correct directory, Kivy still can't find the file.
Possible Symptoms:
GUI launches, but screen is black and empty.
An AttributeError error that depends on the application code.
AttributeError Examples:
This question
My own case:
AttributeError: 'NoneType' object has no attribute 'ids'
Fortunately, this answer solves the problem.

untar files into single directory preserving the directory structure

It is possible, and if so how, to use tar to extract a zip while preserving the directory structure? Currently it just extracts everything into a single directory using the path as the file name.
Example file name
Farlex.581429F59E1D8_1.5.1.9_neutral__wyegy4e46y996.main\Assets\buttons\add.png

Resources