I am a beginner of bash script. I just started to write a script where it checks the contents of b.txt can all be found in a.txt. (line by line preferably). My code is as following:
grep -Ffw b.txt a.txt
As you can see, I want to do fixed string instead of REGEX, I want to check everything from the b.txt file, because there are some strings inside the b.txt and I want to check if all of them exist in a.txt. And I also want to match the whole word only of course. So these are the requirements, however when I run this command it returns me an error says: grep: w: No such file or directory
I am thinking that maybe there are some limitations of the flags in bash? Sorry I am not really familiar with the language, didn't read much about the MAN page etc. If anyone could help me to solve the puzzle it would be appreciated :) In addition, i think if possible I would like to add a -q to surpress the output when there is a match also, right now I didn't add it in the example since it couldn't make it through with 3 flags even. So can anyone give me some hints here? Thanks in advance!
Hereby some explanation from the manpage:
OPTIONS
Generic Program Information
...
-F, --fixed-strings
Interpret PATTERNS as fixed strings, ...
-f FILE, --file=FILE
Obtain patterns from FILE, ...
-w, --word-regexp
Select only those lines ...
As you can see, the options -F and -w are indicated ending immediately (hence the comma in -F, and -w,), but the -f switch is followed by FILE, with means they belong together.
I you want to preserve the order Ffw, that's possible, but then you need to do something like:
grep -Ff b.txt -w a.txt
As mentioned by #kvantour, the solution is simply placing the -f before the b.txt file. grep -Fwf b.txt a.txt
Should have thought it when it says 'no such file or directory' as it was a clear indication that flags after the -f were treated as the path already.
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 3 months ago.
The community reviewed whether to reopen this question 3 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
how can I set default download location in youtube-dl so that everything that I download with youtube-dl goes into that default directory?
You need to use the -o switch with the Configuration file
Output on youtube-dl is handled with the --output or -o switch; pass it as an option, followed by the destination you want to save your downloads to:
youtube-dl -o "%USERPROFILE%\Desktop\%(title)s-%(id)s.%(ext)s" www.youtube.com/link/to/video
Note that -o has a dual function in that it also sets a template for how your output files will be named, using variables. In this example, it will output the title of the original downloaded video followed by the file extension, which is my personal preference. For all of the variables that can be used in a filename, have a look at the youtube-dl documentation here.
youtube-dl also allows use of a configuration file - a file that can be used to configure the switches you most frequently use so the program can pull them from there instead, saving you from having to explicitly call them each time you run it. This is what you'll need for the default download location that you're looking for. The configuration file can be used to set a default output destination so that you never have to explicitly set an output again.
To set up a configuration file for youtube-dl, assuming you have Windows:
In %APPDATA%\Roaming, create a youtube-dl folder if one doesn't already exist.
Inside that folder, create a plain text file named config.txt.
Place youtube-dl options in the file as you'd normally use them on the command line with youtube-dl, placing each one on a new line. For example, for the output switch, you'd use: -o %USERPROFILE%\Desktop. For more on the Configuration file, read the documentation on it here.
Overriding the Configuration file
Even when an option is configured in a configuration file, it can be overridden by calling it explicitly from the command line. So, if you have -o set in a configuration file to be the default location for downloads, but want to save downloads to somewhere else for a current job, simply calling -o on the command line will override the configuration file for the current run of the program only.
I find a way to directly download files in Downloads folder. I search for long hours. I copied my entire function then you can understand the context around. Here is my code it will maybe helpful for someone:
import os
def download_audio(request):
SAVE_PATH = '/'.join(os.getcwd().split('/')[:3]) + '/Downloads'
ydl_opts = {
'format': 'bestaudio/best',
'postprocessors': [{
'key': 'FFmpegExtractAudio',
'preferredcodec': 'mp3',
'preferredquality': '192',
}],
'outtmpl':SAVE_PATH + '/%(title)s.%(ext)s',
}
link = request.GET.get('video_url')
with youtube_dl.YoutubeDL(ydl_opts) as ydl:
ydl.download(["https://www.youtube.com/watch?v="+link])
Tell me if there is a problem.
According to the configuration documentation, you can configure youtube-dl with a global or user-specific configuration file:
You can configure youtube-dl by placing any supported command line option to a configuration file. On Linux and macOS, the system wide configuration file is located at /etc/youtube-dl.conf and the user wide configuration file at ~/.config/youtube-dl/config. On Windows, the user wide configuration file locations are %APPDATA%\youtube-dl\config.txt or C:\Users\<user name>\youtube-dl.conf. Note that by default configuration file may not exist so you may need to create it yourself.
On linux, this would be your user config file:
# Save all my videos to the Videos directory:
-o ~/Videos/%(title)s.%(ext)s
Depending on your needs, I think moving the file afterwards would be just as usefull:
--exec CMD Execute a command on the file after
downloading, similar to find's -exec
syntax. Example: --exec 'adb push {}
/sdcard/Music/ && rm {}'
By creating a function which will move the file
Here is the complete solution I use:
from youtube_dl import YoutubeDL
ydl_opts = {
'format': 'best',
'outtmpl': 'DIR-PATH-HERE%(title)s'+'.mp4',
'noplaylist': True,
'extract-audio': True,
}
video = "https://www.youtube.com/watch?v=SlPhMPnQ58k"
with YoutubeDL(ydl_opts) as ydl:
info_dict = ydl.extract_info(video, download=True)
video_url = info_dict.get("url", None)
video_title = info_dict.get('title', None)
video_length = info_dict.get('duration')
# print(video_title)
In command line or in the bash file use the double quotes, like this:
"%userprofile%/Desktop/DL/%(title)s-%(id)s.%(ext)s"
My bash command:
youtube-dl -c -i -f "mp4" -o "/home/Youtube_Downloads/%(title)s-%(id)s.%(ext)s" -a youtube_list
where 'youtube_list' - a raw text file with Youtube links, that goes line by line
I found there is an official comment by the authors about that specific question.
In the manual, here's what they say: (man youtube-dl):
How do I put downloads into a specific folder?
Use the -o to specify an output template, for example -o "/home/user/videos/%(title)s-%(id)s.%(ext)s". If you want this for all of your downloads, put the option into your configuration file.
That filename pattern is the default, as per the man as well:
The current default template is %(title)s-%(id)s.%(ext)s.
I agree it would be nice to have the output folder decoupled from the default template in case the default changes one day, but I'm guessing the authors must have had a reason to have it this way.
This is the EXACT ANOTHER USEFUL method to download your video into a desired DIRECTORY, and also keep the native filename of the download.
Decide where you want to create a configuration file.
Create a file, "youtube-dl.conf". You can create a youtube-dl.txt first it it's easier, but the file must be "youtube-dl.conf".
Here is a basic sample of a config file: this is where you want your downloads to go. This is all you have to put into the file. Where -o is the flag, %userprofile%/Desktop/DL/ is where I want the download to go, and %(title)s-%(id)s.%(ext)s is the command to keep the native filename.This is your config file below:
-o %userprofile%/Desktop/DL/%(title)s-%(id)s.%(ext)s
Options found here
Config here
The command paramaters: %program% -f %option% "%youtubelink%" "%MYCONFIG%" "%MYPATH%"
Batch File setup:
::Variables:
Set program="%USERPROFILE%\Desktop\YOUTUBE-DL\v20201209\youtube-dl.exe"
Set option=best
SET MYPATH="%USERPROFILE%\Desktop\YOUTUBE-DL\v20201209\config"
SET MYCONFIG="--config-location"
SET MYDLDIR="%USERPROFILE%\Desktop\DL"
SET INSTR='%%(title)s-%%(id)s.%%(ext)s'
MKDIR "%USERPROFILE%\Desktop\DL"
::Ask user for input.
Set /P youtubelink=[Past Link]:
:: For use of config file, for default download location.
%program% -f %option% "%youtubelink%" "%MYCONFIG%" "%MYPATH%"
:: There are many ways to accomplish this:
:: For Batch File, NOTE extra (%) character needed.
:: "%program%" -f "option" --merge-output-format mp4 -o "%MYDLDIR%"\%%(title)s-%%(id)s.%%(ext)s %youtubelink%
:: or this use of variable
:: "%program%" -f "option" --merge-output-format mp4 -o "%MYDLDIR%"\%INSTR% %youtubelink%
NOTE: The use of "quotes" when there are spaces in your variable options.
Final Message:
Create the config file, put it in a folder (directory) that you wish to refer to it. Go to your youtube-dl.exe file and pass the "parameters" listed above to it using your CMD or a batch file. Done. (contribution, and being kind)
I have the following line in my Inno Setup script:
SignTool=MySign cmd /c C:\SigningTools\signtool.exe sign /f C:\MyCert.pfx /p MyPassword $f
This works on my local machine.
I then commit my changes to our server and Jenkins will compile and make a build automatically. Jenkins does not work and I get the following error.
Error on line 43 in C:\Windows\TEMP\fxbundler8328922406343131203\images\win-exe.image\MyProgram.iss: Value of [Setup] section directive "SignTool" is invalid.
Compile aborted.
I have no idea what the issue is, I have tried numerous things but can't seem to figure it out. I would settle with learning some better options to output error messages with Inno Setup.
I have verired that MySign exists on the server's compiler IDE (http://www.jrsoftware.org/ishelp/index.php?topic=setup_signtool)
I have tried numerous variations of having $q surround file paths
I have verified that the file paths match the two machines
You need to define the SignTool in your call to the compiler via the /s switch.
Example: "/sMySign$q=sign_application.bat$q $f"
sign_application.bat receives the path of the file to sign as first parameter and calls signtool.exe as you've already tried.
Take a look here: http://www.jrsoftware.org/ishelp/index.php?topic=setupcmdline
Do not forget to Configure Sign Tools in the Inno Setup Compiler. I simply added signtool $p string.
In my case, the certificate has expired.
I found the following article usefull:
https://www.nextofwindows.com/how-to-check-a-pfx-certifications-expiry-date-on-windows
I opened a command prompt in the directory where my pfx file was and used this command to get details about the certificate:
certutil -dump "nameofcertfile.pfx"
Change nameofcertfile.pfx to your file name. You probably will be prompted for a password. Enter the password you used in your script (MyPassword in the OPs script). You may also copy/paste it.
NOTE: You will not see any character beeing typed while entering or pasting the password - so don't be confused.
I'm trying to set up a build in jenkins that reads a config file generated during a previous build (if it exists) as part of a shell build step. However, the variables I define in the config file don't seem to get put into the environment of the current shell, and I can't figure out why.
The first build step will pull in the config file from the upstream project.
The config file has just simple variable defs:
VAR_ONE=foo
VAR_TWO=bar
#etc...
The second build step is the shell, which looks like this:
if [ -f $WORKSPACE/build_config ];
then
source $WORKSPACE/build_config
echo $VAR_ONE
echo $VAR_TWO
fi
In the jenkins console output for the job I see:
+ '[' -f /var/lib/hudson-slave/workspace/build_config ']'
+ source /var/lib/hudson-slave/workspace/build_config
++ VAR_ONE=foo
++ VAR_TWO=bar
+ echo
+ echo
I don't know what the double plus means, maybe it's being exported into a different scope? If it is, why?
I haven't really looked into the EnvInject or EnvFile plugins yet. I plan to after this exercise in frustration, but I figured this would be a good question to ask anyway since I thought this would be possible to just bash out.
Anyone know what the heck is going on?
Just a shot in the dark, but have you tried to surround the variables with {} like in ${VAR_ONE} when using them?
About the double plus i just can guess either. I would say it is because of the variables declaration coming from the config file which is loaded into the current shell... But this is really just me guessing.
Hope this helps though.
I've just downloaded Powershell 2.0 and I'm using the ISE. In general I really like it but I am looking for a workaround on a gotcha. There are a lot of legacy commands which are interactive. For example xcopy will prompt the user by default if it is told to overwrite a file.
In the Powershell ISE this appears to hang
mkdir c:\tmp
cd c:\tmp
dir > tmp.txt
mkdir sub
xcopy .\tmp.txt sub # fine
xcopy .\tmp.txt sub # "hang" while it waits for a user response.
The second xcopy is prompting the user for permission to overwrite C:\tmp\sub\tmp.txt, but the prompt is not displayed in the ISE output window.
I can run this fine from cmd.exe but then what use is ISE? How do I know when I need which one?
In a nutshell, Interactive console applications are not supported in ISE (see link below). As a workaround, you can "prevent" copy-item from overwriting a file by checking first if the file exists using test-path.
http://blogs.msdn.com/powershell/archive/2009/02/04/console-application-non-support-in-the-ise.aspx
Why would you be using XCOPY from PowerShell ISE? Use Copy-Item instead:
Copy-Item -Path c:\tmp\tmp.txt -Destination c:\tmp\sub
It will overwrite any existing file without warning, unless the existing file is hidden, system, or read-only. If you want to overwrite those as well, you can add the -force parameter.
See the topic "Working with Files and Folders" in the PowerShell ISE help file for more info, or see all the commands at MSDN.