I am working on a flex parser using flex 2.6.4 with the -s option specified, a particular start condition has the following patterns (I am trying to read everything to the next unescaped newline):
\\(.|\n)
[^\\\n]+
\n
Yet I get the warning: "-s option given but default rule can be matched"
I don't see any holes in the above pattern set, am I missing something or is this a flex error?
Your set of rules does not match a backslash at the end of the file.
Your first rule requires the backslash to be followed by something and the other ones don't match backslashes at all.
Related
I have windows 10 and installed GNU Make 4.3 Built for Windows32. This is my first time using GNU make and it gave me an error like this
.env:6: *** unterminated variable reference. Stop.
my .env file contain
POSTGRES_USER="postgres"
POSTGRES_PASS="<MY_DB_PASS>"
POSTGRES_DB="<MY_DB>"
PG_HOST="localhost"
PG_PORT="5432"
SECRET_KEY="<MY_SECRET_KEY>"
DEBUG=True
ALLOWED_HOSTS="localhost 127.0.0.1"
You may think that you're saving yourself by adding quotes. And, if this variable is parsed by a shell then you are.
But, make doesn't use quotes. The value of a variable is the complete contents to the right of the equal sign (after any initial whitespace is skipped). So for example:
POSTGRES_USER="postgres"
If parsed by the shell, the value of the POSTGRES_USER variable is postgres because the shell interprets the quotes. But make doesn't interpret quotes so the above line results in POSTGRES_USER make variable having the value "postgres" (including the quotes).
Now for your issue. Line 6 of your file is:
SECRET_KEY="<MY_SECRET_KEY>"
and you don't show us the text of your secret key.
First, this is wrong even in shell syntax: you must use single-quotes here not double quotes, and even that will not be right if your secret key contains single quotes itself; you'd have to escape that.
However that error means that in your secret key you have the character sequence $( or ${ which make interprets as starting a variable reference: since there is no close paren or brace you get this error.
The short answer is, there's no portable way to use the same file sourced by both make and the shell if the values of the variable assignments contain any sort of special character (including whitespace).
Usually people do something like base64 encode their secret keys, so that those special characters are not a problem.
I am writing an interpreter for assembly using lex and yacc. The problem is that I need to parse a word that will strictly be at the end of the file. I've read that there is an anchor $, which can help. However it doesn't work as I expected. I've wrote this in my lex file:
ABC$ {printf("QWERTY\n");}
The input file is:
ABC
without spaces or any other invisible symbols. So I expect the outputput to be QWERTY, however what I get is:
ABC
which I guess means that the program couldn't parse it. Then I thought, that $ might be a regular symbol in lex, so I changed the input file into this:
ABC$
So, if $ isn't a special symbol, then it will be parsed as a normal symbol, and the output will be QWERTY. This doesn't happen, the output is:
ABC$
The question is whether $ in lex is a normal symbol or special one.
In (f)lex, $ matches zero characters followed by a newline character.
That's different from many regex libraries where $ will match at the end of input. So if your file does not have a newline at the end, as your question indicates (assuming you consider newline to be an invisible character), it won't be matched.
As #sepp2k suggests in a comment, the pattern also won't be matched if the input file happens to use Windows line endings (which consist of the sequence \r\n), unless the generated flex file was compiled for Windows. So if you created the file on Windows and run the flex-generated scanner in a Unix environment, the \r will also cause the pattern to fail to match. In that case, you can use (f)lex's trailing context operator:
ABC/\r?\n { puts("Matched ABC at the end of a line"); }
See the flex documentation for patterns for a full description of the trailing context operator. (Search for "trailing context" on that page; it's roughly halfway down.) $ is exactly equivalent to /\n.
That still won't match ABC at the very end of the file. Matching strings at the very end of the file is a bit tricky, but it can be done with two patterns if it's ok to recognise the string other than at the end of the file, triggering a different action:
ABC/. { /* Do nothing. This ABC is not at the end of a line or the file */ }
ABC { puts("ABC recognised at the end of a line"); }
That works because the first pattern will match as long as there is some non-newline character following ABC. (. matches any character other than a newline. See the above link for details.) If you also need to work with Windows line endings, you'll need to modify the trailing context in the first pattern.
The following works directly in my Mac OS X terminal, creating a file with a few lines:
awk '!/^1499\||^1598\||^1599\||^1999\||^2298\||^2299\||^2403\|/' "#{working_path}" > "#{filtered_file_path}"
However, when I attempt to use it in Ruby on Rails using backticks, the resulting file is empty:
`awk '!/^1499\||^1598\||^1599\||^1999\||^2298\||^2299\||^2403\|/' "#{working_path}" > "#{filtered_file_path}"`
An awk with a simple regex works. For example:
`awk '!/SMITH/' "#{working_path}" > "#{filtered_file_path}"`
So, the issue appears to be with the escaped pipe characters.
Ideas?
Some background I should have provided:
The file I am processing is pipe-delimited. I am filtering out lines with certain codes that are in the first value on the line. So, the regex I am using is something like ^2298\|.
The other pipes in the expression in single quotes are regex OR operators.
"working_path" and "filtered_file_path" are Ruby variables.
I just figured it out. The backslash that is escaping the pipe characters also needs to be escaped. Not sure why there is a difference between the regular Terminal and Ruby, but there it is. The working version:
`awk '!/^1499\\||^1598\\||^1599\\||^1999\\||^2298\\||^2299\\||^2403\\|/' "#{working_path}" > "#{filtered_file_path}"`
After challenging my assumption that the problem was Ruby on Rails, the accepted answer here is what explained it:
Pipe symbol | in AWK field delimiter
Is it possible to cancel or skip a job in Jenkins due to special commit-message patterns? I thought the option "Excluded Commit comments" in the job configuration does this for me out of the box, like mentioned here. But no matter which regular expression i write in this field, the build is performed nevertheless.
For example:
I want to perform the build job only if the commit message includes the expression "release". So i write the regular expression [^(?:release)] in the Excluded Commit comments field. I thought if i commit a revision with, for example "test commit" the build-job does not perform, right? Is this the right way to do when not using a post-commit hook?
Jenkins Git plugin exposes you the environment variable GIT_COMMIT which of course, contains the current git commit hash.
Use [Jenkins Conditional Step] and build a step which execute the following bash shell:
echo "==========================="
if [ "git show $GIT_COMMIT | grep "your-pattern-here" == false ] ; then
echo "pattern failed";
exit 1
else
echo "ok"
fi
echo "==========================="
And mark that if the shell fails, fail the build.
Late reply but may help some one in future,
There is a plugin to skip build depending on git commit message, just include a [ci-skip] in the commit message junkin will skip the build
jenkins-ci-skip-plugin
TL;DR
To trigger builds only for commits with "release" word (case-insensitive) set this in the "Excluded Commit comments" field in job configuration:
(?i)(?s)(?!.*\brelease\b.*)^.*$
Better still, use a trigger phrase which is unlikely to be added to a commit message accidentally. For example, use "[ci build]":
(?i)(?s)(?!.*\[ci build\].*)^.*$
How does this work?
(?i) tells regex do do case-insensitive match. This is optional, but useful if you want to match "Release" and "RELEASE" as well as "release".
(?s) makes dot to match line-ends (aka dotall option), so that we look for matches within the entire commit message. By default dot doesn't match line-ends, so if there is no "release" keyword on one of the lines in the commit message, the pattern would match on that line, and commit would be incorrectly ignored by Jenkins. With dotall, we look at the entire commit message, ignoring any line ends.
(?!.*\brelease\b.*) - negative look-ahead pattern. Any match is discarded if this pattern is found within it. In this pattern:
.* matches anything before our trigger phrase and after it. We need this because of the way java regex matching works (quote from the tutorial):
myString.matches("regex") returns true or false depending whether the string can be matched entirely by the regular expression. It is important to remember that String.matches() only returns true if the entire string can be matched. In other words: "regex" is applied as if you had written "^regex$" with start and end of string anchors. This is different from most other regex libraries, where the "quick match test" method returns true if the regex can be matched anywhere in the string. If myString is abc then myString.matches("bc") returns false. bc matches abc, but ^bc$ (which is really being used here) does not.
\b makes sure that there is a word boundary before and after the keyword, as you probably don't want to match "unreleased" etc.
^.*$ is the actual matching pattern we are looking for. Note that ^ and $ match start of the string and end of the string, not the start/end of lines within that string. This is default behavior for java regex, unless multi-line mode is enabled. In other words, this pattern matches the entire commit message, because dotall mode was enabled by (?s) and dot matches newlines.
So matching algorithm would match the entire commit message, and then discard it or not depending on whether it finds negative look-ahead pattern anywhere in it.
Why your expression didn't work?
There were two problems with your suggested regex expression. First, you used incorrect regex syntax for excluding a pattern. Second, you didn't tell what your pattern should include, only what it should exclude. Therefore it would never match anything even if you used correct syntax. And because it doesn't match anything, then nothing is excluded from triggering jobs, i.e. any commits would trigger.
References
If you need more information, look for java.util.regex package which is used by Jenkins uses for matching. I used this online java regex tester to test my expressions. I've also found a nice tutorial - learned about (?m), (?s) and (?i) there.
Here is my sample string.
[echo] The SampleProject solution currently has 85% code coverage.
My desired output should be.
The SampleProject solution currently has 85% code coverage.
Btw, I had this out because I'm getting through the logs in my CI using Jenkins.
Any help? Thanks..
You can try substText parameter in BUILD_LOG_REGEX token to substitute the text matching your regex
New optional arg: ${BUILD_LOG_REGEX, regex, linesBefore, linesAfter, maxMatches, showTruncatedLines, substText} which allows substituting text for the matched regex. This is particularly useful when the text contains references to capture groups (i.e. $1, $2, etc.)
Using below will remove prefix [echo] from all of your logs ,
${BUILD_LOG_REGEX, regex="^\[echo] (.*)$", maxMatches=0, showTruncatedLines=false, substText="$1"}
\[[^\]]*\] will match the bit you want to remove. Just use a string replace function to replace that bit with an empty string.
Andrew has the right idea, but with Perl-style regex syntaxes (which includes Java's built-in regex engine), you can do even better:
str.replaceAll("\\[.*?\\]", "");
(i.e., use the matching expression \[.*?\]. The ? specifies minimal match: so it will finish matching upon the first ] found.)