uci - how to revert all unstaged changes - openwrt

uci documentation says:
All "uci set", "uci add", "uci rename" and "uci delete" commands are staged into a temporary location and written to flash at once with "uci commit".
If I get it right, you first run some commands like the ones mentioned above and to make the changes be written to the configuration files you run uci commit. For example, let's say I have done the following changes...
root#OpenWrt:~# uci changes
network.vlan15.ifname='eth1.15'
network.vlan15.type='bridge'
network.vlan15.proto='static'
network.vlan15.netmask='255.255.255.0'
network.vlan15.ipaddr='192.168.10.0'
...but I don't want to continue and commit them. Is there an easy way to revert all staged changes and avoid doing it one by one?

This should be possible by the following command:
root#firlefanz:~# rm -rf /tmp/.uci/

There is a command to revert all staged changes
revert <config>[.<section>[.<option>]] Revert the given option, section or configuration file.
So, in your case, it should be
uci revert network.vlan15
See https://openwrt.org/docs/guide-user/base-system/uci

This one-liner should do the trick:
uci changes | sed -rn 's%^[+-]?([^=+-]*)([+-]?=.*|)$%\1%' | xargs -n 1 uci revert
tl;dr The sed command extracts the option names from the staged changes. The xargs command executes the revert command for every extracted option.
Now let's have a deep dive into everything:
uci changes prints the prepared changes which are then piped to the sed command.
The sed opton -r enables extended regular expressions and -n suppress automatic printing of pattern matches.
The sed command s is used to do a search and replace and % is used as separation character for the search and replace term.
The uci change lines have different formats.
Removed configuration options are prefixed with -.
Added configuration options are prefixed with +
Changed options don't have a prefix.
To match the prefixes [+-]? is used. A question mark means, that one of the characters in the square brackets can be matched optional.
The option name will be matched with the pattern [^=+-]*. This regex has the meaning of any number of characters as long as the character is not one of =+-.
It is inside round brackets to mark it as group to reuse it later.
The next pattern ([+-]?=.*|) is also a pattern group. There are two different groups spitted by the pipe.
The second part is the easy one and means no character at all. This happens when a uci option is deleted.
The fist part means that the character = can optional prepended with + or -. After the = can be one or more characters which is indicated by .*. =<value> happens on added configuration. The prepending of - or + indicates the value is removed from the list or added to the list if the option is a list.
In the replace pattern the whole line is replaced with the first group by its reference \1. In other words: only the option name is printed.
All the option names are then send to xargs. With option -n 1 xargs execuse uci revert <option_name> for every option_name send by sed.
This are some examples for the different formats of the uci changes output:
-a
+b='create new option with this value'
c='change an existing option to this value'
d+='appended to list'
e-='removed from list'
The extraced option names will be the following:
a
b
c
d
e
xargs -n 1 will then executed the following commands:
uci revert a
uci revert b
uci revert c
uci revert d
uci revert e
This is the whole magic of the one-liner.

I didn't find a uci command to revert all uncommitted changes, but you can probably parse the output of the uci changes command with some shell scripting to achieve the desired result. Here is an example script:
#!/bin/ash
# uci-revert-all.sh
# Revert all uncommitted uci changes
# Iterate over changed settings
# Each line has the form of an equation, e.g. parameter=value
for setting in $(uci changes); do
# Extract parameter from equation
parameter=$(echo ${setting} | grep -o '^\(\w\|[._-]\)\+')
# Display a status message
echo "Reverting: ${parameter}"
# Revert the setting for the given parameter
uci revert "${parameter}"
done
A simpler alternative might be to use the uci revert <config> syntax, e.g.:
#!/bin/ash
# uci-revert-all.sh
# Revert all uncommitted uci changes
for config in /etc/config/*; do
uci revert $(basename ${config})
done
Both of these approaches worked well for me on a router running LEDE 4.

Here's another short one-liner to revert ALL unstaged changes (as per the question):
for i in /etc/config/* ; do uci revert ${i##*/} ; done
(FYI, this uses posix parameter expansion's "Remove Largest Prefix Pattern".)

Related

correct locale setting for devnagari unicode text

The following output is wrong. There should be only 1 word returned insted of 2
$ echo 'उद्योजकता' | grep -o -E '\w+'
उद
योजकता
I have been told this is due to locale setting. I have checked it on 2 different servers with 2 different O/S and the results are the same.
Ubuntu
$ locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=
AWS EC2
# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
I am not sure which locale setting should be selected to get the Devnagari unicode text to break only at space.
Edited to add:
You can use something like this instead if the grep on your machine supports Perl-compatible regular expressions (PCRE). This should be the case e.g., on Amazon Linux.
echo 'उद्योजकता' |grep -o -P '[\w\pL\pM]+'
which will match "word" characters OR "Letter" code points OR "Mark" code points,
OR
echo 'उद्योजकता' |grep -o -P '(*UCP)[\w\pM]+'
which will enable unicode character property (UCP) matching so that \w matches all letters and numbers no matter the script, but you must still include \pM in the pattern because \w simply does not match "Mark" points in grep -- they are not alphanumeric code points.
Be careful with the above! I don't know Devanagari script so I don't know if it's appropriate or not to consider all such "Mark" characters as being part of a word for your purposes. It might be that a narrower set such as \Mn (for non-spacing marks) is more appropriate for your needs, or perhaps there are only a few specific points to include in which case you'd need to select them individually in your pattern.
In the old days, \w just meant [A-Za-z0-9_]. It was oriented around ASCII, and C code. Today, in typical interpretations, it still means "alphanumeric and underscore", which can vary depending on locale.
You say that the output is "wrong", but I'm afraid "wrong" is dependent on which regular expression engine you are using. So even though you are using "grep", the question is which grep, on which OS, etc., etc.
Your input contains 0x094d, as far as I can tell, which is not a "Letter", according to the unicode character definition (at least, not per the link above). It is a "Mark".
There is a Unicode "document" (recommendation) which includes how an engine might define "\w" to be Unicode-smart, and indeed it suggests to include Mark codepoints in the match. So your expectation is natural in that sense. However, you can see from the same link that there is no way to do this and also to be strictly POSIX-compliant at the same time, which lots of regex engines want to do.
Wikipedia indicates that there are some engines which support the Unicode property definitions, but in general, grep isn't going to do it. I'm not familiar enough with those engines (ruby, etc) to say exactly how you should attempt the same thing on command line as you are trying to do with grep.
The macOS (11.2.3) man page for grep has this note at the bottom:
BUGS
The grep utility does not normalize Unicode input, so a pattern containing composed characters will not match decomposed input, and vice versa.
If you are okay with a solution for Devnagari text alone, these would help. As per wikipedia, the Unicode range is U+0900 to U+097f. So, if your shell supports $'...' form, you can use:
$ echo 'उद्योजकता' | grep -oE $'[\u0900-\u097f]+'
उद्योजकता
If PCRE is available:
$ echo 'उद्योजकता' | grep -oP '[\x{900}-\x{97f}]+'
उद्योजकता
Use ripgrep for better Unicode support.
$ echo 'उद्योजकता' | rg -o '\w+'
उद्योजकता

Git diff: How to ignore starting space of an empty line?

I'm an iOS developer, when I press enter, xcode automatically indent the new line with 4 spaces, that's convenient for developing.
But when it comes to using git diff, every empty line will be marked with red color.
This can be annoying in team development.
So how to deal with it?
Thanks in advance!
Use this when using diff
git diff -w // (--ignore-all-space)
You can create an alias for this so you will not have to type it every time.
git config --global alias.NAME 'diff --ignore-space-change'
git diff
-b / --ignore-space-change
Ignore changes in amount of whitespace.
This ignores whitespace at line end, and considers all other sequences of one or more whitespace characters to be equivalent.
-w / --ignore-all-space
Ignore whitespace when comparing lines.
This ignores differences even if one line has whitespace where the other line has none.
--ignore-blank-lines
Ignore changes whose lines are all blank.
With Git 2.25 (Q1 2020), three+ years later, the "diff" machinery learned not to lose added/removed blank lines in the context when --ignore-blank-lines and --function-context are used at the same time.
So your intermediate 4 empty lines won't show up.
But in the context of functions, they might.
See commit 0bb313a (05 Dec 2019) by René Scharfe (rscharfe).
(Merged by Junio C Hamano -- gitster -- in commit f0070a7, 16 Dec 2019)
xdiff: unignore changes in function context
Signed-off-by: René Scharfe
Changes involving only blank lines are hidden with --ignore-blank-lines, unless they appear in the context lines of other changes.
This is handled by xdl_get_hunk() for context added by --inter-hunk-context, -u and -U.
Function context for -W and --function-context added by xdl_emit_diff() doesn't pay attention to such ignored changes; it relies fully on xdl_get_hunk() and shows just the post-image of ignored changes appearing in function context.
That's inconsistent and confusing.
Improve the result of using --ignore-blank-lines and --function-context together by fully showing ignored changes if they happen to fall within function context.

Jenkins: avoid build due to commit message

Is it possible to cancel or skip a job in Jenkins due to special commit-message patterns? I thought the option "Excluded Commit comments" in the job configuration does this for me out of the box, like mentioned here. But no matter which regular expression i write in this field, the build is performed nevertheless.
For example:
I want to perform the build job only if the commit message includes the expression "release". So i write the regular expression [^(?:release)] in the Excluded Commit comments field. I thought if i commit a revision with, for example "test commit" the build-job does not perform, right? Is this the right way to do when not using a post-commit hook?
Jenkins Git plugin exposes you the environment variable GIT_COMMIT which of course, contains the current git commit hash.
Use [Jenkins Conditional Step] and build a step which execute the following bash shell:
echo "==========================="
if [ "git show $GIT_COMMIT | grep "your-pattern-here" == false ] ; then
echo "pattern failed";
exit 1
else
echo "ok"
fi
echo "==========================="
And mark that if the shell fails, fail the build.
Late reply but may help some one in future,
There is a plugin to skip build depending on git commit message, just include a [ci-skip] in the commit message junkin will skip the build
jenkins-ci-skip-plugin
TL;DR
To trigger builds only for commits with "release" word (case-insensitive) set this in the "Excluded Commit comments" field in job configuration:
(?i)(?s)(?!.*\brelease\b.*)^.*$
Better still, use a trigger phrase which is unlikely to be added to a commit message accidentally. For example, use "[ci build]":
(?i)(?s)(?!.*\[ci build\].*)^.*$
How does this work?
(?i) tells regex do do case-insensitive match. This is optional, but useful if you want to match "Release" and "RELEASE" as well as "release".
(?s) makes dot to match line-ends (aka dotall option), so that we look for matches within the entire commit message. By default dot doesn't match line-ends, so if there is no "release" keyword on one of the lines in the commit message, the pattern would match on that line, and commit would be incorrectly ignored by Jenkins. With dotall, we look at the entire commit message, ignoring any line ends.
(?!.*\brelease\b.*) - negative look-ahead pattern. Any match is discarded if this pattern is found within it. In this pattern:
.* matches anything before our trigger phrase and after it. We need this because of the way java regex matching works (quote from the tutorial):
myString.matches("regex") returns true or false depending whether the string can be matched entirely by the regular expression. It is important to remember that String.matches() only returns true if the entire string can be matched. In other words: "regex" is applied as if you had written "^regex$" with start and end of string anchors. This is different from most other regex libraries, where the "quick match test" method returns true if the regex can be matched anywhere in the string. If myString is abc then myString.matches("bc") returns false. bc matches abc, but ^bc$ (which is really being used here) does not.
\b makes sure that there is a word boundary before and after the keyword, as you probably don't want to match "unreleased" etc.
^.*$ is the actual matching pattern we are looking for. Note that ^ and $ match start of the string and end of the string, not the start/end of lines within that string. This is default behavior for java regex, unless multi-line mode is enabled. In other words, this pattern matches the entire commit message, because dotall mode was enabled by (?s) and dot matches newlines.
So matching algorithm would match the entire commit message, and then discard it or not depending on whether it finds negative look-ahead pattern anywhere in it.
Why your expression didn't work?
There were two problems with your suggested regex expression. First, you used incorrect regex syntax for excluding a pattern. Second, you didn't tell what your pattern should include, only what it should exclude. Therefore it would never match anything even if you used correct syntax. And because it doesn't match anything, then nothing is excluded from triggering jobs, i.e. any commits would trigger.
References
If you need more information, look for java.util.regex package which is used by Jenkins uses for matching. I used this online java regex tester to test my expressions. I've also found a nice tutorial - learned about (?m), (?s) and (?i) there.

How to replace a path with another path in sed? [duplicate]

This question already has answers here:
Using different delimiters in sed commands and range addresses
(3 answers)
Closed 9 months ago.
I have a csh script (although I can change languages if it has any relevance) where I have to:
sed s/AAA/BBB/ file
The problem is that AAA and BBB are paths, and so contain '/'. AAA is fixed, so I can say:
sed s/\\\/A\\\/A\\\A/BBB/ file
However, BBB is based on variables, including $PWD. How do I escape the '/' in $PWD?
OR is there some other way I should be doing this entirely?
sed can use any separator instead of / in the s command. Just use something that is not encountered in your paths:
s+AAA+BBB+
and so on.
Alternatively (and if you don't want to guess), you can pre-process your path with sed to escape the slashes:
pwdesc=$(echo $PWD | sed 's_/_\\/_g')
and then do what you need with $pwdesc.
In circumstances where the replacement string or pattern string contain slashes, you can make use of the fact that GNU sed allows an alternative delimiter for the substitute command. Common choices for the delimiter are the pipe character | or the hash # - the best choice of delimiting character will often depend on the type of file being processed. In your case you can try
sed -i 's#/path/to/AAA#/path/to/BBB#g' your_file
Note: The g after last # is to change all occurrences in file if you want to change first ouccurence do not use g
sed -i "s|$fileWithPath|HAHA|g" file
EDIT 1
sed -i 's|path/to/foo|path/to/bar|g' file
Using csh for serious scripting is usually not recommended. However, that is tangential to the issue at hand.
You're probably after something like:
sed -e "s=$oldpath=$newpath="
where the shell variable $oldpath contains the value to be replaced and $newpath contains the replacement, and it is assumed that neither variable contains an equals sign. That is, you're allowed to choose the delimiter on pattern, and avoiding the usual / delimiter avoids problems with slashes in pathnames. If you think = might appear in your file names, choose something less likely to appear, such as control-A or control-G.
In my case the below method works.
sed -i 's/playstation/PS4/' input.txt
Can be also be written as
sed -i 's+playstation+PS4+' input.txt
sed : is stream editor
-i : Allows to edit the source file
+: Is delimiter.
I hope the above information works for you 😃.
You can use parenthesis expansion ${i/p/r} to escape the slashes.
In this case ${i//p/r} for escaping all occurrences.
$p1=${p1//\//\\/}
$p2=${p2//\//\\/}
sed s/$p1/$p2/ file
Or, more concise, in one line sed s/${p1//\//\\/}/${p2//\//\\/}/ file
The two fist slashes // are a separator in parenthesis expansion saying we are matching all occurrences, then \/ is for escaping the slash in the search template, the / as a second separator in the expansion, and then \\/ is the replacement, in witch the backslash must be escaped.
We just needed to get the /h/ network path references out of the path. if we pointed them back to the /c/ drive they would map to non-existant directories but resolve quickly. In my .bashrc I used
PATH=`echo $PATH | sed -e "s+/h/+/c/+g"`

Find stored procedures not referenced in source code

I am trying to clean up a legacy database by dropping all procedures that are not used by the application. Using grep, I have been able to determine that a single procedure does not occur in the source code. Is there a way to do this for all of the procedures at once?
UPDATE: While using -E "proc1|proc2" produces an output of all lines in all files which match either pattern, this is not very useful. The legacy database has 2000+ procedures.
I tried to use the -o option thinking that I could use its output as the pattern for an inverse search on the original pattern. However, I found that there is no output when you use the -o option with more than one pattern.
Any other ideas?
UPDATE: After further experimenting, I found that it is the combination of the -i and -o options which are preventing the output. Unfortunately, I need a case insensitive search in this context.
feed the list of stored procedures to egrep separated by "|"
or:
for stored_proc in $stored_procs
do
grep $stored_proc $source_file
done
I've had to do this in the past as well. Don't forget about any procs that may be called from other procs.
If you are using SQL Server you can use this:
SELECT name,
text
FROM sysobjects A
JOIN syscomments B
ON A.id = B.id
WHERE xtype = 'P'
AND text LIKE '%< sproc name >%'
I get output under the circumstances described in your edit:
$ echo "aaaproc1bbb" | grep -Eo 'proc1|proc2'
proc1
$ echo $?
0
$ echo "aaabbb" | grep -Eo 'proc1|proc2'
$ echo $?
1
The exit code shows if there was no match.
You might also find these options to grep useful (-L may be specific to GNU grep):
-c, --count
Suppress normal output; instead print a count of matching lines
for each input file. With the -v, --invert-match option (see
below), count non-matching lines. (-c is specified by POSIX.)
-L, --files-without-match
Suppress normal output; instead print the name of each input
file from which no output would normally have been printed. The
scanning will stop on the first match.
-l, --files-with-matches
Suppress normal output; instead print the name of each input
file from which output would normally have been printed. The
scanning will stop on the first match. (-l is specified by
POSIX.)
-q, --quiet, --silent
Quiet; do not write anything to standard output. Exit
immediately with zero status if any match is found, even if an
error was detected. Also see the -s or --no-messages option.
(-q is specified by POSIX.)
Sorry for quoting the man page at you, but sometimes it helps to screen things a bit.
Edit:
For a list of filenames that do not contain any of the procedures (case insensitive):
grep -EiL 'proc1|proc2' *
For a list of filenames that contain any of the procedures (case insensitive):
grep -Eil 'proc1|proc2' *
To list the files and show the match (case insensitive):
grep -Eio 'proc1|proc2' *
Start with your list of procedure names. For easy re-use later, sort them and make them lowercase, like so:
tr "[:upper:]" "[:lower:]" < list_of_procedures | sort > sorted_list_o_procs
... now you have a sorted list of the procedure names. Sounds like you're already using gnu grep, so you've got the -o option.
fgrep -o -i -f sorted_list_o_procs source1 source2 ... > list_of_used_procs
Note the use of fgrep: these aren't regexps, really, so why treat them as such. Hopefully you will also find that this magically corrects your output issues ;). Now you have an ugly list of the used procedures. Let's clean them up as we did the orginal list above.
tr "[:upper:]" "[:lower:]" < list_of_used_procs | sort -u > short_list
Now you have a short list of the used procedures. Let's find the ones in the original list that aren't in the short list.
fgrep -v -f short_list sorted_list_o_procs
... and there they are.

Resources