I was running vagrant provision today and it failed with the following error:
Package configuration┌──────────────────────────┤ Configuring grub-pc ├──────────────────────────┐│││
The GRUB boot loader was previously installed to a disk that is no
││ longer present, or whose unique identifier has changed for some reason.
││ It is important to make sure that the installed GRUB core image stays in
││ sync with GRUB modules and grub.cfg. Please check again to make sure
││ that GRUB is written to the appropriate boot devices.
││││ If you're unsure which drive is designated as boot drive by your BIOS,
││ it is often a good idea to install GRUB to all of them.
││││ Note: it is possible to install GRUB to partition boot records as well,
││ and some appropriate partitions are offered here. However, this forces
││ GRUB to use the blocklist mechanism, which makes it less reliable, and
││ therefore is not recommended.
││││<Ok>│││└───────────────────────────────────────────────────────────────────────────┘
But then, to my surprise, it started printing a stack trace in a very strange encoding...
STDERR: F▒␋┌␊␍ ├⎺ ⎺⎻␊┼ ├␊⎼└␋┼▒┌.␍␊␉␌⎺┼°: ┬␋⎻├▒␋┌ ⎺┤├⎻┤├ ├␊ ▒␉⎺┴␊ ␊⎼⎼⎺⎼⎽, ±␋┴␋┼± ┤⎻!
␍⎻┐±: ␊⎼⎼⎺⎼ ⎻⎼⎺␌␊⎽⎽␋┼± ±⎼┤␉-⎻␌ (--␌⎺┼°␋±┤⎼␊):
⎽┤␉⎻⎼⎺␌␊⎽⎽ ␋┼⎽├▒┌┌␊␍ ⎻⎺⎽├-␋┼⎽├▒┌┌▒├␋⎺┼ ⎽␌⎼␋⎻├ ⎼␊├┤⎼┼␊␍ ␊⎼⎼⎺⎼ ␊│␋├ ⎽├▒├┤⎽ 255
E⎼⎼⎺⎼⎽ ┬␊⎼␊ ␊┼␌⎺┤┼├␊⎼␊␍ ┬␋┌␊ ⎻⎼⎺␌␊⎽⎽␋┼±:
±⎼┤␉-⎻␌
E: S┤␉-⎻⎼⎺␌␊⎽⎽ /┤⎽⎼/␉␋┼/␍⎻┐± ⎼␊├┤⎼┼␊␍ ▒┼ ␊⎼⎼⎺⎼ ␌⎺␍␊ (1)
---- E┼␍ ⎺┤├⎻┤├ ⎺° ▒⎻├-±␊├ ┤⎻±⎼▒␍␊ -≤ ----
R▒┼ ▒⎻├-±␊├ ┤⎻±⎼▒␍␊ -≤ ⎼␊├┤⎼┼␊␍ 100
R␊⎽⎺┤⎼␌␊ D␊␌┌▒⎼▒├␋⎺┼:
---------------------
# I┼ /┴▒⎼/␌␊°/␌▒␌␊/␌⎺⎺┐␉⎺⎺┐⎽/␌⎺└└⎺┼-⎽⎺°├┬▒⎼␊/⎼␊␌␋⎻␊⎽/␍␊°▒┤┌├.⎼␉
49: ␊│␊␌┤├␊ "▒⎻├-±␊├-┤⎻±⎼▒␍␊-≤" ␍⎺
50: ␌⎺└└▒┼␍ "▒⎻├-±␊├ ┤⎻±⎼▒␍␊ -≤"
51: ␊┼␍
C⎺└⎻␋┌␊␍ R␊⎽⎺┤⎼␌␊:
------------------
# D␊␌┌▒⎼␊␍ ␋┼ /┴▒⎼/␌␊°/␌▒␌␊/␌⎺⎺┐␉⎺⎺┐⎽/␌⎺└└⎺┼-⎽⎺°├┬▒⎼␊/⎼␊␌␋⎻␊⎽/␍␊°▒┤┌├.⎼␉:49:␋┼ ◆°⎼⎺└_°␋┌␊'
␊│␊␌┤├␊("▒⎻├-±␊├-┤⎻±⎼▒␍␊-≤") ␍⎺
⎼␊├┤⎼┼⎽ 0
␌⎺└└▒┼␍ "▒⎻├-±␊├ ┤⎻±⎼▒␍␊ -≤"
▒␌├␋⎺┼ "⎼┤┼"
⎼␊␌␋⎻␊_┼▒└␊ "␍␊°▒┤┌├"
⎼␊├⎼≤_␍␊┌▒≤ 2
␌⎺⎺┐␉⎺⎺┐_┼▒└␊ "␌⎺└└⎺┼-⎽⎺°├┬▒⎼␊"
␉▒␌┐┤⎻ 5
⎼␊├⎼␋␊⎽ 0
␊┼␍
And the terminal prompt returned...in that same encoding! All my input was converted to that encoding (capital letters appeared as usual though).
My question is: why does Vagrant change the encoding of my terminal window, and does it have something to do with the GRUB error?
I'm on a Macbook Pro.
If you are using apt-get upgrade/dist-upgrade, try it:
sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade
I wrote a function for it. Just copy paste it into your provision script:
function aptget {
# to prevent encoding problem
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
$#
}
aptget update
aptget upgrade
Note: I had posted this as a comment to #gustavohenrique's answer but it didn't had a good format as a comment. So, I've posted this as an answer although it's a duplicate answer. It's more of a supplement. Sorry for the disruption.
Related
I am new to EDK2.
For porting ekd2 firmware to a new ARM64 platform, it would be good to first get a minimum edk2 port which can run UEFI Shell at least, improvements can be added gradually based on that.
It seems that the first step is rather steep, e.g., how to determine a minimal set of "items" in .dsc and .fdf file for a platform? In my case, I would like to build the .fd for my platform and treat it as BL33 of TF-A, effectively I would like to build an edk2 firmware to replace u-boot.
It seems that such a guide is hard to find on the web. I found a old version of edk2 which contains some instructions, but apparently they are obsolete (not exist in latest master branch, while can be found in UDK branches such as UDK2014), and I am not sure why those documents are removed from master branch.
Currently I can build .fd for FVP (edk2-platforms/Platform/ARM/VExpressPkg/ArmVExpress-FVP-AArch64.dsc), and it seems that the build output FVP_AARCH64_EFI.fd is supposed to be treated as BL33. Theoretically this could be a prototype for my new ARM64 platform, but to me it's too complex to start with: the firmware is about 2.5MiB in size (as compare to 500K of u-boot), so I guess it's far from a "minimum" version. but it's hard to figure out what features to be removed (and how).
I am wondering if there is a detailed guide on such topic...
After 1 month of trial and error, today I managed to bring my ARM64 platform into a UEFI Shell environment. I treat it as my 1st milestone on the EDK2 journey. Below I will try to summarize the steps I took so far, as a tentative answer to my question above. Guidance/corrections/comments are welcomed.
Get familiar with UEFI/PI spec and EDK2 implementation by reading books/specs/articles. Well, UEFI/PI specs are thousands of pages long...how to start? My main reading list is:
"Beyond Bios--Developing with the Unified Extensible Firmware Interface", 3rd ed, by Vincent Zimmer, et al. As the authors explained, the book is a kind of high level summary of the thousands-paged specs. And I find that the book is well organized for a new comer to get familiar with various UEFI related concepts. The main purpose of the 1st read (before playing with edk2 code base) is to get familiar with concepts and architectural ideas, not the details yet. Related sections need to be consulted later when reading EDK2 implementations.
EDK2 specs, including:
EDKII User Manual
EDKII Build Specification
EDKII DSC/FDF/DEC/INF File Specification
Various articles on the web...
Get a reference platform which can correctly boot a FD image built from latest EDK2 source, and play with the boot manager and Shell environment a bit. In my case, I chose RPi4B. For me, this is very important, as the reference platform serves as a handrail during the whole process, that whenever I encounter bugs or have doubts, I check the source/log of the reference platform. This solves most of the problems I encountered. Btw, always generating "build log" and "build report" for both reference platform and the target platform, as the two files contains very detailed information for comparison and check. Consult the EDK2 build spec on how to generate these two files during build.
I use the following script to build for RPi4B platform:
#!/bin/bash
# https://github.com/tianocore/edk2-platforms#how-to-build-linux-environment
export WORKSPACE=/home/bruin/work/tianocore
export PACKAGES_PATH=$WORKSPACE/edk2:$WORKSPACE/edk2-platforms:$WORKSPACE/edk2-non-osi
pushd $WORKSPACE
rm -rf ./Build/RPi4
source edk2/edksetup.sh
echo "Building BaseTools..."
make -C edk2/BaseTools all
#sudo apt install acpica-tools # iasl
# pip install antlr4-python3-runtime # -Y EXECUTION_ORDER
echo "Building firmware for Pi4B..."
GCC5_AARCH64_PREFIX=aarch64-none-linux-gnu- build \
-n 4 \
-a AARCH64 \
-p Platform/RaspberryPi/RPi4/RPi4.dsc \
-t GCC5 \
-b NOOPT \
-v -d 9 -j RPi4-build.log \
-y RPi4-build-report.txt \
-Y PCD \
-Y LIBRARY \
-Y DEPEX \
-Y HASH \
-Y BUILD_FLAGS \
-Y FLASH \
-Y FIXED_ADDRESS \
-Y EXECUTION_ORDER \
all
How to use the build result RPI_EFI.fd on RPi4B, consult the following:
edk2-platforms/Platform/RaspberryPi/RPi4/Readme.md
readme.md inside https://github.com/pftf/RPi4/releases/download/v1.17/RPi4_UEFI_Firmware_v1.32.zip. btw, I need to replace the original start4.elf and fixup4.dat with the ones in the zip file, otherwise, the boot of RPi4 will fail, complaining something like below:
RpiFirmwareGetClockRate: Get Clock Rate return: ClockRate=0 ClockId=C
ASSERT [ArasanMMCHost] /home/bruin/work/tianocore/edk2-platforms/Platform/RaspberryPi/
Drivers/ArasanMmcHostDxe/ArasanMmcHostDxe.c(263): BaseFrequency != 0
It's worth to analysis the RPI_EFI.fd content to some extend, by using some UEFI utilities. I mainly use the GUI version UEFITool of sudo apt install uefitool uefitool-cli. Other tools are also available. The anotomy of RPI_EFI.fd is of help when reading EDK2 build specs for checking understanding of the concepts.
One special aspect of RPI_EFI.fd is that the 1st 128K is bl31.bin binary from ATF. I guess this is due to the special booting connfiguration methods for RPi. For my platform, I don't need such kind of packaging, I only need to build the UEFI image MY.fd, which is treated as BL33 image and packaged into fip.bin togehter with BL2 and BL31 images by ATF build script.
Another aspect to notice is the "reset vector" in the begining of the .fd file. This related to the entry point of UEFI image (and entry point of each EDK2 modules), as well as interpreting the BL instruction for AArch64. Basically, it can be summarized as below:
The first [Components] in RPI_EFI.fd is ArmPlatformPkg/PrePi/PeiUniCore.inf, which is of MODULE_TYPE = SEC.
What's this component: this is the first (and only) SEC (Security) module in RPi4. What the name PrePi and Pei implies?
... the PI spec is not tied to edk2 PEIMs, and I don't see where EDKII PEI modules are currently the only "acknowledged" silicon init environment. The edk2 tree itself seems to contain platforms that don't use the edk2 PEI module set at all, but (IIRC) jump from SEC to DXE. I believe "ArmPlatformPkg/PrePi" and "ArmVirtPkg/PrePi" are related to this.
--- https://listman.redhat.com/archives/edk2-devel-archive/2020-November/msg00021.html
Its entry point: all UEFI components have the same entry point (_ModuleEntryPoint).
By "component", it means either a UEFI driver and UEFI app, both are PE32 executables, usually with suffix .efi.
The .efis are converted from ELF executables (.dll) by GenFw tool: modifying the file headers.
To verify that "all components' entry point is _ModuleEntryPoint":
Check the .dll generating command line in build report (build -y <BUILD_REPORT_FILE>), we have two flags "aarch64-none-linux-gnu-gcc" -o xxx.dll -u _ModuleEntryPoint -Wl,-e,_ModuleEntryPoint ...:
-u: gcc --help -v|grep "undefined SYMBOL" gives -u SYMBOL --undefined SYMBOL: star with undefined reference to SYMBOL.
Wl,-e: ld --help|grep "entry" gives -e ADDRESS, --entry ADDRESS Set start address.
Check all .dll files that Entry point address == _ModuleEntryPoint: find . -type f -name "*.dll" -exec sh -c "readelf -a {} |grep -E 'Entry point address|_ModuleEntryPoint'" \;
Its entry point is the entry point of whole UEFI FD image (i.e., from bl33_base_addr jump to this _ModuleEntryPoint):
Topology of the UEFI Firmware File
A UEFI Firmware File (actually a UEFI Firmware Device - FD file) is a collection of UEFI binaries encapsulated into a single image. The format of this image is defined by the Platform Initialization Specification Volume 3. A Vector Table is located at the base of this file. A 'BL' branch instruction at the base of the firwmare (location of the Reset Entry into the Vector Table) will jump to the first 'SEC' module of the UEFI Firmware Image.
--- https://github.com/lzeng14/tianocore/wiki/ArmPkg-Debugging
To verify the statements above:
Disassember the reset vector (i.e., the 1st word) of generated .FD (we got offset=0x360):
$ xxd -l 4 -e TEST.fd <== dump 4 bytes in little endian
00000000: 140000d8 <== BL {PC}+(0xd8<<2); offset=0x360
Check the Entry point in .dll (we got offset=0x240):
$ aarch64-none-elf-objdump -t ArmPlatformPrePiUniCore.dll|grep _ModuleEntryPoint
0000000000000240 g F .text 0000000000000000 _ModuleEntryPoint
$ readelf -h ArmPlatformPrePiUniCore.dll|grep Entry
Entry point address: 0x240
Compare contents of two files at different offset (we got identicial content):
$ xxd -s 0x360 -l 64 TEST.fd <== skip 0x360 bytes, dump 64 bytes
00000360: 901e 0094 050a 0094 ea03 00aa a1cd 0a58 ...............X
00000370: 0200 e0d2 2200 c0f2 0240 a0f2 0200 80f2 ...."....#......
00000380: c303 a0d2 e3ff 9ff2 6304 00d1 6300 028b ........c...c...
00000390: 0400 a1d2 0400 80f2 2000 03eb 8400 0054 ........ ......T
$ xxd -s 0x240 -l 64 ArmPlatformPrePiUniCore.dll <== skip 0x240 bytes
00000240: 901e 0094 050a 0094 ea03 00aa a1cd 0a58 ...............X
00000250: 0200 e0d2 2200 c0f2 0240 a0f2 0200 80f2 ...."....#......
00000260: c303 a0d2 e3ff 9ff2 6304 00d1 6300 028b ........c...c...
00000270: 0400 a1d2 0400 80f2 2000 03eb 8400 0054 ........ ......T
Prepare an empty pkg, and make it build ok. The main purpuse is to do some exercise with EDK2 build system, and use the empty pkg as the start point for the new platform.
Make a copy of RaspberryPi.dec, change all gRaspberry to gMyPlatform.
Make a copy of RPi4.dsc and RPi4.fdf, and comment out all stuff in DSC and FDF file.
Replace all GUIDs in DSC/FDF/DEC files, generating new ones using online guid generator.
Note that PCD are declared in DEC files, and DEC files are refered by modules (INF files). As the empty package contains no module, no PCD definition will be available in FDF. So for a success build of the empty package, we need to comment out all PCD reference in FDF.
The NOOPT build command for MyPlatform is as below:
#!/bin/bash
export WORKSPACE=/home/bruin/work/tianocore
export PACKAGES_PATH=$WORKSPACE/edk2:$WORKSPACE/edk2-platforms:$WORKSPACE/edk2-non-osi
pushd $WORKSPACE
source edk2/edksetup.sh
echo "Building BaseTools..."
make -C edk2/BaseTools all
echo "Building UEFI firmware for MyPlatform..."
GCC5_AARCH64_PREFIX=aarch64-none-linux-gnu- build \
-n 4 \
-a AARCH64 \
-p Platform/MyCorp/MyPlatform/MyPlatform.dsc \
-t GCC5 \
-b NOOPT \
-v -d 9 -j MyPlatform-build.log \
-y MyPlatform-build-report.txt \
-Y EXECUTION_ORDER \
-Y PCD \
-Y LIBRARY \
-Y DEPEX \
-Y HASH \
-Y BUILD_FLAGS \
-Y FLASH \
-Y FIXED_ADDRESS \
all
popd
Add the 1st component ArmPlatformPrePiUniCore. This component is to prepare the HOBs for DXE phae. The main purpose is to get serial port working and memory config correct. Another purpose of this step is to familiar with steps for adding a component/module/lib. Below is a brief summary of the steps:
Uncomment the module's INF into both DSC ([Components] section), and FDF ([FV.FVMAIN_COMPACT]).
Rebuild the pkg, and resolve all Instance of library class [xxxLib] is not found errors reported, by updating [LibraryClasses] sections of DSC.
This step is a repeating process for dozens of times.
Some lib-class has multiple lib-instances, making sure choose the appropriate lib-instance (ref the build-report of RPi4).
if encounter ModuleEntryPoint.iiii:31: Error: immediate out of range: enable gArmTokenSpaceGuid.PcdFdBaseAddress and gArmTokenSpaceGuid.PcdFdSize in FDF.
if encounter undefined reference to _gPcd_BinaryPatch_PcdSerialClockRate: set PcdSerialClockRate in [PcdsPatchableInModule] section in DSC. FIXME: why? ref.
Check the PCDs listed in build log: inspect any abnormal PCD values, and supply correct values.
Customize platform-specific drivers or libraries.
SerialPortLib: locate the lib-class header file (MdePkg/Include/Library/SerialPortLib.h) by find edk2 -type f -name "*.dec" -exec grep -Hn SerialPortLib. The following functions are required:
SerialPortInitialize()
SerialPortWrite()
SerialPortRead()
SerialPortPoll()
SerialPortSetControl(): RETURN_UNSUPPORTED
SerialPortGetControl(): RETURN_UNSUPPORTED
SerialPortSetAttributes(): RETURN_UNSUPPORTED
ArmPlatformLib: interface header at Include/Library/ArmPlatformLib.h. The following functions are required:
ArmPlatformGetCorePosition(): return cpu idx in the cluster given the MPIDR value. this function is used in _ModuleEntryPoint for setting stack for secondary cores. Assuming one cluster for now.
ArmPlatformIsPrimaryCore()
ArmPlatformGetPrimaryCoreMpId()
ArmPlatformGetBootMode()
ArmPlatformPeiBootAction()
ArmPlatformInitialize()
ArmPlatformGetVirtualMemoryMap()
ArmPlatformGetPlatformPpiList()
etc...
Uncomment more modules in DSC/FDF, module by module...For driver/libs which are RPi platform specific, we can:
either search the edk2/edk2-platform for similiar driver or lib instances, or
copy the RPi4 implementation and comment out most of the content, make the pkg build success first, and then bug fixing.
Debugging: my current main debugging method is through adding "printf()", i.e., the edk2 macro DEBUG((DEBUG_INFO,)). One needs to set gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel to an appropriate value to see more debug info.
{ Error: Command failed: identify: delegate failed `'ufraw-batch' --silent --create-id=also --out-type=png --out-depth=16 '--output=%u.png' '%i'' # error/delegate.c/InvokeDelegate/1919.
identify: unable to open image `/tmp/magick-53080Z_JGYvtuPOl8.ppm': No such file or directory # error/blob.c/OpenBlob/2841.
I got an issue while working with ImageMagick in my node application I did the search but can't get the proper solution for my problem.
Edit: ufraw is no longer maintained, so use at your own risk and see dcraw alternative lower down.
You need to install 'ufraw-batch'
sudo apt-get install ufraw-batch
If this does not work, as it did for me. I got a "Segmentation fault (core dumped)"
This seems to be common problem with ufraw: https://sourceforge.net/p/ufraw/bugs/
It does not seem to get update much anymore, at least not in the main repos.
Converting RAW image into other types of image you can use DCRaw (it is also used by UFRaw).
Here is an example to convert a .cr2 image to a .png:
dcraw -c -w "./IMG_1.CR2" | pnmtopng > "./IMG_1.png";
Macos:
brew install ufraw
Solved it for me
I'm running into a problem attempting to upload the "blink" app onto the motes. I can't seem to run the command make telosb reinstall bsl,/dev/ttUSB0 or make telosb reinstall while in the apps/Blink directory, which is preventing me from moving on with my project. I've tried as a user, superuser, and as root. I've outlined below the responses from variable commands. [Running Ubuntu 16.04, TinyOS 2.1.2, ncc version 1.4.2, nescc version 1.3.6]
(A)With root and the make telosb command I get back:
mkdir -p build/telosb
compiling BlinkAppC to a telosb binary
ncc -o build/telosb/main.exe -Os -fnesc-separator=__ -Wall -Wshadow -Wnesc-all -target=telosb -fnesc-cfile=build/telosb/app.c -board= -DDEFINED_TOS_AM_GROUP=0x22 -DIDENT_APPNAME=\"BlinkAppC\" -DIDENT_USERNAME=\"root\" -DIDENT_HOSTNAME=\"liam-Latitude-E\" -DIDENT_USERHASH=0x9236fe46L -DIDENT_TIMESTAMP=0x59384a62L -DIDENT_UIDHASH=0xdc08609fL BlinkAppC.nc -lm
compiled BlinkAppC to build/telosb/main.exe
2538 bytes in ROM
56 bytes in RAM
msp430-objcopy --output-target=ihex build/telosb/main.exe build/telosb/main.ihex
writing TOS image
(B)With regular user and the make telosb command I get back:
mkdir -p build/telosb
/bin/sh: 1: cannot create build/telosb/ident_flags.txt: Permission denied
/home/liam/tinyos-main/support/make/ident_flags.extra:13: recipe for target 'ident_cache' failed
make: *** [ident_cache] Error 2
(C)With a super user and the sudo make telosb command I get back:
make: *** No rule to make target 'telosb'. Stop.
(D)With root and the make telosb reinstall command I get back:
cp build/telosb/main.ihex build/telosb/main.ihex.out
found mote on /dev/ttyUSB0 (using bsl,auto)
installing telosb binary using bsl
tos-bsl --telosb -c /dev/ttyUSB0 -r -e -I -p build/telosb/main.ihex.out
MSP430 Bootstrap Loader Version: 1.39-goodfet-8
Mass Erase...
Transmit default password ...
Invoking BSL...
Transmit default password ...
Current bootstrap loader version: 1.61 (Device ID: f16c)
Changing baudrate to 38400 ...
MSP430 Bootstrap Loader Version: 1.39-goodfet-8
Mass Erase...
Transmit default password ...
Invoking BSL...
Transmit default password ...
Current bootstrap loader version: 1.61 (Device ID: f16c)
Changing baudrate to 38400 ...
Traceback (most recent call last):
File "/usr/bin/tos-bsl", line 1918, in <module>
main(0);
File "/usr/bin/tos-bsl", line 1843, in main
speed=speed,
File "/usr/bin/tos-bsl", line 1218, in actionStartBSL
self.actionChangeBaudrate(speed) #change baudrate
File "/usr/bin/tos-bsl", line 1345, in actionChangeBaudrate
self.serialport.setBaudrate(baudrate)
AttributeError: 'Serial' object has no attribute 'setBaudrate'
/home/liam/tinyos-main/support/make/msp/bsl.extra:45: recipe for target 'program' failed
make: *** [program] Error 1
(E)Whereas with a regular user and make telosb reinstall I get back:
cp build/telosb/main.ihex build/telosb/main.ihex.out
cp: cannot create regular file 'build/telosb/main.ihex.out': Permission denied
/home/liam/tinyos-main/support/make/msp/msp.rules:92: recipe for target 'setid' failed
make: *** [setid] Error 1
I have been all over the internet and online forums and haven't found a fix yet. I researched around (D) and found that perhaps python 2 might have changed the name of 'setBaudRate' variable. I'm not sure how to change that either.
Thank you for your time and help!
edit: added ncc and nescc versions.
This stack overflow answer is applicable to TinyOS.
So it turns out that pyserial 3.0.1 is not compatible with the TinyOS app, Blink. I'm fairly certain that pyserial 3.0.1 came with the package for TinyOS. To fix this, use the command sudo pip install "pySerial>=2.0,<=2.99999". For me, it kicked back:
The directory '/home/liam/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/liam/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pySerial<=2.99999,>=2.0
Installing collected packages: pySerial
Found existing installation: pyserial 3.0.1
DEPRECATION: Uninstalling a distutils installed project (pySerial) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling pyserial-3.0.1:
Successfully uninstalled pyserial-3.0.1
Successfully installed pySerial-2.7
But the blink app still uploaded and worked fine.
I am attempting to rebuild my development/test environment on a new laptop running windows7 32 bit. imageMagick is one of the modules I had installed in an (ancient) version on my old laptop. I have downloaded and installed perl from dwimperl, which is v5.14.2 and had a couple modules install, and several did not.
cpanm Image::Magick - failed
cpanm DB_File - failed
cpanm Time::HiRes - failed
I searched around and found a discussion on magick failing to install on v5.12, but couldn't tell if that was supposed to have been fixed, or if I need to attempt to recreate the fix discussed for 5.12
should I try installing perl from Strawberry? looks like it is a little newer v5.18 ?
is there something I can tweak and re-run cpan installs?
I installed the binaries from imagemagick, from Link first, then cpan again, and cpan still fails....
This is the top part of the build log down to where it really goes south and starts kicking out errors:
cpanm (App::cpanminus) 1.6941 on perl 5.014002 built for MSWin32-x86-multi-thread
Work directory is C:\Users\dtbaker/.cpanm/work/1377281741.8420
You have make C:\Dwimperl\c\bin\dmake.exe
You have LWP 6.03
Falling back to Archive::Tar 1.80
Searching Image::Magick on cpanmetadb ...
--> Working on Image::Magick
Fetching http://www.cpan.org/authors/id/J/JC/JCRISTY/PerlMagick-6.86.tar.gz
-> OK
Unpacking PerlMagick-6.86.tar.gz
Entering PerlMagick-6.86
META.yml/json not found. Creating skelton for it.
Configuring PerlMagick-6.86
Running Makefile.PL
################################### WARNING! ###################################
# It seems that you are trying to install Perl::Magick on a MS Windows box with
# perl + gcc compiler (e.g. strawberry perl), however we cannot find ImageMagick
# binaries installed on your system.
#
# Please check the following prerequisites:
#
# 1) You need to have installed ImageMagick Windows binaries from
# http://www.imagemagick.org/script/binary-releases.php#windows
#
# 2) We only support dynamic (DLL) ImageMagick binaries
# note: it is not possible to mix 32/64-bit binaries of perl and ImageMagick
#
# 3) During installation select that you want to install ImageMagick's
# development files (libraries+headers)
#
# 4) You also need to have ImageMagick's directory in your PATH
# note: we are checking the presence of convert.exe and/or identify.exe tools
#
# 5) You might need Visual C++ Redistributable Package installed on your system
# see instructions on ImageMagick's Binary Release webpage
#
# We are gonna continue, but chances for successful build are very low!
################################################################################
Checking if your kit is complete...
Looks good
Note (probably harmless): No library found for -lMagickCore-6.Q16
Writing Makefile for Image::Magick
Writing MYMETA.yml and MYMETA.json
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have ExtUtils::MakeMaker 0 ... Yes (6.62)
Building and testing Image-Magick-6.86
cp Magick.pm blib\lib\Image\Magick.pm
AutoSplitting blib\lib\Image\Magick.pm (blib\lib\auto\Image\Magick)
C:\Dwimperl\perl\bin\perl.exe C:\Dwimperl\perl\lib\ExtUtils\xsubpp -typemap C:\Dwimperl\perl\lib\ExtUtils\typemap -typemap typemap Magick.xs > Magick.xsc && C:\Dwimperl\perl\bin\perl.exe -MExtUtils::Command -e mv -- Magick.xsc Magick.c
gcc -c -s -O2 -DWIN32 -DPERL_TEXTMODE_SCRIPTS -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fno-strict-aliasing -mms-bitfields -s -O2 -DVERSION=\"6.86\" -DXS_VERSION=\"6.86\" "-IC:\Dwimperl\perl\lib\CORE" -D_LARGE_FILES=1 -DHAVE_CONFIG_H Magick.c
Magick.xs:60:31: error: magick/MagickCore.h: No such file or directory
Magick.xs:167: error: expected specifier-qualifier-list before 'MagickRealType'
Magick.xs:188: error: expected specifier-qualifier-list before 'ImageInfo'
Magick.xs:210: error: 'MagickNoiseOptions' undeclared here (not in a function)
however we cannot find ImageMagick binaries installed on your system.
Is ImageMagick is in your path, as recommanded in the 4) point? Open cmd.exe and type convert -v or convert.exe -v. If you don't see informations about Image Magick (Windows have a built-in convert command), it is very likely you have to add it to your path.
You also have to check points 1), 3), and 5). After that try again to run installation process through cpanm.
Trying to install for Citrus Perl, I discovered on debugging through the Perl part of the install that the mingw64 install had not included 'pexports.exe'. Downloading that from https://sourceforge.net/projects/mingw/files/MinGW/Extension/pexports/
and placing it in the mingw64 directory was necessary to solve the problem of a long list of library export symbols not found.
Prior to that I had also set the environment variables CPATH, C_INCLUDE_PATH, and CPLUS_INCLUDE_PATH to point to the "include" directory of the ImageMagick install include directory in C:\Program Files (x86). (When you install ImageMagick you should check the box to install also for Strawberry Perl.)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed last year.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
Improve this question
This is the PATH variable without sudo:
$ echo 'echo $PATH' | sh
/opt/local/ruby/bin:/usr/bin:/bin
This is the PATH variable with sudo:
$ echo 'echo $PATH' | sudo sh
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin
As far as I can tell, sudo is supposed to leave PATH untouched. What's going on? How do I change this? (This is on Ubuntu 8.04).
UPDATE: as far as I can see, none of the scripts started as root change PATH in any way.
From man sudo:
To prevent command spoofing, sudo
checks ``.'' and ``'' (both denoting
current directory) last when searching
for a command in the user's PATH (if
one or both are in the PATH). Note,
however, that the actual PATH
environment variable is not modified
and is passed unchanged to the program
that sudo executes.
This is an annoying function a feature of sudo on many distributions.
To work around this "problem" on ubuntu I do
the following in my ~/.bashrc
alias sudo='sudo env PATH=$PATH'
Note the above will work for commands that don't reset the $PATH themselves.
However `su' resets it's $PATH so you must use -p to tell it not to. I.E.:
sudo su -p
In case someone else runs accross this and wants to just disable all path variable changing for all users.
Access your sudoers file by using the command:visudo. You should see the following line somewhere:
Defaults env_reset
which you should add the following on the next line
Defaults !secure_path
secure_path is enabled by default. This option specifies what to make $PATH when sudoing. The exclamation mark disables the feature.
PATH is an environment variable, and as such is by default reset by sudo.
You need special permissions to be permitted to do this.
From man sudo
-E The -E (preserve environment) option will override the env_reset
option in sudoers(5)). It is only available when either the match-
ing command has the SETENV tag or the setenv option is set in sudo-
ers(5).
Environment variables to be set for the command may also be passed on
the command line in the form of VAR=value, e.g.
LD_LIBRARY_PATH=/usr/local/pkg/lib. Variables passed on the command
line are subject to the same restrictions as normal environment vari-
ables with one important exception. If the setenv option is set in
sudoers, the command to be run has the SETENV tag set or the command
matched is ALL, the user may set variables that would overwise be for-
bidden. See sudoers(5) for more information.
An Example of usage:
cat >> test.sh
env | grep "MYEXAMPLE" ;
^D
sh test.sh
MYEXAMPLE=1 sh test.sh
# MYEXAMPLE=1
MYEXAMPLE=1 sudo sh test.sh
MYEXAMPLE=1 sudo MYEXAMPLE=2 sh test.sh
# MYEXAMPLE=2
update
man 5 sudoers :
env_reset If set, sudo will reset the environment to only contain
the LOGNAME, SHELL, USER, USERNAME and the SUDO_* vari-
ables. Any variables in the caller's environment that
match the env_keep and env_check lists are then added.
The default contents of the env_keep and env_check
lists are displayed when sudo is run by root with the
-V option. If sudo was compiled with the SECURE_PATH
option, its value will be used for the PATH environment
variable. This flag is on by default.
So may need to check that this is/is not compiled in.
It is by default in Gentoo
# ( From the build Script )
....
ROOTPATH=$(cleanpath /bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/opt/bin${ROOTPATH:+:${ROOTPATH}})
....
econf --with-secure-path="${ROOTPATH}"
Looks like this bug has been around for quite a while! Here are some bug references you may find helpful (and may want to subscribe to / vote up, hint, hint...):
Debian bug #85123 ("sudo: SECURE_PATH still can't be overridden") (from 2001!)
It seems that Bug#20996 is still present in this version of sudo. The
changelog says that it can be overridden at runtime but I haven't yet
discovered how.
They mention putting something like this in your sudoers file:
Defaults secure_path="/bin:/usr/bin:/usr/local/bin"
but when I do that in Ubuntu 8.10 at least, it gives me this error:
visudo: unknown defaults entry `secure_path' referenced near line 10
Ubuntu bug #50797 ("sudo built with --with-secure-path is problematic")
Worse still, as far as I can tell, it
is impossible to respecify secure_path
in the sudoers file. So if, for
example, you want to offer your users
easy access to something under /opt,
you must recompile sudo.
Yes. There needs to be a way to
override this "feature" without having
to recompile. Nothing worse then
security bigots telling you what's
best for your environment and then not
giving you a way to turn it off.
This is really annoying. It might be
wise to keep current behavior by
default for security reasons, but
there should be a way of overriding it
other than recompiling from source
code! Many people ARE in need of PATH
inheritance. I wonder why no
maintainers look into it, which seems
easy to come up with an acceptable
solution.
I worked around it like this:
mv /usr/bin/sudo /usr/bin/sudo.orig
then create a file /usr/bin/sudo containing the following:
#!/bin/bash
/usr/bin/sudo.orig env PATH=$PATH "$#"
then your regular sudo works just like the non secure-path sudo
Ubuntu bug #192651 ("sudo path is always reset")
Given that a duplicate of this bug was
originally filed in July 2006, I'm not
clear how long an ineffectual env_keep
has been in operation. Whatever the
merits of forcing users to employ
tricks such as that listed above,
surely the man pages for sudo and
sudoers should reflect the fact that
options to modify the PATH are
effectively redundant.
Modifying documentation to reflect
actual execution is non destabilising
and very helpful.
Ubuntu bug #226595 ("impossible to retain/specify PATH")
I need to be able to run sudo with
additional non-std binary folders in
the PATH. Having already added my
requirements to /etc/environment I was
surprised when I got errors about
missing commands when running them
under sudo.....
I tried the following to fix this
without sucess:
Using the "sudo -E" option - did not work. My existing PATH was still reset by sudo
Changing "Defaults env_reset" to "Defaults !env_reset" in /etc/sudoers -- also did not work (even when combined with sudo -E)
Uncommenting env_reset (e.g. "#Defaults env_reset") in /etc/sudoers -- also did not work.
Adding 'Defaults env_keep += "PATH"' to /etc/sudoers -- also did not work.
Clearly - despite the man
documentation - sudo is completely
hardcoded regarding PATH and does not
allow any flexibility regarding
retaining the users PATH. Very
annoying as I can't run non-default
software under root permissions using
sudo.
This seemed to work for me
sudo -i
which takes on the non-sudo PATH
I think it is in fact desirable to have sudo reset the PATH: otherwise an attacker having compromised your user account could put backdoored versions of all kinds of tools on your users' PATH, and they would be executed when using sudo.
(of course having sudo reset the PATH is not a complete solution to these kinds of problems, but it helps)
This is indeed what happens when you use
Defaults env_reset
in /etc/sudoers without using exempt_group or env_keep.
This is also convenient because you can add directories that are only useful for root (such as /sbin and /usr/sbin) to the sudo path without adding them to your users' paths. To specify the path to be used by sudo:
Defaults secure_path="/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin"
Works now using sudo from the karmic repositories. Details from my configuration:
root#sphinx:~# cat /etc/sudoers | grep -v -e '^$' -e '^#'
Defaults env_reset
Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/grub-1.96/sbin:/opt/grub-1.96/bin"
root ALL=(ALL) ALL
%admin ALL=(ALL) ALL
root#sphinx:~# cat /etc/apt/sources.list
deb http://au.archive.ubuntu.com/ubuntu/ jaunty main restricted universe
deb-src http://au.archive.ubuntu.com/ubuntu/ jaunty main restricted universe
deb http://au.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted universe
deb-src http://au.archive.ubuntu.com/ubuntu/ jaunty-updates main restricted universe
deb http://security.ubuntu.com/ubuntu jaunty-security main restricted universe
deb-src http://security.ubuntu.com/ubuntu jaunty-security main restricted universe
deb http://au.archive.ubuntu.com/ubuntu/ karmic main restricted universe
deb-src http://au.archive.ubuntu.com/ubuntu/ karmic main restricted universe
deb http://au.archive.ubuntu.com/ubuntu/ karmic-updates main restricted universe
deb-src http://au.archive.ubuntu.com/ubuntu/ karmic-updates main restricted universe
deb http://security.ubuntu.com/ubuntu karmic-security main restricted universe
deb-src http://security.ubuntu.com/ubuntu karmic-security main restricted universe
root#sphinx:~#
root#sphinx:~# cat /etc/apt/preferences
Package: sudo
Pin: release a=karmic-security
Pin-Priority: 990
Package: sudo
Pin: release a=karmic-updates
Pin-Priority: 960
Package: sudo
Pin: release a=karmic
Pin-Priority: 930
Package: *
Pin: release a=jaunty-security
Pin-Priority: 900
Package: *
Pin: release a=jaunty-updates
Pin-Priority: 700
Package: *
Pin: release a=jaunty
Pin-Priority: 500
Package: *
Pin: release a=karmic-security
Pin-Priority: 450
Package: *
Pin: release a=karmic-updates
Pin-Priority: 250
Package: *
Pin: release a=karmic
Pin-Priority: 50
root#sphinx:~# apt-cache policy sudo
sudo:
Installed: 1.7.0-1ubuntu2
Candidate: 1.7.0-1ubuntu2
Package pin: 1.7.0-1ubuntu2
Version table:
*** 1.7.0-1ubuntu2 930
50 http://au.archive.ubuntu.com karmic/main Packages
100 /var/lib/dpkg/status
1.6.9p17-1ubuntu3 930
500 http://au.archive.ubuntu.com jaunty/main Packages
root#sphinx:~# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/grub-1.96/sbin:/opt/grub-1.96/bin
root#sphinx:~# exit
exit
abolte#sphinx:~$ echo $PATH
/home/abolte/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/grub-1.96/sbin:/opt/grub-1.96/bin:/opt/chromium-17593:/opt/grub-1.96/sbin:/opt/grub-1.96/bin:/opt/xpra-0.0.6/bin
abolte#sphinx:~$
It's wonderful to finally have this solved without using a hack.
# cat .bash_profile | grep PATH
PATH=$HOME/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
export PATH
# cat /etc/sudoers | grep Defaults
Defaults requiretty
Defaults env_reset
Defaults env_keep = "SOME_PARAM1 SOME_PARAM2 ... PATH"
Just comment out "Defaults env_reset" in /etc/sudoers
Just edit env_keep in /etc/sudoers
It looks something like this:
Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASURE MENT LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME LC_ALL L ANGUAGE LINGUAS XDG_SESSION_COOKIE"
Just append PATH at the end, so after the change it would look like this:
Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASURE MENT LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME LC_ALL L ANGUAGE LINGUAS XDG_SESSION_COOKIE PATH"
Close the terminal and then open again.
Secure_path is your friend, but if you want to exempt yourself from secure_path just do
sudo visudo
And append
Defaults exempt_group=your_goup
If you want to exempt a bunch of users create a group, add all the users to it, and use that as your exempt_group. man 5 sudoers for more.
the recommended solution in the comments on the OpenSUSE distro suggests to change:
Defaults env_reset
to:
Defaults !env_reset
and then presumably to comment out the following line which isn't needed:
Defaults env_keep = "LANG LC_ADDRESS LC_CTYPE LC_COLLATE LC_IDENTIFICATION LC_MEASURE MENT LC_MESSAGES LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME LC_ALL L ANGUAGE LINGUAS XDG_SESSION_COOKIE"
comment out both "Default env_reset" and "Default secure_path ..." in /etc/sudores file works for me
You can also move your file in a sudoers used directory :
sudo mv $HOME/bash/script.sh /usr/sbin/
Er, it's not really a test if you don't add something to your path:
bill#bill-desktop:~$ ls -l /opt/pkg/bin
total 12
-rwxr-xr-x 1 root root 28 2009-01-22 18:58 foo
bill#bill-desktop:~$ which foo
/opt/pkg/bin/foo
bill#bill-desktop:~$ sudo su
root#bill-desktop:/home/bill# which foo
root#bill-desktop:/home/bill#
The PATH will be reset when using su or sudo by the definition of ENV_SUPATH, and ENV_PATH defined in /etc/login.defs
$PATH is an environment variable and it means that value of $PATH can differ for another users.
When you are doing login into your system then your profile setting decide the value of the $PATH.
Now, lets take a look:-
User | Value of $PATH
--------------------------
root /var/www
user1 /var/www/user1
user2 /var/www/html/private
Suppose that these are the values of $PATH for different user. Now when you are executing any command with sudo then in actual meaning root user executes that command .
You can confirm by executing these commands on terminal :-
user#localhost$ whoami
username
user#localhost$ sudo whoami
root
user#localhost$
This is the reason. I think its clear to you.
It may be counter-intuitive but the first time it happened to me, I knew what was going on. Believe me, you don't want root running someone else's PATH
"Hey root? Can you help me, something is wrong" and he comes over and sudo's from my shell and I wrote a "${HOME}/bin/ls" shell script that first gives me superuser privileges, and then calls the real /bin/ls.
# personal ls
usermod -a -G sudo ${USER}
/bin/ls
The minute root user does "sudo ls" from my shell, he's done and the box is wide open to me.