I cloned a large tar file from the address: https://github.com/uni-due-syssec/evmpatch-eval-data/tree/master/large-scale;
but when I untar it in the Linux system with the command tar -I zstd -xvf osiris_dataset_14k.tar.zst, I encounter the following errors:
zstd: /*stdin*\: unsupported format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
I surf on the internet but don't find the solution
The file in question is stored in that git repo as a Git LFS Record. I suspect what you are trying to decompress is just the LFS pointer, which looks like this:
version https://git-lfs.github.com/spec/v1
oid sha256:1b6282bef881e59ad83ac4aecc27fc41025bec88ed8b44983e904372fdd22fd4
size 753961811
So Zstd is accurately reporting that it is being given data that is not zstd-compressed, and it can't decompress it.
The file you have pasted is a Git LFS record (reference to, not the actual file).
The first step should be to retrieve the file, therefore, install Git LFS - see more details on installing LFS here: https://packagecloud.io/github/git-lfs/install
After its installed run:
git lfs fetch
That will initiate the downloading process, and then you can run the tar command.
If you already have Git LFS installed you can just use:
git clone ...
And you don't need the two steps mentioned above - you should see if you have LFS installed during the cloning:
Downloading LFS objects: 0% (0/1), 285 MB | 5.3 MB/s
Related
I'm trying to get started with a simple Hello World derivation using the Nix manual.
But it's not clear to me how to go about building it.
Is there somewhere I can download the source files from so I don't
have to copy them line-by-line?
Is there a way I can nix-build it
without having to modify anything global (eg pkgs/top-level/all-packages.nix)?
Where is pkgs/top-level/all-packages.nix?
One option is to clone the nixpkgs repository and then build the hello package recipe provided in that repository:
git clone https://github.com/NixOS/nixpkgs
cd nixpkgs
nix-build -A hello
Doing it this way, you don't have to modify all-packages.nix, because it already has an entry for hello. If you do want to modify all-packages.nix, you can find it in the nixpkgs repository that you cloned. Just take the path of the repository you cloned, (e.g. ~/nixpkgs) and add pkgs/top-level/all-packages.nix to get the path to all-packages.nix. You can see a copy of that file here:
https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/all-packages.nix
When you start building your own software that is not part of nixpkgs, you might chose to write your own default.nix file in your own repository and put a line like this in there to import nixpkgs, using the NIX_PATH environment variable:
let
nixpkgs = import <nixpkgs> { };
...
Note that I am running NixOs, not sure if my answer will valid for other (non-Linux) system.
Is there somewhere I can download the source files from so I don't have to copy them line-by-line?
You could browse nixpkgs, All source files are there.
Is there a way I can nix-build it without having to modify anything global (eg pkgs/top-level/all-packages.nix)?
David Grayson give excellent answer already.
I would love to add some information.
nix-build will looking for default.nix on the current directory and the build result will symlink named result on current working directory.
Another way to test if nix expression could build is nix-shell which also looking for default.nix or shell.nix on current directory. If the build success you will get shell prompt with your packages avaliable.
Both nix-build and nix-shell have -I argument that you can point to any nix repository including remote one.
For example, if I use nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/master.tar.gz -p hello, nix will download binary cache if exists or build hello using current master branch expression and give me a shell which hello is avaliable.
$ nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/master.tar.gz -p hello
downloading ‘https://github.com/NixOS/nixpkgs/archive/master.tar.gz’... [12850/0 KiB, 1404.5 KiB/s]
unpacking ‘https://github.com/NixOS/nixpkgs/archive/master.tar.gz’...
these paths will be fetched (0.04 MiB download, 0.19 MiB unpacked):
/nix/store/s3vlmp0k8b07h0r81bn7lh24q2mqcai8-hello-2.10
fetching path ‘/nix/store/s3vlmp0k8b07h0r81bn7lh24q2mqcai8-hello-2.10’...
*** Downloading ‘https://cache.nixos.org/nar/1ax9cr6qqqqrb4pdm1mpqn7whm6alwp56dvsh5hpgs5g8rrpnjxd.nar.xz’ (signed by ‘cache.nixos.org-1’) to ‘/nix/store/s3vlmp0k8b07h0r81bn7lh24q2mqcai8-hello-2.10’...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 40364 100 40364 0 0 40364 0 0:00:01 0:00:01 --:--:-- 37099
[nix-shell:~]$ which hello
/nix/store/s3vlmp0k8b07h0r81bn7lh24q2mqcai8-hello-2.10/bin/hello
Where is pkgs/top-level/all-packages.nix?
There is NIX_PATH enviroment variable. The nixpkgs portion will point you to your current repository
$ echo $NIX_PATH
nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels
My all-packages.nix is located at /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/top-level/all-packages.nix
I have an app that i am working on that i did not set up and i don't have communication with the dev's that did. I have pushed up to it in the past but today for some reason i was not able to. Rails 4 with capistrano. When i looked at the app directory on the server i could find the .gitignore but no .git folder. I did a "git init && git remote add git_ssh_url" but it didn't work. I was thinking about a git clone but i have no clue what that will do to the app. The app is used daily at a company and i'm sure they wouldn't like me screwing it up :) I'm at a loss, Any help is appreciated.
**
Edit
**
So i realized that there is a staging server set up on the same droplet as well and the code to deploy to each is nearly identical. I had no problems updating the staging server
Here is the error code:
refreshing local cache to revision 4ea064e4826a9880c6bac3e0a2689b688e223911 at /var/folders/qb/6fdgzljs1r9ddgkvqk1ky0y40000gn/T/mpm
executing locally: cd /var/folders/qb/6fdgzljs1r9ddgkvqk1ky0y40000gn/T/mpm && git fetch -q origin && git fetch --tags -q origin && git reset -q --hard 4ea064e4826a9880c6bac3e0a2689b688e223911 && git clean -q -d -x -f
fatal: Not a git repository (or any of the parent directories): .git
command finished in 9ms
shell command failed with return code pid 97172 exit 128
So the solution i came across while asking a friend was that somehow the temp cache file that capistrano creates locally here (this path was pulled form the error that i posted above) :
/var/folders/qb/6fdgzljs1r9ddgkvqk1ky0y40000gn/T/mpm
was throwing the error. At some point there must have been an interrupted deploy that messed up that file. When i ls -la the mpm file above there was a .git file but for whatever reason it wasn't recognized.
The solution was to
rm -rf /var/folders/qb/6fdgzljs1r9ddgkvqk1ky0y40000gn/T/mpm
and rerun the cap production deploy command and it rebuilds the temp cache file and deploys the code.
Hope this helps someone!
Question How do I avoid "fatal: write error: Read-only file" problem when using git clone.
Error occurs inside a docker instance
docker run -it imagename /bin/bash
RESULTS:
root#imagename:~/# git clone user#address:repo dir_name
Cloning into 'dir_name'...
remote: Counting objects: 79344, done.
remote: Compressing objects: 100% (30878/30878), done.
fatal: write error: Read-only file system76 GiB | 5.31 MiB/s
fatal: index-pack failed
What is the correct magic invocation to do this successfully ?
I saw this error when the docker storage space were full (limit in 20GB).
If you pulled too much images or run too many containers, you will see this error.
Clean some images or container will help.
The official install instructions say:
If you want to build the wx application, you will need to get
wxWidgets-3.0 (wxWidgets-3.0.0.tar.bz2 from
http://sourceforge.net/projects/wxwindows/files/3.0.0/) or get it from
github with bug fixes:
$ git clone --branch WX_3_0_branch git#github.com:wxWidgets/wxWidgets.git
Who wouldn't want bug fixes:
$ git clone --branch WX_3_0_branch git#github.com:wxWidgets/wxWidgets.git
Cloning into 'wxWidgets'...
fatal: Remote branch WX_3_0_branch not found in upstream origin
Does anyone know where the bug fix version is located?
Response to comment:
With all caps, I get:
$ git clone --branch WX_3_0_BRANCH git#github.com:wxWidgets/wxWidgets.git
Cloning into 'wxWidgets'...
ssh_exchange_identification: read: Connection reset by peer
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
For future searchers: in order to use github commands like above, you need to setup ssh keys for your computer. I had already done that, but for some reason it didn't work a few hours later. Firewall problems at my new location? I don't know. I moved to a different location/server from where I was getting the Connection reset by peer error, and then I created a new ssh key, and I was able to download and install wxWidgets. The git clone command just downloads the source, so you probably want to be at the ~/Downloads prompt when you issue that command. The Erlang install directions tell you how to install wxWidgets.
The branch in question is named WX_3_0_BRANCH — all uppercase.
After running make distcheck I get the message that I have successfully built the package and is ready for distribution. If I untar the tar.gz with tar -zxvf hello-0.2.tar.gz it successfully extracts all of its contents. However, when I try to extract them in different machines I get:
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
The weird thing is that it was working before.
On the machine I'm trying to build the package, I've updated my automake 1.10.1, autoconf 2.61, and tar 1.20 to automake 1.11.1, autoconf 2.65, and tar 1.23 and still the same issue.
Any ideas what could be the problem?
The problem is not on the build machine; the problem is on the target machines.
Not all versions of tar automatically recognize the decompression to apply to a compressed tar file. Given that gunzip followed by tar does work, then the tar on your target machine is one such. The versions of tar on the mainstream Unix systems (AIX, HP-UX, Solaris) do not recognize compressed tar files automatically. Those on Linux and MacOS X do.
Note that you can use:
gzip -dc hello-0.2.tar.gz | tar -xf -
to avoid creating the intermediate uncompressed file.
Actually this could happen when the server you download from applies another round of GZip and the client you used to download the file doesn't read/respect the HTTP Content-Encoding header and stores the HTTP payload as it was on the wire.
Although the file appears to have only the extension .tar.gz it is in fact .tar.gz.gz. after you run the gunzip once the file gets the extension .tar only but still this time running the tar command tar xf hello-0.2.tar recognizes the GZip format and implicitly runs the file through gunzip one more time before extracting.
You can check this by running head hello-02.tar.gz and head hello-02.tar. GZip is a very binary format, whereas tar is quite human readable. If the .tar file appears "too binary" you have a doubly encoded file on your hands.