I've been attempting to get cross-compilation to an iOS target working via nix. My tenative expression starts out like this:
{iphone ? false}:
let
crossSystem = if iphone then {
config = "aarch64-apple-ios";
sdkVer = "13.7";
xcodeVer = "11";
xcodePlatform = "iPhoneOS";
useiOSPrebuilt = true;
platform = {};
} else null;
pkgs_fn = (import ../repo.nix).nixos_2009;
cmake = (pkgs_fn {}).cmake;
pkgs = pkgs_fn { config.allowUnfree = true; crossSystem = crossSystem; };
in
pkgs.stdenv.mkDerivation rec {
nativeBuildInputs = [
cmake
];
buildInputs = [
pkgs.zlib
# ...
];
}
The problem I run into is that the current xcode package does not have hashes that seem to correspond to any version I can download on Apple's website. In addition, the versions it nominally supports are extremely old (only the 10.x series and 11).
What's the idiomatic way to get this working? Thanks.
Related
Background
When I've added the first Overlay for Nixpkgs, I found out that a bunch of system utils got built for a different version:
these derivations will be built:
/nix/store/028dqnwq36xja16gba3gckq5mcprpn06-postfix-main.cf.drv
/nix/store/b2sch2538ny2drdf9zzknf38grn8d8r3-pcre-8.42.drv
/nix/store/i1k9ksk32ca441zap40z3zddy7bhqx3n-zlib-1.2.11.drv
/nix/store/sypawsb3cwqnnhdl1barv2d8nyvbsxyv-coreutils-8.29.drv
/nix/store/xa4vnajxck2zgvjwp7l71lm11hqnz32r-findutils-4.6.0.drv
...
which is time and space consuming. I tried to figure out what's going on then end up with this question.
Summary
The idea of self and super of overlays is that self is the accumulated result after all overlays have applied, while super is the result of applying the previous overlay.
I thought that an attribute that have not been touched by will be the same between self and super, but some are not.
let
nixpkgs = import <nixpkgs> {
overlays = [
# This overlay obtains self.bash and super.bash, and save them under the
# attrs "bash-from-self" and "bash-from-super" for further examination
(self: super: {
bash-from-self = self.bash;
bash-from-super = super.bash;
})
];
};
# Retrieve bash-from-self (self.bash) and bash-from-super (super.bash)
# from the overlayed nixpkgs
inherit (nixpkgs) bash-from-self bash-from-super;
in {
# Check if bash-from-self (self.bash) and bash-from-super (super.bash)
# are same
isBashSame = (bash-from-self == bash-from-super);
inherit bash-from-self bash-from-super;
}
The above evaluates to:
{ isBashSame = false;
bash-from-self = «derivation /nix/store/zvy7mbpxqlplqpflqn5xk9szx25s4mhg-bash-4.4-p23.drv»;
bash-from-super = «derivation /nix/store/2i91sj16snsphvjrbsa62z8m4zhs261c-bash-4.4-p23.drv»; }
Showing that self.bash and super.bash is not the same, even that the bash attribute isn't touched in any overlays. Why is this happening, or did have some concepts missing in mind?
Details
More different attributes
Besides of bash, there're more attributes that are different:
let
isAttrSame =
attrName:
let
nixpkgs = import <nixpkgs> {
overlays = [
(_self: _super: { inherit _self _super; })
];
};
self = nixpkgs._self."${attrName}";
super = nixpkgs._super."${attrName}";
isSame = self == super;
in
isSame
;
in {
coreutils = isAttrSame "coreutils";
bash = isAttrSame "bash";
zsh = isAttrSame "zsh";
zlib = isAttrSame "zlib";
stdenv = isAttrSame "stdenv";
findutils = isAttrSame "findutils";
gnutar = isAttrSame "gnutar";
gcc = isAttrSame "gcc";
}
{
bash = false;
coreutils = false;
findutils = false;
gcc = true;
gnutar = true;
stdenv = true;
zlib = false;
zsh = true;
}
The builder of self.bash is super.bash?
let
nixpkgs = import <nixpkgs> {
overlays = [
(self: super: {
bash-from-self = self.bash;
bash-from-super = super.bash;
})
];
};
inherit (nixpkgs) bash-from-self bash-from-super;
in {
bash-from-self-builder = bash-from-self.drvAttrs.builder;
bash-from-super-builder = bash-from-super.drvAttrs.builder;
bash-from-self-outPath = bash-from-self.outPath;
bash-from-super-outPath = bash-from-super.outPath;
}
{ bash-from-self-builder = "/nix/store/2ws9cmamvr7xyvdg4d2nnd1bmr1zjrrq-bootstrap-tools/bin/bash";
bash-from-self-outPath = "/nix/store/06z61jbgs0vkw4i9cqqf9yl7zsfkkhw2-bash-4.4-p23";
bash-from-super-builder = "/nix/store/06z61jbgs0vkw4i9cqqf9yl7zsfkkhw2-bash-4.4-p23/bin/bash";
bash-from-super-outPath = "/nix/store/2avim7j13k75k26w18g6br8gai869nm9-bash-4.4-p23"; }
bash-from-self-outPath is the bash-from-super-builder (06z61...khw2-bash-4.4-p23).
So super.bash uses self.bash to build itself, producing another bash (2avim...9nm9-bash-4.4-p23)?
bootstrap-tools/bash --builds--> self.bash --builds--> super.bash
Why this is a problem
I want some of my packages defined in an overlay to depend on bash, coreutils and stuff. I want to use the original version of them provided directly from <nixpkgs>, not those may be overwritten by later overlays. So, in this case, seems that I should choose super.* instead of self.* to be the dependencies.
But some super.stuff is not the original nixpkgs.stuff, this causes rebuilding them as there's no binary cache and is a waste of disk space. And self.stuff can probably be overwritten by later overlays. What can I do?
I'm trying to write a custom rule to compile C++ code using the cc_common API. Here's my current attempt at an implementation:
load("#bazel_tools//tools/cpp:toolchain_utils.bzl", "find_cpp_toolchain")
load("#bazel_tools//tools/build_defs/cc:action_names.bzl", "C_COMPILE_ACTION_NAME")
def _impl(ctx):
cc_toolchain = find_cpp_toolchain(ctx)
feature_configuration = cc_common.configure_features(
cc_toolchain = cc_toolchain,
unsupported_features = ctx.disabled_features,
)
compiler = cc_common.get_tool_for_action(
feature_configuration=feature_configuration,
action_name=C_COMPILE_ACTION_NAME
)
compile_variables = cc_common.create_compile_variables(
feature_configuration = feature_configuration,
cc_toolchain = cc_toolchain,
)
compiler_options = cc_common.get_memory_inefficient_command_line(
feature_configuration = feature_configuration,
action_name = C_COMPILE_ACTION_NAME,
variables = compile_variables,
)
outfile = ctx.actions.declare_file("test.o")
args = ctx.actions.args()
args.add_all(compiler_options)
ctx.actions.run(
outputs = [outfile],
inputs = ctx.files.srcs,
executable = compiler,
arguments = [args],
)
return [DefaultInfo(files = depset([outfile]))]
However, this fails with the error "execvp(external/local_config_cc/wrapped_clang, ...)": No such file or directory. I assume this is because get_tool_for_action returns a string representing a path, not a File object, so Bazel doesn't add wrapped_clang to the sandbox. Executing the rule with sandboxing disabled seems to confirm this, as it completes successfully.
Is there a way to implement this custom rule without disabling the sandbox?
If you use ctx.actions.run_shell you can add the files associated with the toolchain to the input (ctx.attr._cc_toolchain.files). Also, you'll want to add the compiler environment variables. E.g.
srcs = depset(ctx.files.srcs)
tools = ctx.attr._cc_toolchain.files
...
compiler_env = cc_common.get_environment_variables(
feature_configuration = feature_configuration,
action_name = C_COMPILE_ACTION_NAME,
variables = compiler_variables,
)
...
args = ctx.actions.args()
args.add_all(compiler_options)
ctx.actions.run_shell(
outputs = [outfile],
inputs = depset(transitive = [srcs, tools]), # Merge src and tools depsets
command = "{compiler} $*".format(compiler = compiler),
arguments = [args],
env = compiler_env,
)
Bazel doesn't add files as action inputs automatically, you have to do it explicitly, as you did in your second approach (ctx.attr._cc_toolchain.files). With that, ctx.actions.run should work just fine.
when developing in nix i usually use builtins.trace for debugging:
in code
a = { foo = "bar"; ... more complex nested attrset ...};
builtins.trace a a;
in nix-repl
using :p a on a complex data structure is really hard to read also.
the problem
however, the output is just a single line without any formating and for complex data structure this is barely usable for debugging.
the question
is there a pretty print function in nix which does some indentation and adding of newline? or even better colored output?
ideal output
i'd like to see something like this:
default = {
active = {
check_ssl = [
{
tags = [ "mycustomtag" ];
host = "kotalla.de";
ipv6 = false;
name = "ssl11";
}
{
tags = [ "mycustomtag" ];
host = "kotalla.de";
ipv6 = false;
name = "ssl2";
}
];
check_http = [
{
host = "kotalla.de";
port = 80;
url = "/foo";
contains = "Labor";
name = "http";
}
];
check_ssh = [
{
host = "mail.lastlog.de";
port = 20202;
name = "ssh";
}
];
};
my hack
https://github.com/kamilchm/go2nix/issues/22#issuecomment-347693233
what i'm aware of:
https://github.com/NixOS/nix/issues/832
https://github.com/Gabriel439/nixfmt
we've now written our own formatter:
https://github.com/nixcloud/nix-beautify
Nixfmt is a popular Nix code formatter:
https://github.com/Gabriel439/nixfmt
https://github.com/haskell-nix/hnix/tree/50e63f80afa8323b25b692533e731eea641e56af#parse--print
To parse a file with hnix and pretty print the result:
hnix file.nix
In my recently aided in the development of a Dataframe package for Torch. As the code base has quickly doubled there is a need to split the class into several sections for better organization and follow-up (issue #8).
A simple test-class would be a test.lua file in the root folder of the test-package:
test = torch.class('test')
function test:__init()
self.data = {}
end
function test:a()
print("a")
end
function test:b()
print("b")
end
Now the rockspec for this would simply be:
package = "torch-test"
version = "0.1-1"
source = {
url = "..."
}
description = {
summary = "A test class",
detailed = [[
Just an example
]],
license = "MIT/X11",
maintainer = "Jon Doe"
}
dependencies = {
"lua ~> 5.1",
"torch >= 7.0",
}
build = {
type = 'builtin',
modules = {
["test"] = 'test.lua',
}
}
In order to get multiple files to work for a single class it is necessary to return the class object initially created and pass it to the subsections. The above example can be put into the file structure:
\init.lua
\main.lua
\test-0.1-1.rockspec
\Extensions\a.lua
\Extensions\b.lua
The luarocks install/make copies the files according to 'require' syntax where each . signifies a directory and the .lua is left out, i.e. we need to change the rockspec to:
package = "torch-test"
version = "0.1-1"
source = {
url = "..."
}
description = {
summary = "A test class",
detailed = [[
Just an example
]],
license = "MIT/X11",
maintainer = "Jon Doe"
}
dependencies = {
"lua ~> 5.1",
"torch >= 7.0",
}
build = {
type = 'builtin',
modules = {
["test.init"] = 'init.lua',
["test.main"] = 'main.lua',
["test.Extensions.a"] = 'a.lua',
["test.Extensions.b"] = 'b.lua'
}
}
The above will thus create a test-folder where the packages reside together with the files and subdirectories. The class initialization now resides in the init.lua that returns the class object:
test = torch.class('test')
function test:__init()
self.data = {}
end
return test
The subclass-files now need to pickup the class object that is passed using loadfile() (see init.lua file below). The a.lua should now look like this:
local params = {...}
local test = params[1]
function test:a()
print("a")
end
and similar addition for the b.lua:
local params = {...}
local test = params[1]
function test:b()
print("b")
end
In order to glue everything together we have the init.lua file. The following is probably a little over-complicated but it takes care of:
Finding all extensions available and loading them (Note: requires lua filesystem that you should add to the rockspec and you still need to add each file into the rockspec or it won't be in the Extensions folder)
Identifies the paths folder
Loads the main.lua
Works in a pure testing environment without the package installed
The code for init.lua:
require 'lfs'
local file_exists = function(name)
local f=io.open(name,"r")
if f~=nil then io.close(f) return true else return false end
end
-- If we're in development mode the default path should be the current
local test_path = "./?.lua"
local search_4_file = "Extensions/load_batch"
if (not file_exists(string.gsub(test_path, "?", search_4_file))) then
-- split all paths according to ;
for path in string.gmatch(package.path, "[^;]+;") do
-- remove trailing ;
path = string.sub(path, 1, string.len(path) - 1)
if (file_exists(string.gsub(path, "?", "test/" .. search_4_file))) then
test_path = string.gsub(path, "?", "test/?")
break;
end
end
if (test_path == nil) then
error("Can't find package files in search path: " .. tostring(package.path))
end
end
local main_file = string.gsub(test_path,"?", "main")
local test = assert(loadfile(main_file))()
-- Load all extensions, i.e. .lua files in Extensions directory
ext_path = string.gsub(test_path, "[^/]+$", "") .. "Extensions/"
for extension_file,_ in lfs.dir (ext_path) do
if (string.match(extension_file, "[.]lua$")) then
local file = ext_path .. extension_file
assert(loadfile(file))(test)
end
end
return test
I hope this helps if you run into the same problem and find the documentation a little too sparse. If you happen to know a better solution, please share.
I have been playing with the RtspPlay1 from mooncatventures to try and stream a live stream from an ffmpeg streaming source with as little delay as possible. The problem is even when I modify the code to indicate the -analyzedelay 0 flags in RtspPlay1 it does not seem to do anything. I came to this conclusion because the delay is the same on my computer without the -analyzeduration 0 flag as the iOS device. Any thoughts would be helpful.
Here is the command I am trying to emulate on the iPhone:
ffplay rtp:///224.1.1.1:11326 -analyzeduration 0
Here is the modified code I tried with RtspPlay1:
forward_argc=1;
forward_argv[1] = "-analyzeduration";
forward_argv[2] = "0";
//forward_argv[3] = "30";
//forward_argv[4] = "-fast";
//forward_argv[5] = "-sync";
//forward_argv[6] = "video";
//forward_argv[7] = "-drp";
//forward_argv[8] = "-skipidct";
//forward_argv[9] = "10";
//forward_argv[10] = "-skiploop";
//forward_argv[11] = "50";
//forward_argv[12] = "-threads";
//forward_argv[13] = "5";
//argv[14] = "-an";
forward_argv[3] = cString;
NSLog(#"glflag %#\n ",[parms objectForKey:#"glflag"] );
if ([parms objectForKey:#"glflag"]!=#"1") {
forward_argv[4]="0";
}else {
forward_argv[4]="1";
}
forward_argc += 4;