Automake include all libraries - clang

My directory structure looks like the following :
-misc
-- misc.h
-- misc.c
-- misc.la
-dict
-- dict.h
-- dict.c
-- dict.la
-main
-- main.c
I'm trying to build main.c into a program. main.c contains a reference to a function in dict.c and dict.c contains references to misc.c
How do I tell automake to include dict.la, misc.la to build main ?
In other words, I want to include all the *.la files that were built by the top level make.
This is of course just an example, in the real scenario, i'm dealing with more than 20 such libraries each in a directory similar to the above.

the recursive make case
You sound as if you were using recursive make with one top level Makefile.am, one main/Makefile.am and one FOO/Makefile.am for FOO in misc, dict, and the 20 other such libraries.
If possible, I would try to change to using non-recursive make for this use case (see below).
In that recursive make case, you will need to adapt main/Makefile.am to link main with all those libraries, and to make sure those libraries are actually (re)built if necessary:
bin_PROGRAMS = mainprog
mainprog_SOURCES = main.c
mainprog_LDADD =
mainprog_LDADD += $(top_builddir)/misc/libmisc.la
$(top_builddir)/misc/libmisc.la:
cd $(top_builddir)/misc && $(MAKE) libmisc.la
You will need to repeat that _LDADD += and build rule pattern for each and every of those 20+ libraries.
the non-recursive make case
Personally, I would switch to using non-recursive make with one single top level Makefile.am, while putting both the mainprog executable and the 20+ libfoo.la files into the $(top_builddir). Then adding a library to the main program can be reduced to adding a single line
mainprog_LDADD += libmisc.la
without any extra rules being required. This also parallelizes building of those 20+ libraries, which is a very welcome side effect.
The new top level Makefile.am would contain one line per library like
include misc/Makefile-files
ending with one line
include main/Makefile-files
and the file misc/Makefile-files would contain something like
# -*- Makefile -*-
lib_LTLIBRARIES += libmisc.la
libmisc_la_SOURCES = %reldir%/misc.c
while main/Makefile-files would look like
# -*- Makefile -*-
bin_PROGRAMS += mainprog
mainprog_SOURCES = main.c
mainprog_CPPFLAGS =
mainprog_LDADD =
# repeat for 20+ libraries
mainprog_LDADD += dict/libdict.la
mainprog_LDADD += misc/libmisc.la
If your main program uses #include "misc.h", you will need 20+ per library mainprog_CPPFLAGS += -I$(top_srcdir)/misc additions. If you could change that to #include "misc/misc.h", a single mainprog_CPPFLAGS = -I$(top_srcdir) will do instead.
You could also include main/Makefile-files in before all those library Makefile-files and consistently add the respective library to mainprog in the library Makefile-files:
bin_PROGRAMS += mainprog
mainprog_SOURCES = main.c
mainprog_CPPFLAGS =
mainprog_LDADD =
with misc/Makefile-files containing
lib_LTLIBRARIES += libmisc.la
libmisc_la_SOURCES = %reldir%/misc.c
mainprog_CPPFLAGS += -I$(top_srcdir)/%reldir%
mainprog_LDADD += libmisc.la
If you can guarantee that all the lib_LTLIBRARIES built from the top level Makefile.am are to be linked to the main program, and that the main program sources can use #include "misc/misc.h", the following top level Makefile.am will do:
include main/Makefile-files
# 20+ library includes
include dict/Makefile-files
include misc/Makefile-files
mainprog_LDADD += $(lib_LTLIBRARIES)
with main/Makefile-files being
bin_PROGRAMS += mainprog
mainprog_SOURCES = main.c
mainprog_LDADD =
and misc/Makefile-files and the other libraries:
lib_LTLIBRARIES += libmisc.la
libmisc_la_SOURCES = misc.c
For a complete working example of a single library and a main program using non-recursive make for inspiration, see my answer https://stackoverflow.com/a/60306382/182675 and the github example I built for that.

Related

qmake 4.8.4 has broken custom targets due to capitalisation. How do I work around it?

My .pro file has extra stuff in it:
win32 {
OUT_PWD_SHELL = $$replace(OUT_PWD, /, \\)
autoversion.target = $$OUT_PWD\\autoversioninfo.h
autoversion.depends = FORCE
autoversion.commands = $$PWD/../../AutoBuildVersion.exe $$replace(PWD, /, \\) $$OUT_PWD_SHELL
QMAKE_EXTRA_TARGETS += autoversion
PRE_TARGETDEPS += $$OUT_PWD\\autoversioninfo.h
}
This fails to work as expected because in the generated makefile DESTDIR_TARGET has a new dependency added that starts d:\ but the rule generated for autoversion starts with D:/. I can improve this slightly by replacing all / with \, but the case sensitivity still breaks it and the target is not built.
If I remove the full path from autoversion.target and PRE_TARGETDEPS then it solves that problem, but then when calculating dependencies, the rule for the cpp file that includes the generated header changes to give an explicit path to the header in the dependencies, and that path points to the source directory and not the output directory where the generated file is produced. This causes make to barf and not produce the generated file.
I don't know why qmake changes the case handling of the drive, it is very irritating, but how do I get this all to work correctly?
There is no good solution. The best I came up with is to use a phony target that always runs to generate the header file. On the downside this slows the build when the header file already exists, but on the upside, it allows the build to complete.
win32 {
OUT_PWD_SHELL = $$replace(OUT_PWD, /, \\)
gen_autoversion.target = GENERATE_AUTOVERSIONINFO
gen_autoversion.commands = $$PWD/../../AutoBuildVersion.exe $$replace(PWD, /, \\) $$OUT_PWD_SHELL
QMAKE_EXTRA_TARGETS += gen_autoversion
PRE_TARGETDEPS += GENERATE_AUTOVERSIONINFO
}
I am not sure that PRE_TARGETDEPS is actually needed here.

Qmake get the target output file path

I have a qmake project that looks like this:
TEMPLATE = lib
CONFIG += dll
TARGET = mydll
SOURCES += ...
HEADERS += ....
Now I want to add an INSTALLS section, so I have:
target.path = /path/to/somedir/
target.files =./$$TARGET
INSTALLS+= target
Unfortunately this will not work, because $$TARGET contains the target name, and not the output file name. Is there a portable way to obtain the output file name? (Please no platform dependent string concatenation like lib + $$TARGET + .so)
You don't have to specify target.files, target is a special case and it's predefined in qmake.
http://qt-project.org/doc/qt-4.8/qmake-environment-reference.html#installs
If you append a built-in install set to the INSTALLS variable and do not specify files or extra members, qmake will decide what needs to be copied for you. Currently, the only supported built-in install set is target:
target.path = /usr/local/myprogram
INSTALLS += target
In the above lines, qmake knows what needs to be copied, and will handle the installation process automatically.

GNU m4 macros auto generated file

I download latex package on which I want do some changes, but in this packege exist file include.m4 and I don't know what it does and how it was generated. Here its lines:
m4_changequote([[, ]])m4_dnl
m4_dnl
m4_define([[m4_FILE_INIT]], [[m4_dnl
%
% This is automaticaly generated file, do not edit it.
%
]])m4_dnl
m4_dnl
m4_define([[m4_FILE_ID]], [[m4_dnl
m4_patsubst([[$1]], [[\$Date::? \([0-9]+\)-\([0-9]+\)-\([0-9]+\).*]], [[\1/\2/\3]])m4_dnl
v[[]]m4_ESKDX_VERSION]])m4_dnl
m4_dnl
m4_define([[m4_FILE_DATE]], [[m4_dnl
m4_patsubst([[$1]], [[\$Date::? \([0-9]+\)-\([0-9]+\)-\([0-9]+\).*]], [[\1/\2/\3]])]])m4_dnl
m4_dnl
Can you explain with which tool it was generated?
Thk. So this file is not autogenerated? ANd can you help me understand these lines from Makefile:
M4FLAGS = -P -Dm4_ESKDX_INIT="m4_include($(TOP_DIR)/include.m4)" \
-Dm4_ESKDX_VERSION=$(VERSION) -Dm4_ESKDX_DATE=$(RELEASE_DATE)
And rule:
%.def: %.def.in $(M4DEPS)
m4 $(M4FLAGS) $< >$#
%.sty: %.sty.in $(M4DEPS)
m4 $(M4FLAGS) $< >$#
%.cls: %.cls.in $(M4DEPS)
m4 $(M4FLAGS) $< >$#
As I can see GNU m4 options '-D' substitutes macro m4_ESKDX_INIT in .sty .cls files to m4_include(../include.m4) and then options '-P' first expands file include.m4 and furthemore expands macros in include.m4.
This is a macro for the GNU m4 macro processor. This file is designed to be used with the -P or --prefix-builtins commandline option. The m4_ part will be stripped away when m4 evaluates this file. This file doesn't do anything itself, it just defines three macros (FILE_INIT, FILE_ID and FILE_DATE) which presumably will be used in another step. You might want to look in the other files for references to this one. The basic idea will be to load this file before running another file through m4 and it will replace those macros as it goes.
The message about automatically generated is supposed to end up in the final file as a comment. As we can see in the rules in the Makefile, each of the .def, .sty and .cls files are generated from an equivalently named .in file (so result.cls will be built from result.cls.in. by evaluating the macros in these files and replacing them with the equivalents.
So, to modify these files, you will want to edit the .in files.

qmake extra compiler with multiple outputs per file

As described in Undocumented qmake, I declared an extra compiler in my qmake project file:
TEST = somefile.h
test_c.output = ${QMAKE_VAR_OBJECTS_DIR}${QMAKE_FILE_BASE}_1.cpp
test_c.input = TEST
test_c.commands = C:/somedir/some.exe /i ${QMAKE_FILE_IN} /o ${QMAKE_VAR_OBJECTS_DIR}${QMAKE_FILE_BASE}_1.cpp
test_c.variable_out = SOURCES
test_c.name = MyTestC
QMAKE_EXTRA_COMPILERS += test_c
And this works fine. But I also want to generate a header file. I can easily make a second custom tool for parsing this file (or files, if >1 will be in TEST), but I don't want to parse each file twice. I tried:
test_c.output = ${QMAKE_VAR_OBJECTS_DIR}${QMAKE_FILE_BASE}_1.cpp \
${QMAKE_VAR_OBJECTS_DIR}${QMAKE_FILE_BASE}_2.cpp
Just to test that the extra compiler can make two files per run. I expected some error like "file somefile_2.cpp doesn't exist", but project compiles without errors and second output file is ignored. In Makefile somefile_2.cpp is not present.
Now I'm thinking about two variants:
Make an extra compiler that produces an archive, where all needed output files will be saved at once. Set tool1.variable_out = TOOL_1_OUT, and add two more extra compilers with toolN.input = TOOL_1_OUT to just "unzip" the archived files (one per tool) and append them to some variables.
In this case three executes will be called per one input file. This is not optimal, but at least the parser will run only once per file.
Experiment with the .output_function option. Make a qmake function that returns the same name as .output now does, but also append second filename to HEADERS.
P.S. I am using MinGW x32 4.7, QtCreator 2.7.1, Qt 5.1.0, C++11.
Your variant #2 is the right idea. This works for me:
defineReplace(addToHeaders) {
source = $$1
source_split = $$split(source, ".")
source_without_extension = $$first(source_split)
HEADERS += ${QMAKE_VAR_OBJECTS_DIR}$${source_without_extension}_1.h
return(${QMAKE_VAR_OBJECTS_DIR}$${source_without_extension}_1.cpp)
}
defineReplace(FILE_IN_addToHeaders) {
# qmake emits a warning unless this function is defined; not sure why.
}
TEST = somefile.h
test_c.output_function = addToHeaders
test_c.input = TEST
test_c.commands = cp ${QMAKE_FILE_IN} ${QMAKE_VAR_OBJECTS_DIR}${QMAKE_FILE_BASE}_1.cpp ; cp ${QMAKE_FILE_IN} ${QMAKE_VAR_OBJECTS_DIR}${QMAKE_FILE_BASE}_1.h
test_c.variable_out = SOURCES
test_c.name = MyTestC
QMAKE_EXTRA_COMPILERS += test_c
It produces a Makefile which builds somefile_1.cpp and somefile_1.h, with somefile_1.cpp added to SOURCES and somefile_1.h added to HEADERS.
This works ok (variant #1):
MY_COMP = src/precompiled.h \
src/file2.h
GENERATE_FOLDER = generated/
# build package file
my_build.clean = $${GENERATE_FOLDER}gen_${QMAKE_FILE_BASE}.pack
my_build.depends = [somepath]/my_precompiler.exe
my_build.output = $${GENERATE_FOLDER}gen_${QMAKE_FILE_BASE}.pack
my_build.input = MY_COMP
my_build.commands = [somepath]/my_precompiler.exe /i ${QMAKE_FILE_IN} /o $${GENERATE_FOLDER}gen_${QMAKE_FILE_BASE}.pack
my_build.variable_out = MY_PACKAGES
my_build.name = "package build"
# unpack cpp
my_unpack_cpp.clean = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.cpp
my_unpack_cpp.depends = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.h
my_unpack_cpp.output = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.cpp
my_unpack_cpp.input = MY_PACKAGES
my_unpack_cpp.commands = [somepath]/my_precompiler.exe /unpack cpp /i ${QMAKE_FILE_IN} /o $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.cpp
my_unpack_cpp.variable_out = GENERATED_SOURCES
my_unpack_cpp.dependency_type = TYPE_C
my_unpack_cpp.name = "unpack code"
my_unpack_cpp.CONFIG = no_link
# unpack header
my_unpack_h.clean = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.h
my_unpack_h.output = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.h
my_unpack_h.input = MY_PACKAGES
my_unpack_h.commands = [somepath]/my_precompiler.exe /unpack h /i ${QMAKE_FILE_IN} /o $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.h
my_unpack_h.variable_out = HEADERS
my_unpack_h.name = "unpack header"
my_unpack_h.CONFIG = no_link
QMAKE_EXTRA_COMPILERS += my_build my_unpack_h my_unpack_cpp
With this technique number of output files per one parse may vary, but may be constant for all files in project, of course.
In my_precompiler I parse file if unpack option isn't preserved and build two files (cpp + h) into two QBuffers. After that I simply write builded data to QFile:
QDataStream ds(&file);
ds.setVersion(QDataStream::Qt_5_1);
ds << qCompress(output_cpp.data(), 9);
ds << qCompress(output_h.data(), 9);
file.close();
In fact, now qCompress isn't profitable, because generated files too small to compression size exceeded the size of the headers zlib - sizeof(.pack) > size(.h + .h).
Unpacking:
QByteArray ba;
QDataStream ds(&file);
ds.setVersion(QDataStream::Qt_5_1);
ds >> ba;
if(unpack != "cpp")
{
ds >> ba;
}
file.close();
ba = qUncompress(ba);
file.setFileName(output);
if(!file.open(QFile::WriteOnly | QFile::Truncate)) return 1;
file.write(ba);
file.close();
When generating:
Write #include "original header" in begin of generated header
Write #include "generated header" in begin of generated code
Therefore I set this:
my_unpack_cpp.depends = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.h
So /unpack cpp (and, therefore, building) performs after building needed header file. And this:
my_build.depends = [somepath]/my_precompiler.exe
Sets builded pack (and, therefore, generated cpp+h) depends on my_precompiler, so all will be rebuilded if I modify and rebuild precompiler.
P.S. IMHO these lines must works as cleaners before rebuilding:
my_build.clean = $${GENERATE_FOLDER}gen_${QMAKE_FILE_BASE}.pack
my_unpack_cpp.clean = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.cpp
my_unpack_h.clean = $${GENERATE_FOLDER}${QMAKE_FILE_BASE}.h
But they don't :( At present I ignore that, but now if building .pack is failed than previously builded pack-file is used

How to quell qmake's "WARNING: Failure to find:"?

I'm using PRE_TARGETDEPS to generate source files, and I'm adding the generated source files to SOURCES for compilation.
The output of my generator obviously doesn't exist at the time qmake is run, so qmake outputs WARNING: Failure to find: for each of the to-be-created source files.
How can I quell this warning, since I know my PRE_TARGETDEPS is going to produce those files?
Or, is there a better way to generate intermediate files using qmake?
Example
Here's a complete test.pro file that exhibits the problem:
TEMPLATE = lib
preprocess.commands += cat test.cc.in | sed 's/a/0/g' > test.0.cc ;
preprocess.commands += cat test.cc.in | sed 's/a/1/g' > test.1.cc ;
preprocess.depends = test.cc.in
QMAKE_EXTRA_TARGETS += preprocess
PRE_TARGETDEPS += preprocess
SOURCES = test.0.cc test.1.cc
Place this in an empty folder, and also create an empty test.cc.in file. Run qmake, and you'll see these warnings:
WARNING: Failure to find: test.0.cc
WARNING: Failure to find: test.1.cc
How can I quell this warning
From my reading of the qmake code, it looks like you can:
either have qmake ignore any filenames that don't exist - in which case they won't get built by your later steps
or have it write these warnings
I don't think either of these would be satisfactory for you.
Here's my reasoning.... I had a hunt for the text Failure to find in a Qt distribution I had to hand: qt4.8.1.
It appeared 3 times in in qmake/generators/makefile.cpp. The two blocks of code look like this:
QStringList
MakefileGenerator::findFilesInVPATH(QStringList l, uchar flags, const QString &vpath_var)
{
....
debug_msg(1, "%s:%d Failure to find %s in vpath (%s)",
__FILE__, __LINE__,
val.toLatin1().constData(), vpath.join("::").toLatin1().constData());
if(flags & VPATH_RemoveMissingFiles)
remove_file = true;
else if(flags & VPATH_WarnMissingFiles)
warn_msg(WarnLogic, "Failure to find: %s", val.toLatin1().constData());
....
else if(flags & VPATH_WarnMissingFiles)
warn_msg(WarnLogic, "Failure to find: %s", val.toLatin1().constData());
and this is called with:
l = findFilesInVPATH(l, (comp.flags & Compiler::CompilerRemoveNoExist) ?
VPATH_RemoveMissingFiles : VPATH_WarnMissingFiles, "VPATH_" + comp.variable_in);
So the flags parameter passed in to the first block will be either RemoveMissingFiles or WarnMissingFiles, depending on comp.flags & Compiler::CompilerRemoveNoExist.
Or, is there a better way to generate intermediate files using qmake?
I'm not sure that it's better - i.e. it's certainly complex - but this is what is done where I work...
In the .pro file, a system call is done, that:
generates the required files,
and then writes out their names to stdout.
Here's an example from the .pro, to show how it would be called:
SOURCES += $$system( python my_script_name.py )
You can of course pass arguments in to the python script, if you like
Things to note/limitations:
This means that the python script gets run whenever you run qmake, but not during individual make invocations
Each invocation of python really slows down our qmake steps - taking roughly twice as long as running qmake without launching python - but you could always use a different scripting language
This would fix your problem, in that by the time qmake processes the SOURCES value, the files have been created by the script.

Resources