mirror of
https://github.com/openssl/openssl.git
synced 2024-12-18 06:23:58 +08:00
unified build scheme: add a design document
This documents describes the three steps from build.info files via the %unified_info database to the build-file templates, along with some examples showing how the data gets processed along the way. Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
This commit is contained in:
Richard Levitte
parent
ddf1847dc8
commit
f83133a5ed
550
Configurations/README.design
Normal file
550
Configurations/README.design
Normal file
@ -0,0 +1,550 @@
|
||||
Design document for the unified scheme data
|
||||
===========================================
|
||||
|
||||
How are things connected?
|
||||
-------------------------
|
||||
|
||||
The unified scheme takes all its data from the build.info files seen
|
||||
throughout the source tree. These files hold the minimum information
|
||||
needed to build end product files from diverse sources. See the
|
||||
section on build.info files below.
|
||||
|
||||
From the information in build.info files, Configure builds up an
|
||||
information database as a hash table called %unified_info, which is
|
||||
stored in configdata.pm, found at the top of the build tree (which may
|
||||
or may not be the same as the source tree).
|
||||
|
||||
Configurations/common.tmpl uses the data from %unified_info to
|
||||
generate the rules for building end product files as well as
|
||||
intermediary files with the help of a few functions found in the
|
||||
build-file templates. See the section on build-file templates further
|
||||
down for more information.
|
||||
|
||||
build.info files
|
||||
----------------
|
||||
|
||||
As mentioned earlier, build.info files are meant to hold the minimum
|
||||
information needed to build output files, and therefore only (with a
|
||||
few possible exceptions [1]) have information about end products (such
|
||||
as scripts, library files and programs) and source files (such as C
|
||||
files, C header files, assembler files, etc). Intermediate files such
|
||||
as object files are rarely directly refered to in build.info files (and
|
||||
when they are, it's always with the file name extension .o), they are
|
||||
infered by Configure. By the same rule of minimalism, end product
|
||||
file name extensions (such as .so, .a, .exe, etc) are never mentioned
|
||||
in build.info. Their file name extensions will be infered by the
|
||||
build-file templates, adapted for the platform they are meant for (see
|
||||
sections on %unified_info and build-file templates further down).
|
||||
|
||||
The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare
|
||||
end products.
|
||||
|
||||
The variables SOURCE, DEPEND, INCLUDE and ORDINALS are indexed by a
|
||||
produced file, and their values are the source used to produce that
|
||||
particular produced file, extra dependencies, include directories
|
||||
needed, and ordinal files (explained further below.
|
||||
|
||||
All their values in all the build.info throughout the source tree are
|
||||
collected together and form a set of programs, libraries, engines and
|
||||
scripts to be produced, source files, dependencies, etc etc etc.
|
||||
|
||||
Let's have a pretend example, a very limited contraption of OpenSSL,
|
||||
composed of the program 'apps/openssl', the libraries 'libssl' and
|
||||
'libcrypto', an engine 'engines/ossltest' and their sources and
|
||||
dependencies.
|
||||
|
||||
# build.info
|
||||
LIBS=libcrypto libssl
|
||||
ORDINALS[libcrypto]=crypto
|
||||
ORDINALS[libssl]=ssl
|
||||
INCLUDE[libcrypto]=include
|
||||
INCLUDE[libssl]=include
|
||||
DEPEND[libssl]=libcrypto
|
||||
|
||||
This is the top directory build.info file, and it tells us that two
|
||||
libraries are to be built, there are some ordinals to be used to
|
||||
declare what symbols in those libraries are seen as public, the
|
||||
include directory 'include/' shall be used throughout when building
|
||||
anything that will end up in each library, and that the library
|
||||
'libssl' depend on the library 'libcrypto' to function properly.
|
||||
|
||||
# apps/build.info
|
||||
PROGRAMS=openssl
|
||||
SOURCE[openssl]=openssl.c
|
||||
INCLUDE[openssl]=.. ../include
|
||||
DEPEND[openssl]=../libssl
|
||||
|
||||
This is the build.info file in 'apps/', one may notice that all file
|
||||
paths mentioned are relative to the directory the build.info file is
|
||||
located in. This one tells us that there's a program to be built
|
||||
called 'apps/openssl' (the file name extension will depend on the
|
||||
platform and is therefore not mentioned in the build.info file). It's
|
||||
built from one source file, 'apps/openssl.c', and building it requires
|
||||
the use of '.' and 'include' include directories (both are declared
|
||||
from the point of view of the 'apps/' directory), and that the program
|
||||
depends on the library 'libssl' to function properly.
|
||||
|
||||
# crypto/build.info
|
||||
LIBS=../libcrypto
|
||||
SOURCE[../libcrypto]=aes.c evp.c cversion.c
|
||||
DEPEND[cversion.o]=buildinf.h
|
||||
|
||||
BEGINRAW[Makefile(unix)]
|
||||
crypto/buildinf.h : Makefile
|
||||
perl util/mkbuildinf.h "$(CC) $(CFLAGS)" "$(PLATFORM)" \
|
||||
> crypto/buildinf.h
|
||||
ENDRAW[Makefile(unix)]
|
||||
|
||||
This is the build.info file in 'crypto', and it tells us a little more
|
||||
about what's needed to produce 'libcrypto'. LIBS is used again to
|
||||
declare that 'libcrypto' is to be produced. This declaration is
|
||||
really unnecessary as it's already mentioned in the top build.info
|
||||
file, but can make the info file easier to understand. This is to
|
||||
show that duplicate information isn't an issue.
|
||||
|
||||
This build.info file informs us that 'libcrypto' is built from a few
|
||||
source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'.
|
||||
It also shows us that building the object file inferred from
|
||||
'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it
|
||||
also shows the possibility to include raw build-file statements in a
|
||||
build.info file, in this case showing how 'buildinf.h' is built on
|
||||
Unix-like operating systems.
|
||||
|
||||
Two things are worth an extra note:
|
||||
|
||||
'DEPEND[cversion.o]' mentiones an object file. DEPEND indexes is the
|
||||
only location where it's valid to mention them
|
||||
|
||||
Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as
|
||||
seen from the top directory, no exception.
|
||||
|
||||
# ssl/build.info
|
||||
LIBS=../libssl
|
||||
SOURCE[../libssl]=tls.c
|
||||
|
||||
This is the build.info file in 'ssl/', and it tells us that the
|
||||
library 'libssl' is built from the source file 'ssl/tls.c'.
|
||||
|
||||
# engines/build.info
|
||||
ENGINES=libossltest
|
||||
SOURCE[libossltest]=e_ossltest.c
|
||||
DEPEND[libossltest]=../libcrypto
|
||||
INCLUDE[libossltest]=../include
|
||||
|
||||
This is the build.info file in 'engines/', telling us that an engine
|
||||
called 'engines/libossltest' shall be built, that it's source is
|
||||
'engines/e_ossltest.c' and that the include directory 'include/' may
|
||||
be used when building anything that will be part of this engine.
|
||||
Finally, the engine 'engines/libossltest' depends on the library
|
||||
'libcrypto' to function properly.
|
||||
|
||||
When Configure digests these build.info files, the accumulated
|
||||
information comes down to this:
|
||||
|
||||
LIBS=libcrypto libssl
|
||||
ORDINALS[libcrypto]=crypto
|
||||
SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c
|
||||
DEPEND[crypto/cversion.o]=crypto/buildinf.h
|
||||
INCLUDE[libcrypto]=include
|
||||
ORDINALS[libssl]=ssl
|
||||
SOURCE[libssl]=ssl/tls.c
|
||||
INCLUDE[libssl]=include
|
||||
DEPEND[libssl]=libcrypto
|
||||
|
||||
PROGRAMS=apps/openssl
|
||||
SOURCE[apps/openssl]=apps/openssl.c
|
||||
INCLUDE[apps/openssl]=. include
|
||||
DEPEND[apps/openssl]=libssl
|
||||
|
||||
ENGINES=engines/libossltest
|
||||
SOURCE[engines/libossltest]=engines/e_ossltest.c
|
||||
DEPEND[engines/libossltest]=libcrypto
|
||||
INCLUDE[engines/libossltest]=include
|
||||
|
||||
BEGINRAW[Makefile(unix)]
|
||||
crypto/buildinf.h : Makefile
|
||||
perl util/mkbuildinf.h "$(CC) $(CFLAGS)" "$(PLATFORM)" \
|
||||
> crypto/buildinf.h
|
||||
ENDRAW[Makefile(unix)]
|
||||
|
||||
|
||||
A few notes worth mentioning:
|
||||
|
||||
LIBS may be used to declare routine libraries only.
|
||||
|
||||
PROGRAMS may be used to declare programs only.
|
||||
|
||||
ENGINES may be used to declare engines only.
|
||||
|
||||
The indexes for SOURCE, INCLUDE and ORDINALS must only be end product
|
||||
files, such as libraries, programs or engines. The values of SOURCE
|
||||
variables must only be source files (possibly generated)
|
||||
|
||||
DEPEND shows a relationship between different end product files, such
|
||||
as a program depending on a library, or between an object file and
|
||||
some extra source file.
|
||||
|
||||
When Configure processes the build.info files, it will take it as
|
||||
truth without question, and will therefore perform very few checks.
|
||||
If the build tree is separate from the source tree, it will assume
|
||||
that all built files and up in the build directory and that all source
|
||||
files are to be found in the source tree, if they can be found there.
|
||||
Configure will assume that source files that can't be found in the
|
||||
source tree (such as 'crypto/bildinf.h' in the example above) are
|
||||
generated and will be found in the build tree.
|
||||
|
||||
|
||||
The %unified_info database
|
||||
--------------------------
|
||||
|
||||
The information in all the build.info get digested by Configure and
|
||||
collected into the %unified_info database, divided into the following
|
||||
indexes:
|
||||
|
||||
depends => a hash table containing 'file' => [ 'dependency' ... ]
|
||||
pairs. These are directly inferred from the DEPEND
|
||||
variables in build.info files.
|
||||
|
||||
engines => a list of engines. These are directly inferred from
|
||||
the ENGINES variable in build.info files.
|
||||
|
||||
includes => a hash table containing 'file' => [ 'include' ... ]
|
||||
pairs. These are directly inferred from the INCLUDE
|
||||
variables in build.info files.
|
||||
|
||||
libraries => a list of libraries. These are directly inferred from
|
||||
the LIBS variable in build.info files.
|
||||
|
||||
ordinals => a hash table containing 'file' => [ 'word', 'ordfile' ]
|
||||
pairs. 'file' and 'word' are directly inferred from
|
||||
the ORDINALS variables in build.info files, while the
|
||||
file 'ofile' comes from internal knowledge in
|
||||
Configure.
|
||||
|
||||
programs => a list of programs. These are directly inferred from
|
||||
the PROGRAMS variable in build.info files.
|
||||
|
||||
rawlines => a list of build-file lines. These are a direct copy of
|
||||
the BEGINRAW..ENDRAW lines in build.info files. Note:
|
||||
only the BEGINRAW..ENDRAW section for the current
|
||||
platform are copied, the rest are ignored.
|
||||
|
||||
scripts => a list of scripts. There are directly inferred from
|
||||
the SCRIPTS variable in build.info files.
|
||||
|
||||
sources => a hash table containing 'file' => [ 'sourcefile' ... ]
|
||||
pairs. These are indirectly inferred from the SOURCE
|
||||
variables in build.info files. Object files are
|
||||
mentioned in this hash table, with source files from
|
||||
SOURCE variables, and AS source files for programs and
|
||||
libraries.
|
||||
|
||||
As an example, here is how the build.info files example from the
|
||||
section above would be digested into a %unified_info table:
|
||||
|
||||
our %unified_info = (
|
||||
"depends" =>
|
||||
{
|
||||
"apps/openssl" =>
|
||||
[
|
||||
"libssl",
|
||||
],
|
||||
"crypto/cversion.o" =>
|
||||
[
|
||||
"crypto/buildinf.h",
|
||||
],
|
||||
"engines/libossltest" =>
|
||||
[
|
||||
"libcrypto",
|
||||
],
|
||||
"libssl" =>
|
||||
[
|
||||
"libcrypto",
|
||||
],
|
||||
},
|
||||
"engines" =>
|
||||
[
|
||||
"engines/libossltest",
|
||||
],
|
||||
"includes" =>
|
||||
{
|
||||
"apps/openssl" =>
|
||||
[
|
||||
".",
|
||||
"include",
|
||||
],
|
||||
"engines/libossltest" =>
|
||||
[
|
||||
"include"
|
||||
],
|
||||
"libcrypto" =>
|
||||
[
|
||||
"include",
|
||||
],
|
||||
"libssl" =>
|
||||
[
|
||||
"include",
|
||||
],
|
||||
}
|
||||
"libraries" =>
|
||||
[
|
||||
"libcrypto",
|
||||
"libssl",
|
||||
],
|
||||
"ordinals" =>
|
||||
{
|
||||
"libcrypto" =>
|
||||
[
|
||||
"crypto",
|
||||
"util/libeay.num",
|
||||
],
|
||||
"libssl" =>
|
||||
[
|
||||
"ssl",
|
||||
"util/ssleay.num",
|
||||
],
|
||||
},
|
||||
"programs" =>
|
||||
[
|
||||
"apps/openssl",
|
||||
],
|
||||
"rawlines" =>
|
||||
[
|
||||
"crypto/buildinf.h : Makefile",
|
||||
" perl util/mkbuildinf.h \"\$(CC) \$(CFLAGS)\" \"\$(PLATFORM)\" \\"
|
||||
" > crypto/buildinf.h"
|
||||
],
|
||||
"sources" =>
|
||||
{
|
||||
"apps/openssl" =>
|
||||
[
|
||||
"apps/openssl.o",
|
||||
],
|
||||
"apps/openssl.o" =>
|
||||
[
|
||||
"apps/openssl.c",
|
||||
],
|
||||
"crypto/aes.o" =>
|
||||
[
|
||||
"crypto/aes.c",
|
||||
],
|
||||
"crypto/cversion.o" =>
|
||||
[
|
||||
"crypto/cversion.c",
|
||||
],
|
||||
"crypto/evp.o" =>
|
||||
[
|
||||
"crypto/evp.c",
|
||||
],
|
||||
"engines/e_ossltest.o" =>
|
||||
[
|
||||
"engines/e_ossltest.c",
|
||||
],
|
||||
"engines/libossltest" =>
|
||||
[
|
||||
"engines/e_ossltest.o",
|
||||
],
|
||||
"libcrypto" =>
|
||||
[
|
||||
"crypto/aes.c",
|
||||
"crypto/cversion.c",
|
||||
"crypto/evp.c",
|
||||
],
|
||||
"libssl" =>
|
||||
[
|
||||
"ssl/tls.c",
|
||||
],
|
||||
"ssl/tls.o" =>
|
||||
[
|
||||
"ssl/tls.c",
|
||||
],
|
||||
},
|
||||
);
|
||||
|
||||
As can be seen, everything in %unified_info is fairly simple nuggest
|
||||
of information. Still, it tells us that to build all programs, we
|
||||
must build 'apps/openssl', and to build the latter, we will need to
|
||||
build all its sources ('apps/openssl.o' in this case) and all the
|
||||
other things it depends on (such as 'libssl'). All those dependencies
|
||||
need to be built as well, using the same logic, so to build 'libssl',
|
||||
we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the
|
||||
latter...
|
||||
|
||||
|
||||
Build-file templates
|
||||
--------------------
|
||||
|
||||
Build-file templates are essentially build-files (such as Makefile on
|
||||
Unix) with perl code fragments mixed in. Those perl code fragment
|
||||
will generate all the configuration dependent data, including all the
|
||||
rules needed to build end product files and intermediary files alike.
|
||||
At a minimum, there must be a perl code fragment that defines a set of
|
||||
functions that are used to generates specific build-file rules, to
|
||||
build static libraries from object files, to build shared libraries
|
||||
from static libraries, to programs from object files and libraries,
|
||||
etc.
|
||||
|
||||
src2dep - function that produces build file lines to get the
|
||||
dependencies for an object file into a dependency
|
||||
file.
|
||||
|
||||
It's called like this:
|
||||
|
||||
src2dep(obj => "PATH/TO/objectfile",
|
||||
srcs => [ "PATH/TO/sourcefile", ... ],
|
||||
incs => [ "INCL/PATH", ... ]);
|
||||
|
||||
'obj' has the dependent object file as well as
|
||||
object file the dependencies are for; it's *without*
|
||||
extension, src2dep() is expected to add that.
|
||||
'srcs' has the list of source files to build the
|
||||
object file, with the first item being the source
|
||||
file that directly corresponds to the object file.
|
||||
'incs' is a list of include file directories.
|
||||
|
||||
src2obj - function that produces build file lines to build an
|
||||
object file from source files and associated data.
|
||||
|
||||
It's called like this:
|
||||
|
||||
src2obj(obj => "PATH/TO/objectfile",
|
||||
srcs => [ "PATH/TO/sourcefile", ... ],
|
||||
deps => [ "dep1", ... ],
|
||||
incs => [ "INCL/PATH", ... ]);
|
||||
|
||||
'obj' has the intended object file *without*
|
||||
extension, src2obj() is expected to add that.
|
||||
'srcs' has the list of source files to build the
|
||||
object file, with the first item being the source
|
||||
file that directly corresponds to the object file.
|
||||
'deps' is a list of dependencies. 'incs' is a list
|
||||
of include file directories.
|
||||
|
||||
obj2lib - function that produces build file lines to build a
|
||||
static library file ("libfoo.a" in Unix terms) from
|
||||
object files.
|
||||
|
||||
called like this:
|
||||
|
||||
obj2lib(lib => "PATH/TO/libfile",
|
||||
objs => [ "PATH/TO/objectfile", ... ]);
|
||||
|
||||
'lib' has the intended library file name *without*
|
||||
extension, obj2lib is expected to add that. 'objs'
|
||||
has the list of object files (also *without*
|
||||
extension) to build this library.
|
||||
|
||||
libobj2shlib - function that produces build file lines to build a
|
||||
shareable object library file ("libfoo.so" in Unix
|
||||
terms) from the corresponding static library file
|
||||
or object files.
|
||||
|
||||
called like this:
|
||||
|
||||
libobj2shlib(shlib => "PATH/TO/shlibfile",
|
||||
lib => "PATH/TO/libfile",
|
||||
objs => [ "PATH/TO/objectfile", ... ],
|
||||
deps => [ "PATH/TO/otherlibfile", ... ],
|
||||
ordinals => [ "word", "/PATH/TO/ordfile" ]);
|
||||
|
||||
'lib' has the intended library file name *without*
|
||||
extension, libobj2shlib is expected to add that.
|
||||
'shlib' has the correcponding shared library name
|
||||
*without* extension. 'deps' has the list of other
|
||||
libraries (also *without* extension) this library
|
||||
needs to be linked with. 'objs' has the list of
|
||||
object files (also *without* extension) to build
|
||||
this library. 'ordinals' MAY be present, and when
|
||||
it is, its value is an array where the word is
|
||||
"crypto" or "ssl" and the file is one of the ordinal
|
||||
files util/libeay.num or util/ssleay.num in the
|
||||
source directory.
|
||||
|
||||
This function has a choice; it can use the
|
||||
corresponding static library as input to make the
|
||||
shared library, or the list of object files.
|
||||
|
||||
obj2dynlib - function that produces build file lines to build a
|
||||
dynamically loadable library file ("libfoo.so" on
|
||||
Unix) from object files.
|
||||
|
||||
called like this:
|
||||
|
||||
obj2dynlib(lib => "PATH/TO/libfile",
|
||||
objs => [ "PATH/TO/objectfile", ... ],
|
||||
deps => [ "PATH/TO/otherlibfile",
|
||||
... ]);
|
||||
|
||||
This is almost the same as libobj2shlib, but the
|
||||
intent is to build a shareable library that can be
|
||||
loaded in runtime (a "plugin"...). The differences
|
||||
are subtle, one of the most visible ones is that the
|
||||
resulting shareable library is produced from object
|
||||
files only.
|
||||
|
||||
obj2bin - function that produces build file lines to build an
|
||||
executable file from object files.
|
||||
|
||||
called like this:
|
||||
|
||||
obj2bin(bin => "PATH/TO/binfile",
|
||||
objs => [ "PATH/TO/objectfile", ... ],
|
||||
deps => [ "PATH/TO/libfile", ... ]);
|
||||
|
||||
'bin' has the intended executable file name
|
||||
*without* extension, obj2bin is expected to add
|
||||
that. 'objs' has the list of object files (also
|
||||
*without* extension) to build this library. 'deps'
|
||||
has the list of library files (also *without*
|
||||
extension) that the programs needs to be linked
|
||||
with.
|
||||
|
||||
in2script - function that produces build file lines to build a
|
||||
script file from some input.
|
||||
|
||||
called like this:
|
||||
|
||||
in2script(script => "PATH/TO/scriptfile",
|
||||
sources => [ "PATH/TO/infile", ... ]);
|
||||
|
||||
'script' has the intended script file name.
|
||||
'sources' has the list of source files to build the
|
||||
resulting script from.
|
||||
|
||||
Along with the build-file templates is the driving engine
|
||||
Configurations/common.tmpl, which looks through all the information in
|
||||
%unified_info and generates all the rulesets to build libraries,
|
||||
programs and all intermediate files, using the rule generating
|
||||
functions defined in the build-file template.
|
||||
|
||||
As an example with the smaller build.info set we've seen as an
|
||||
example, producing the rules to build 'libssl' would result in the
|
||||
following calls:
|
||||
|
||||
# Note: libobj2shlib will only be called if shared libraries are
|
||||
# to be produced.
|
||||
# Note 2: libobj2shlib gets both the name of the static library
|
||||
# and the names of all the object files that go into it. It's up
|
||||
# to the implementation to decide which to use as input.
|
||||
libobj2shlib(shlib => "libssl",
|
||||
lib => "libssl",
|
||||
objs => [ "ssl/tls.o" ],
|
||||
deps => [ "libcrypto" ]
|
||||
ordinals => [ "ssl", "util/ssleay.num" ]);
|
||||
|
||||
obj2lib(lib => "libssl"
|
||||
objs => [ "ssl/tls.o" ]);
|
||||
|
||||
# Note 3: common.tmpl peals off the ".o" extension, as the
|
||||
# platform at hand may have a different one.
|
||||
src2obj(obj => "ssl/tls"
|
||||
srcs => [ "ssl/tls.c" ],
|
||||
deps => [ ],
|
||||
incs => [ "include" ]);
|
||||
|
||||
src2dep(obj => "ssl/tls"
|
||||
srcs => [ "ssl/tls.c" ],
|
||||
incs => [ "include" ]);
|
||||
|
||||
The returned strings from all those calls are then concatenated
|
||||
together and written to the resulting build-file.
|
||||
Loading…
Reference in New Issue
Block a user