2:45 a.m.
Hello,
There has been quite some churn in Alpine abuild lately and I'd like to
take some time to go through the individual changes and note if they are
something we should cherry-pick or not. I expect most of these changes
are somewhat breaking and therefore should be considered a post-1.0
project, but if that is not the case I will remark as such.
Adélie's abuild is based on Alpine abuild version 3.3.1, released
Tuesday 05 March 2019. In all there have been 158 commits to Alpine
abuild since that time culminating in the release of version 3.6.0 on
Thursday 28 May 2020. There have been no new changes since then, but
there are 12 open merge requests. I will take a look at those also in a
future email.
Please reply if you disagree with any of my assessments or have other
comments.
On a related note, I think we should consider changing the name of the
package to "adelie-abuild" if we continue to release versions that do
not correspond to Alpine's versions. Already we have released versions
3.3.1+adelie, 3.4, and 3.4.1 which have no such correspondence.
Taking a quick look at the patches we carry for abuild in packages.git,
it seems we never applied the "keyhole" patch to our abuild.git. Since I
sent that patch to Alpine as well, that will come up again naturally.
This email is part 1 of 2 where I will look only at changes to
cherry-pick from Alpine. Part 2 will look at sending some of our own
changes back to Alpine.
All commits in their respective sections are sorted oldest to newest.
===== Already applied =====
The first commit is patched in packages.git but needs to be applied to
abuild.git; the latter is present in both.
297de93aef abuild-sudo: don't allow --keys-dir
93cc98930d abuild: Use $APK instead of apk
===== Already addressed =====
77746a0c3d abuild-fetch: enable curl certificate verification
369e7069b8 Revert "abuild: replace command -v with which to fix build issues"
f83d19ce79 newapkbuild: remove obsolete cd statements
635a699365 newapkbuild: fix empty function regression
===== Bootstrapping =====
I think the change in apk-tools it talks about is something we've picked
up so we should apply this as well. As such we should probably apply
this ASAP.
c54d39d8aa abuild: rename makedepends_host virtual package
===== newapkbuild =====
Not really useful, but ok.
da4aca278f Cosmetic: newapkbuild: comment for check sections
This would be good to add.
b743186dfe newapkbuild.in: add default check() for meson packages.
Already applied in the same commit as the "cd" one.
2122fa7476 newapkbuild: use current directory for cmake
We bundle setuptools with the main Python package, so the first commit
isn't necessary. However, maybe we should consider dropping python3-dev
from the default makedepends like the second commit does.
bffe0efc06 newapkbuild: add py3-setuptools to python apkbuild
b7fd57f681 newapkbuild: make python packages only depend on setuptools
This is interesting. As long as the generated tarballs remain as
"stable" as the URLs we've been using, I think this would be good to
include.
a8f564287e newapkbuild: simplify source URL derived from GitHub URL
I'm sure people have Opinions on this. I don't have any so I would
probably opt to skip this.
e125a76489 newapkbuild: do out-of-source CMake builds by default
These seem logical.
d0ce42112f newapkbuild: treat packages that start with py[0-9]- as python packages.
7779e913e7 newapkbuild: allow -n option without url
===== pkgname/pkgver quoting =====
I don't think anyone agrees with this, but it would make onboarding
easier in terms of consistency when writing APKBUILDs. The second commit
reverted it.
49c7560c86 newapkbuild: quote pkgname and pkgver
1888007c38 newapkbuild.in: Allign with CODINGSTYLE quoting
===== Build types =====
The second commit mentions that we might not be passing CPPFLAGS and
LDFLAGS correctly right now for cmake projects. I think these would be
worthwhile to implement but it will cause a lot of breakage.
69fc8b8fc1 newapkbuild: use 'plain' buildtype for meson and 'None'
buildtype for CMake
5cc73d98c3 newapkbuild: do not pass CFLAGS and CXXFLAGS explicitly
===== Reproducible packages =====
This is obviously something we should strive for in the long-term, and
these changes are the bare minimum needed to have any hope of making
reproducible apks. It will need to be adjusted accordingly for
libarchive tar instead of GNU tar, and this may break abuild-gzsplit
which is something I've been talking about with Alpine developers
recently.
ba16a67781 abuild: add SOURCE_DATE_EPOCH support
5b34b696d4 abuild: set SOURCE_DATE_EPOCH to last commit date by default
672032a4be abuild-sign: dont set timestamp in gzip
f04a2ee34b abuild: make built package reproducible
1cfbdf688c abuild: fix git_* functions
114c0cf287 abuild: rename global last_commit to ABUILD_LAST_COMMIT
71d9d5233b abuild: get the git commit date only when needed
51d9e3bcb9 Revert "abuild: make built package reproducible"
918b7b1920 abuild: Fix abuild rootbld
80a2e6f8cf abuild: set SOURCE_DATE_EPOCH from rootpkg
ec96c604e9 abuild: fix building without git
660f793d6d abuild: set fixed atime and ctime in tar
4f5b0a14bd abuild: explicitly sort apk content
===== default_dbg =====
A lot of churn here for what I don't think has material benefit; as far
as I know we have not had any problems with the supposed side effects
mentioned here. It is probably ok to apply though for the sake of QoI.
5b163c2d58 abuild: default_dbg: eliminate side effects
ed88353836 abuild.in: remove duplicate options_has call
acf1fa5553 abuild: default_dbg: do not trigger trap with test failure
cf86b45836 abuild: cleanup default_dbg
===== default_dev =====
The glade change seems ok but is obviously going to break some
building. /usr/lib/pkgconfig vs /usr/share/pkgconfig (and similarly with
glade) seems useless, abuild should only support one and error on the
other for consistency sake.
6e5bd59d6f Add support for parsing pkg-config files in /usr/share/pkgconfig
dbf0e80e62 Make default_dev move to /usr/share/pkgconfig
1bf7e588df move usr/lib/glade/modules and usr/share/glade/catalogs to -dev by default.
===== default_static =====
This is part of a larger policy shift in Alpine to move all static
libraries to separate -static subpackages, which has been somewhat of a
controversial change. However, I think it is something we should pursue
as well because it drastically reduces the complexity of tracking which
packages need to be rebuilt if a static library they depend on has a
security fix or something like that. It also makes it transparent as to
where static libraries are being used anyway, and (not to start a static
vs dynamic linking flamewar) then we can justify why in each case.
There seems to be some insane machinery here to support multiple -dev
subpackages though which seems unnecessary to me. There is also
depends_static which I talk about later.
b849aae4b9 abuild: provide a default_static() and static() functions
30a60d4523 abuild: fix whitespace before EOL
f6bcaee895 Fix condition check for adding static libraries to -dev package.
1dd4382ea1 abuild.in: make is_x_package functions reliant on being given a name
68a054e274 make default_static depend on depends_static
5486d877c5 abuild: -static depends on -dev by default
9be3a6c6a0 abuild: simplify depends_static
===== Default splits for shell completions =====
Seems like a good idea to me; I can't really see anyone objecting to
this.
d5826968b7 abuild.in: add default bashcomp, zshcomp and fishcomp functions
2151c7af80 warn when directories found that hold shell completions.
===== SPDX enforcement =====
Right now I don't believe we ship the SPDX license list anywhere so
`$license` isn't being linted at all. Maybe we should package it, then
pick up these changes.
e476188c6f Better use license.lst in abuild sanitycheck
2162348a9a Allow round brackets in the license variable
===== Removing the gcc dependency =====
This nicely simplifies things. Now CBUILD/CHOST/CARCH et al are resolved
purely from apk --print-arch.
95cd15c025 functions.sh: dont die if gcc is missing
5adf47c168 functions.sh: use apk --print-arch for CARCH if gcc is missing
c9d3df08b3 functions.sh: exclusively use apk --print-arch to detect build arch
===== Bigger package signing keys =====
We should consider implementing this and then changing our official keys
to be bigger the next time we bump adelie-keys.
d5f4982a9a abuild-keygen: make size of private key configurable
===== Validating pc: versions =====
This is something that should be added.
7edafd8c75 abuild: verify that the pkgconf version is valid
9c284d4c33 abuild: convert -{alpha,beta,rc,pre} version suffixes from pkgconf
===== Cosmetic/documentation changes =====
These are all probably worth picking up.
0126fbdffb abuild-sudo: remove unused variable
ff4f2253c1 APKBUILD.5: document depends_* variables for -doc and -openrc
d733d5e570 APKBUILD.5: sort depends_* descriptions
1d854182dd abuild: change word choice in comment
40e6e9995b APKBUILD.5: match install_if example to abuild.in
7a9683a07b abuild usage fix: fetch does not verify sources
beb1b41054 abuild.in: fixup flags and usage text
7a8b3a8e7d Allow for git remote url to end in aports.git, (i.e., allow SSH git remotes)
66177eb5ec abuild: only set sysconfdir in functions.sh
511b934182 abuild: add -V for print abuild version
b09bdddff3 abuild: remove unused print_version function
b3dcf19ef7 make: add `depends` target
918cba2a77 make: install depends as virtual
b80a27b86a abuild-sign: use pigz if available
57185172c5 fix typo in sample for pre and post install
d4c670bc44 Add support for the SUDO variable
77922d5c41 .editorconfig: add Makefile rules
012a179339 use option_has instead of list_has
714be736a7 APKBUILD.5: change 'abuild checkout' to 'abuild snapshot'
3c45858fc0 abuild: fix check_libc
===== Minor bug fixes =====
These seem ok.
f263cb9f49 abuild.in: fix warning with gawk-5.0
2c2a518ac7 change ~ to /home/pdietl
610f1982ef Makefile: clean *.o files
1c075c2855 Fix package version check on checkapk
d8cfcd5dc9 abuild: detect /bin/sh dependency even if shebang has spaces
6d321050e5 Change permissions for sudo to 4555
bad781082d abuild: use stat instead of df to figure filesystem type
6d741ffba8 abuild.in: in snapshot(), fix recursive calls
7d75f85847 abuild-sudo: dont require root to be member of abuild group
===== Patch files =====
I'm ok with this but unless the patch is enormous I think it should be
version controlled by us, not a remote file. This is what we've
generally been doing up to this point anyway.
bb0324e398 abuild: fix applying patches from https
f9e2f96476 abuild: default_prepare: Apply GZIP/XZ-compressed *.patch files
===== amove =====
For those unaware this is a shortcut for automatically creating
directories and moving things into split packages. Personally I don't
like it because it makes APKBUILDs more opaque, but I guess I could come
around on it.
aa86438443 abuild: add amove func to move from $pkgdir to $subpkgdir
ea166888bb Don't pollute stderr with rmdir messages that are ignored
===== abuild-fetch -k =====
Not useful because there is no way to tell abuild from an APKBUILD to
call abuild-fetch with the -k option.
c9d6159637 abuild-fetch: use local insecure variable
59c1c4a97a abuild-fetch: when http:// was used, ignore https:// problems
7bd32679b3 abuild-fetch: add -k (insecure as in curl) option
===== apkbuild-cpan fixes =====
I don't know anything about how CPAN works so someone else will have to
judge these. Currently we do not ship apkbuild-cpan because of how
broken it is.
ede4203749 apkbuild-cpan.in fix issue with version check
422199858c apkbuild-cpan.in fix issues with update, recreate, etc
f98db143cb Many of setings should come from module not release api
c893530293 Fix for issue with apkbuild-cpan upgrade
27c430cc64 Updates for metacpan
122bcd7cb2 apkbuild-cpan: remove prepare function from APKBUILD
67181962bf apkbuild-cpan: include modules that used to be part of core
0238acc51d apkbuild-cpan: use $pkgver in source url
629a780b1c get and use pkgdesc from cpan api data if the module has no metadata files
(returnes unknown)
29bf802907 apkbuild-cpan.in: Update licenses to spdx format and remove redundant directory
change
00beae32aa apkbuild-cpan.in: add OR to licenses to indicate perl_5 GPL or artistic is a
choice
f9da71cc68 apkbuild-cpan.in: add 'configure' prereqs
baf17c8dd7 apkbuild-cpan.in: always start pkgver with a digit
402b582810 apkbuild-cpan.in: add OR to license
3d1c703562 apkbuild-cpan.in: fixes #9990, #9991 and #9992
28bf6f71f0 apkbuild-cpan.in: fix parse_deps for multiple CPAN::Meta::Requirements return
unique do_depends can send multiple meta dependencies as additional parameters parse_deps
was not itterating through each of the parameters this fix gets the entire list without
duplicate entries
0387b8014a apkbuild-cpan.in: improve dealing with old APKBUILD perl module names remove
trailing pkgver from some module names attempt to use additional variable content to
determine perl package name this now appears to support all but 10 or 11 non metacpan api
retrievable module information and add a package mapping for perl-ldap
b60dd09b3b apkbuild-cpan.in: keep provides and replaces on recreate and deal with pkgrel
b9f6741f11 apkbuild-cpan.in: add or between license
b09e522d70 apkbuild-cpan.in: add additional license and package mappings
e448c9a452 apkbuild-cpan.in: Avoid processing duplicate modules in parse_deps
8c62020a34 apkbuild-cpan.in: Fix issue if Build.PL and add perl-module-build as makedeps
aaf56ec102 apkbuild-cpan.in: reset the pkgrel if the pkgver changes on recreate
6670e9479d apkbuild-cpan.in: perl and perl-dev as depends and makedepends fix issue with
how perl-dev was added as a makedepends perl probably should be a dependency of all per
modules
bcbb8979c7 apkbuild-cpan.in: fix missing variables fails and remove empty vars aports
convention is to delete empty variables from APKBUILD files this causes and issue with
updating existing APKBUILD files new depends, makedepends or checkdepends are added to
process the file properly. Empty variables are then deleted once the file processes
correctly
===== apkbuild-pypi =====
This also isn't currently included in our abuild package.
0d4bb95046 gitignore apkbuild-pypi
6310405af8 apkbuild-pypi.in: fix bugs, upgrade for python3 only and add to make
===== ccache support =====
I'm not really interested in supporting this since it's a nightmare for
reproducibility. If someone else wants to apply this go ahead but I
don't think the builders should ever use it of course.
b1b47140ea feat: add support for ccache
3ca7660b66 abuild: only set up ccache in rootbld when USE_CCACHE is set
f9707808ef abuild: various USE_CCACHE fixes
===== D =====
Support for D was added in GCC 10 and Rasmus Thomsen has been building
various stuff in it that we may be interested in (such as a PackageKit
backend for apk) once we also get D support. It's conditional on fixing
up libapk though - expect another few emails about syncing apk-tools in
the future.
This would need to be adjusted on the builders as well.
79cc553d3b abuild: set -g in DFLAGS too for -dbg subpackages
===== Go =====
The first commit here fixes a DRY issue which is worthwhile. The second
one is useless - it addresses a problem with go.mod where it marks files
as read-only for some reason which causes `rm -rf $srcdir` to fail. In
my opinion this is a go.mod problem, not an abuild problem. The third
commit is *another* solution to the same go.mod problem. In my opinion
none of these should be picked up, and go.mod should be dealt with on
its own when we come to it.
30d296ddb3 Make clean() use cleanup()
6a6310f030 Add default_cleanup_srcdir
2e77e3390f abuild-clean: add option to make files writable before cleanup
This is a whole other issue entirely. It would need adjusting for gccgo
if it's something we're interested in.
dcb7ec0c6b abuild.conf: enable PIE binaries by default for go as well
===== Rust =====
I'm not familiar with how we write Rust packages but I presume it's very
different than what Alpine does.
05eee44039 newapkbuild: add rust support
6c3b2e6b58 newapkbuild: run cargo install with --locked
af0c88e6ab set CARGO_HOME to $SRCDEST/cargo
===== Testing abuild =====
We already have our own test suite. I don't know what "bats" is but I
presume it's some external tool, whereas our test suite only needs a
POSIX shell environment and is already more extensive than this one. It
could probably be expanded further by picking up some tests I wrote for
py3-abuild.
43fb2c012e add basic tests using bats
f3dc428ea0 rename datadir -> sharedir
c486b3b4f3 allow override sharedir for testing
===== depends_* =====
Like depends_static there are more depends_* variables (and in the
cosmetic section we saw them documented). I personally think this is a
shortsighted solution to the problem but it is ok for default splits I
guess.
Again with multiple subpackages of the same type, this time with
-openrc which seems slightly more justified but I don't know that it
justifies the added complexity, especially down the road.
801578a06a abuild: -openrc should not inherit depends
f92353f57b abuild: add depends_libs variable to default_libs()
1de902f2fa abuild: fix -openrc to work with multiple subpackages
===== armhf triplet =====
Obviously all of our triplets are different from Alpine in at least the
vendor field. I'm not sure if we need to drop the "gnu" from our armhf
triplet or not, but it is still there. Not a big issue since armhf isn't
even on any of our tiers yet, but could become an issue if we decide to
pick it up since Alpine is dropping it or has already dropped it (I
don't remember).
915c7d868b functions: adjust armhf triplet
===== Removing (some) empty directories =====
I don't know what the rationale is here. I think it is related to an
issue they discussed on IRC recently where a package was breaking
permissions on one of these directories and it was breaking people's
installations as a result. Would be caught by more careful checks of
permissions when changing or bumping packages.
b8b8a651fc abuild: remove empty dirs in main package
cb3a67edca add missing backslash on rmdir command
===== Removing recursive build mode =====
Obviously this cannot be applied anytime soon, though whether it ever
should be applied is a good question (but I'm clearly biased on the
matter).
9a398eac0c abuild: remove recursive mode
===== Cross-repository dependencies =====
Complicated and weird way of solving the problem in my opinion, but
again I'm biased.
baa554477e abuild: install dependencies from other repos
===== Reverted changes =====
Self-explanatory.
8fbbffd201 abuild: unset depends for subpackages
c0dc7acee7 Revert "abuild: unset depends for subpackages"
d0828c0655 abuild: include $pkgname in temporary rootbld directory
19678c5434 Revert "abuild: include $pkgname in temporary rootbld directory"
===== Release commits =====
All of these only change the version number in the Makefile, so they are
obviously not useful to us.
e2a012cba6 ==== release 3.4.0_rc1 ====
dc41c1fc52 ==== release 3.4.0_rc2 ====
0a7026a485 ==== release 3.4.0_rc3 ====
8d092443d9 ==== release 3.4.0_rc4 ====
c26cc11905 ==== release 3.4.0_rc5 ====
0b3f983772 ==== release 3.4.0 ====
21dcfb1f21 ==== release 3.5.0_rc1 ====
0745179c4f ==== release 3.5.0_rc2 ====
478d4e9452 ==== release 3.5.0_rc3 ====
0d3dc71833 ==== release 3.5.0 ====
3ea72b0a9e ==== release 3.6.0_rc1 ====
c53d8ad12a ==== release 3.6.0 ====