This repository has been archived by the owner on Apr 18, 2023. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
travis: check spelling of binaries and manpages, use docker
We now build a docker base container based on debian sid (where the very latest packages are available). That base container is updated once a month, or whenever travis-build.Dockerfile or debian/control change, but re-used for subsequent travis runs. While the initial build might take up to 15 minutes, subsequent builds typically run in a minute or two. All the different steps that we run on travis are now factored into separate scripts in the travis/ directory. Switching to docker should also help with issue #2174.
- Loading branch information
1 parent
065ce6b
commit fbfbdb8
Showing
16 changed files
with
164 additions
and
50 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,49 +1,23 @@ | ||
sudo: required | ||
dist: trusty | ||
services: | ||
- docker | ||
language: c | ||
compiler: | ||
- gcc | ||
- clang | ||
addons: | ||
apt: | ||
sources: | ||
- ubuntu-toolchain-r-test | ||
packages: | ||
- clang-format-3.5 | ||
- libllvm3.5 | ||
- clang-3.5 | ||
- gcc-5 | ||
before_install: | ||
- sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-suggests --no-install-recommends devscripts equivs xdotool | ||
env: | ||
global: | ||
- BASENAME="i3wm/travis-base:$(date +'%Y-%m')-$(./travis/ha.sh)" | ||
- secure: "B5IICA8MPx/FKaB50rTPqL8P1NU+Q0yuWl+lElL4+a9xSyLikfm3NzUPHoVwx8lNw2AVK6br7p0OmF7vMFjqAgrgc1cajTtEae5uFRKNUrWLpXM046YgNEYLLIHsQOjInxE+S4O6EFVzsUqsu8aeo2Xhq4sm4iUocG7e5isYgYo=" # DOCKER_PASS | ||
- secure: "EIvrq8PG7lRjidppG0RCv4F0X4GP3DT9F5+ixVuGPfhK/hZT3jYC2AVY9G+NnUcXVwQEpW92rlqpftQ/qZ13FoyWokC8ZyoyD06fr5FPCfoFF3OczZwAJzZYkObI/hE9+/hXcylx/Os6N4INd2My1ntGk3JPsWL9riopod5EjSg=" # DOCKER_EMAIL | ||
- secure: "hvhBunS4xXTgnIOsk/BPT7I7FrJhvVwCSt5PfxxvMqNaztOJI9BuK7ZrZ5Cy38KyHwlh3VHAH5AaCygJcPauoSQCV3bpnlbaWn3ruq2F0Q697Q5uNf73liXzyUqb9/Zvfvge4y4WWOhP5tVz1C6ZBe/NfhU7pqKLMA+6ads+99c=" # DOCKER_USER | ||
install: | ||
- sudo DEBIAN_FRONTEND=noninteractive mk-build-deps --install --remove --tool 'apt-get -yq --no-install-suggests --no-install-recommends' debian/control | ||
# Install as many dependencies as possible via apt because cpanm is not very reliable/easy to debug. | ||
- sudo DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-suggests --no-install-recommends libanyevent-perl libanyevent-i3-perl libextutils-pkgconfig-perl xcb-proto cpanminus xvfb xserver-xephyr xauth libinline-perl libxml-simple-perl libmouse-perl libmousex-nativetraits-perl libextutils-depends-perl perl-modules libtest-deep-perl libtest-exception-perl libxml-parser-perl libtest-simple-perl libtest-fatal-perl libdata-dump-perl libtest-differences-perl libxml-tokeparser-perl libtest-use-ok-perl libipc-run-perl libxcb-xtest0-dev | ||
- sudo /bin/sh -c 'cpanm -n -v X11::XCB || true' | ||
- sudo /bin/sh -c 'cpanm -n -v AnyEvent::I3 || true' | ||
script: | ||
- if [ -a .git/shallow ]; then git fetch --unshallow; fi | ||
- if [ "$CC" = "clang" ]; then export CC="clang-3.5"; fi | ||
- if [ "$CC" = "gcc" ]; then export CC="gcc-5"; fi | ||
- CFLAGS="-Wformat -Wformat-security -Wextra -Wno-unused-parameter -Werror" make -j ASAN=1 | ||
- (cd testcases && xvfb-run ./complete-run.pl --parallel=1 || (cat latest/complete-run.log; false)) | ||
- clang-format-3.5 -i $(find . -name "*.[ch]" | tr '\n' ' ') && git diff --exit-code || (echo 'Code was not formatted using clang-format!'; false) | ||
- | | ||
funcs='malloc|calloc|realloc|strdup|strndup|asprintf|write' | ||
cstring='"([^"\\]|\\.)*"' | ||
cchar="'[^\\\\]'|'\\\\.[^']*'" | ||
regex="^([^'\"]|${cstring}|${cchar})*\<(${funcs})\>" | ||
detected=0 | ||
while IFS= read -r file; do | ||
if { cpp -w -fpreprocessed "$file" || exit "$?"; } | grep -E -- "$regex"; then | ||
echo "^ $file calls a function that has a safe counterpart." | ||
detected=1 | ||
fi | ||
done << EOF | ||
$(find -name '*.c' -not -name safewrappers.c -not -name strndup.c) | ||
EOF | ||
if [ "$detected" -ne 0 ]; then | ||
echo | ||
echo "Calls of functions that have safe counterparts were detected." | ||
exit 1 | ||
fi | ||
- docker pull ${BASENAME} || ./travis/docker-build-and-push.sh | ||
script: | ||
- docker run -v $PWD:/usr/src/i3/ -w /usr/src/i3 ${BASENAME} ./travis/check-safe-wrappers.sh | ||
- docker run -v $PWD:/usr/src/i3/ -w /usr/src/i3 ${BASENAME} ./travis/check-formatting.sh | ||
- docker run -v $PWD:/usr/src/i3/ -w /usr/src/i3 -e CC -e CFLAGS="-Wformat -Wformat-security -Wextra -Wno-unused-parameter -Werror" ${BASENAME} make all mans -j ASAN=1 | ||
- docker run -v $PWD:/usr/src/i3/ -w /usr/src/i3 ${BASENAME} ./travis/check-spelling.pl | ||
- docker run -v $PWD:/usr/src/i3/ -w /usr/src/i3 ${BASENAME} ./travis/run-tests.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# vim:ft=Dockerfile | ||
FROM debian:sid | ||
|
||
RUN echo force-unsafe-io > /etc/dpkg/dpkg.cfg.d/docker-apt-speedup | ||
# Paper over occasional network flakiness of some mirrors. | ||
RUN echo 'APT::Acquire::Retries "5";' > /etc/apt/apt.conf.d/80retry | ||
|
||
# NOTE: I tried exclusively using gce_debian_mirror.storage.googleapis.com | ||
# instead of httpredir.debian.org, but the results (Fetched 123 MB in 36s (3357 | ||
# kB/s)) are not any better than httpredir.debian.org (Fetched 123 MB in 34s | ||
# (3608 kB/s)). Hence, let’s stick with httpredir.debian.org (default) for now. | ||
|
||
# Install mk-build-deps (for installing the i3 build dependencies), | ||
# clang and clang-format-3.5 (for checking formatting and building with clang), | ||
# lintian (for checking spelling errors), | ||
# test suite dependencies (for running tests) | ||
RUN apt-get update && \ | ||
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ | ||
dpkg-dev devscripts git equivs \ | ||
clang clang-format-3.5 \ | ||
lintian \ | ||
libanyevent-perl libanyevent-i3-perl libextutils-pkgconfig-perl xcb-proto cpanminus xvfb xserver-xephyr xauth libinline-perl libinline-c-perl libxml-simple-perl libmouse-perl libmousex-nativetraits-perl libextutils-depends-perl perl-modules libtest-deep-perl libtest-exception-perl libxml-parser-perl libtest-simple-perl libtest-fatal-perl libdata-dump-perl libtest-differences-perl libxml-tokeparser-perl libipc-run-perl libxcb-xtest0-dev libx11-xcb-perl libanyevent-i3-perl && \ | ||
rm -rf /var/lib/apt/lists/* | ||
|
||
# Install i3 build dependencies. | ||
COPY debian/control /usr/src/i3-debian-packaging/control | ||
RUN apt-get update && \ | ||
DEBIAN_FRONTEND=noninteractive mk-build-deps --install --remove --tool 'apt-get --no-install-recommends -y' /usr/src/i3-debian-packaging/control && \ | ||
rm -rf /var/lib/apt/lists/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
#!/bin/sh | ||
clang-format-3.5 -i $(find . -name "*.[ch]" | tr '\n' ' ') && git diff --exit-code || (echo 'Code was not formatted using clang-format!'; false) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
#!/bin/sh | ||
funcs='malloc|calloc|realloc|strdup|strndup|asprintf|write' | ||
cstring='"([^"\\]|\\.)*"' | ||
cchar="'[^\\\\]'|'\\\\.[^']*'" | ||
regex="^([^'\"]|${cstring}|${cchar})*\<(${funcs})\>" | ||
detected=0 | ||
while IFS= read -r file; do | ||
if { cpp -w -fpreprocessed "$file" || exit "$?"; } | grep -E -- "$regex"; then | ||
echo "^ $file calls a function that has a safe counterpart." | ||
detected=1 | ||
fi | ||
done << EOF | ||
$(find -name '*.c' -not -name safewrappers.c -not -name strndup.c) | ||
EOF | ||
if [ "$detected" -ne 0 ]; then | ||
echo | ||
echo "Calls of functions that have safe counterparts were detected." | ||
exit 1 | ||
fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
#!/usr/bin/env perl | ||
# vim:ts=4:sw=4:expandtab | ||
# | ||
# © 2016 Michael Stapelberg | ||
# | ||
# Checks for spelling errors in binaries and manpages (to be run by continuous | ||
# integration to point out spelling errors before accepting contributions). | ||
|
||
use strict; | ||
use warnings; | ||
use v5.10; | ||
use autodie; | ||
use lib 'testcases/lib'; | ||
use i3test::Util qw(slurp); | ||
use Lintian::Check qw(check_spelling); | ||
|
||
# Lintian complains if we don’t set a vendor. | ||
use Lintian::Data; | ||
use Lintian::Profile; | ||
Lintian::Data->set_vendor( | ||
Lintian::Profile->new('debian', ['/usr/share/lintian'], {})); | ||
|
||
my $exitcode = 0; | ||
|
||
# Whitelist for spelling errors in manpages, in case the spell checker has | ||
# false-positives. | ||
my $binary_spelling_exceptions = { | ||
#'exmaple' => 1, # Example for how to add entries to this whitelist. | ||
'betwen' => 1, # asan_flags.inc contains this spelling error. | ||
}; | ||
my @binaries = qw( | ||
i3 | ||
i3-config-wizard/i3-config-wizard | ||
i3-dump-log/i3-dump-log | ||
i3-input/i3-input | ||
i3-msg/i3-msg | ||
i3-nagbar/i3-nagbar | ||
i3bar/i3bar | ||
); | ||
for my $binary (@binaries) { | ||
check_spelling(slurp($binary), $binary_spelling_exceptions, sub { | ||
my ($current, $fixed) = @_; | ||
say STDERR qq|Binary "$binary" contains a spelling error: "$current" should be "$fixed"|; | ||
$exitcode = 1; | ||
}); | ||
} | ||
|
||
# Whitelist for spelling errors in manpages, in case the spell checker has | ||
# false-positives. | ||
my $manpage_spelling_exceptions = { | ||
}; | ||
|
||
for my $name (glob('man/*.1')) { | ||
for my $line (split(/\n/, slurp($name))) { | ||
next if $line =~ /^\.\\\"/o; | ||
check_spelling($line, $manpage_spelling_exceptions, sub { | ||
my ($current, $fixed) = @_; | ||
say STDERR qq|Manpage "$name" contains a spelling error: "$current" should be "$fixed"|; | ||
$exitcode = 1; | ||
}); | ||
} | ||
} | ||
|
||
exit $exitcode; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#!/bin/sh | ||
|
||
set -e | ||
|
||
# .dockerignore is created on demand so that release.sh and other scripts are | ||
# not influenced by our travis setup. | ||
echo .git > .dockerignore | ||
|
||
docker build --pull --no-cache --rm -t=${BASENAME} -f travis-build.Dockerfile . | ||
docker login -e ${DOCKER_EMAIL} -u ${DOCKER_USER} -p ${DOCKER_PASS} | ||
docker push ${BASENAME} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
#!/bin/sh | ||
# Returns a hash to be used as version number suffix for the i3/travis-base | ||
# docker container. The hash is over all files which influence what gets | ||
# installed in the container, so that any changes in what needs to be installed | ||
# will result in a cache invalidation. | ||
|
||
cat debian/control travis-build.Dockerfile | sha256sum | dd bs=1 count=8 status=none |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
#!/bin/sh | ||
cd testcases | ||
# Try running the tests in parallel so that the common case (tests pass) is | ||
# quick, but fall back to running them in sequence to make debugging easier. | ||
if ! xvfb-run ./complete-run.pl | ||
then | ||
xvfb-run ./complete-run.pl --parallel=1 || (cat latest/complete-run.log; false) | ||
fi |