1
0
mirror of https://github.com/Zygo/bees.git synced 2025-05-17 21:35:45 +02:00

241 Commits

Author SHA1 Message Date
Zygo Blaxell
68d48b3d63 Merge branch 'master' into subvol-threads 2018-01-11 21:26:34 -05:00
Kai Krakow
6d6aedd8ec Makefile: Fail gracefully if markdown is not installed
Previously, MARKDOWN may end up empty. This commit should fix it.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 21:25:12 +01:00
Kai Krakow
025b14f38f Installation: Depend Gentoo ebuild on markdown
Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 20:56:48 +01:00
Kai Krakow
dd6d8caaa2 Installation: Remove superfluous cruft from Gentoo ebuild
Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 20:56:34 +01:00
Zygo Blaxell
58e53d7e0c Merge branch 'master' into subvol-threads 2018-01-10 23:43:09 -05:00
Zygo Blaxell
4bfb637b0e Merge remote-tracking branch 'nefelim4ag/master' 2018-01-10 23:43:00 -05:00
Zygo Blaxell
addb18354e hash: fix FTBFS in "hash: reduce mutex contention using one mutex per hash table extent"
Somehow a delete failed to get merged.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-01-10 23:42:20 -05:00
Zygo Blaxell
54d30485a7 subvol-threads: increase resource and thread limits
With kernel 4.14 there is no sign of the previous LOGICAL_INO performance
problems, so there seems to be no need to throttle threads using this
ioctl.

Increase the FD cache size limits and scan thread count.  Let the kernel
figure out scheduling.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-01-10 23:27:04 -05:00
Zygo Blaxell
93fb29a461 Merge branch 'master' into subvol-threads 2018-01-10 23:26:59 -05:00
Zygo Blaxell
4aa5978a89 hash: reduce mutex contention using one mutex per hash table extent
This avoids PERFORMANCE warnings when large hash tables are used on slow
CPUs or with lots of worker threads.  It also simplifies the code (no
locksets, only one object-wide mutex instead of two).

Fixed a few minor bugs along the way (e.g. we were not setting the dirty
flag on the right hash table extent when we detected hash table errors).

Simplified error handling:  IO errors on the hash table are ignored,
instead of throwing an exception into the function that tried to use the
hash table.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-01-10 23:25:45 -05:00
Kai Krakow
365a913a26 Installation: Add Gentoo ebuild
This commit adds an ebuild for Gentoo. Version 9999 is building live
from current git, currently using kakra:integration because it has some
installation and build fixes important for Gentoo.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 03:03:17 +01:00
Kai Krakow
634a1d0bf6 Installation: -fPIC should not be used unconditionally
According to Gentoo packaging guide, -fPIC should only be used on shared
libraries, and not added unconditionally to every linker call.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 02:30:12 +01:00
Kai Krakow
3a24cd3010 Installation: Fix soname QA warning in Gentoo
Gentoo warns about libs missing a proper soname during QA phase. Let's
fix this.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 02:30:12 +01:00
Kai Krakow
3391593cb9 Installation: Keep version tag in a variable
To prepare soname handling, we need to keep the version tag in a
variable.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 02:02:36 +01:00
Kai Krakow
fdd8350239 Installation: Improve filesystem layout flexibility
In preparation for Gentoo QA checks during ebuild merge phase, let's
make some more of the filesystem layout adjustable.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-11 02:02:36 +01:00
Kai Krakow
60cd9c6165 Installation: Introduce DESTDIR into Makefile
In Gentoo, usage of DESTDIR is automatically handled by the build system
to support installation into a clean image from which the package is
created.

Thus, let's add DESTDIR to the install targets. One can now correctly
install bees with packaging systems simply by running:

$ DESTDIR=/tmp/bees-image make all install

This will no longer mess up with the PREFIX setting.

CC: Timofey Titovets <nefelim4ag@gmail.com>
Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-10 22:35:22 +01:00
Kai Krakow
f0e02478ef Installation: Document optional dependency on blkid
If using `scripts/beesd`, we need `blkid` which is part of util-linux.
It should be available on every distribution but let's document it
anyway.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-10 21:47:01 +01:00
Kai Krakow
421641e242 Makefile: Document scripts/beesd
Add a paragraph about the helper script `scripts/beesd` to automatically
setup and configure bees.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-10 21:06:49 +01:00
Kai Krakow
0fce10991b
Installation: Add Arch Linux instructions
Closes #34

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-10 20:43:54 +01:00
Kai Krakow
a465d997bd
Makefile: Document Makefile changes 2018-01-10 20:41:56 +01:00
Kai Krakow
361ef0bebf
Installation: Add new section to README 2018-01-10 20:41:37 +01:00
Kai Krakow
1fcf07cc2a
Installation: Prepare README
Rename a section in preparation for a new install section.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-10 20:41:00 +01:00
Kai Krakow
333b2f7746 Makefile improvement
Now you can make bees fly as pointed out in the README... ;-)
2018-01-10 20:09:38 +01:00
Timofey Titovets
ff9e0e3571 Fix: exec bees - breaks bash trap handling of umount bees workdir
Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2018-01-09 23:25:57 +01:00
Timofey Titovets
2d49d98bd2 Fix: exec bees - breaks bash trap handling of umount bees workdir
Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2018-01-09 22:33:57 +03:00
Kai Krakow
92aa13a6ae Add beesd@.service to gitignore
It's a generated file. We should ignore it, so it won't be accidently be
checked in.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 01:56:22 +01:00
Kai Krakow
f0c516f33b Makefile: let "make install" install the complete distribution
It happened more than once that I ran just "make install" only, which
doesn't install the scripts.

Let's fix this by renaming the previous install target to install_bees,
and then make a new install target which depends on each install target
and thus installs the complete distribution.

It doesn't hurt to install those few scripts. I don't see the point in
separating the install targets as it was previously done.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 01:54:41 +01:00
Kai Krakow
8e2139d6ed Makefile: depend install_scripts on scripts
For consistency with the other install target, let's depend
install_scripts on its build targets.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 01:51:18 +01:00
Kai Krakow
b959af1a15 systemd: Provide URL and better description
Let's direct users to the support site when they ask systemd for help
about the service unit, or by looking at error messages.

Also, let's adjust the description to be more pleasing to the eyes. The
previous long description with uncommon formatting really stuck out in
the boot logs.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 01:32:17 +01:00
Kai Krakow
78f96a9fbd systemd: Don't start without essential system services
Starting bees right after local-fs.target is probably not what we want,
as basic setup of the system might not have been done (like udev,
cryptsetup, sysctl, swap, etc).

Let's start only after sysinit.target instead which guarantees that all
basic setup has been done, most importantly, sysctl, udev, and swap have
been setup which may apply important tweaks, configuration, and tuning.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 01:29:05 +01:00
Kai Krakow
953c158868 systemd: Don't start in system-update.target
Due to bees installing into the local-fs.target, bees also runs during
system-update.target. This should not be done, system-update.target is
meant as an isolated bootup mode for applying updates offline, that is:
Only essential services are running.

Fix this by making it WantedBy basic.target instead. According to
system-update.target and "man bootup", system-update.target pulls in
sysinit.target, as does basic.target. So essentially, basic.target is
not part of the system-update.target transaction.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 01:24:35 +01:00
Kai Krakow
f7f99f52b5 Generalize sed invocation rule
Remove the redundant sed call by generalizing the rule to apply sed to
.in templates.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 00:48:14 +01:00
Kai Krakow
abeb6e74b2 Add scripts to "make all" target
This prevents scripts being generated by "root" during "sudo make
install" phase.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 00:48:14 +01:00
Kai Krakow
6c67ae0d5e Don't zap localconf in "make clean"
When you run "make clean", localconf is being removed. This is probably
in most cases not intentional.

Signed-off-by: Kai Krakow <kai@kaishome.de>
2018-01-09 00:48:14 +01:00
Zygo Blaxell
c39b72b4a7 Merge branch 'master' into subvol-threads 2018-01-07 21:39:54 -05:00
Zygo Blaxell
ba981c133a Merge remote-tracking branches 'kakra/feature/add-relative-path-option' and 'kakra/integration' 2018-01-07 21:39:01 -05:00
Zygo Blaxell
53f8fc506a crucible: fixup cleanup header
Make the #include guard name match the file name.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-01-07 21:07:35 -05:00
Zygo Blaxell
2a2632a101 Merge branch 'master' into subvol-threads 2018-01-07 21:06:09 -05:00
Zygo Blaxell
9d295fab4e Makefile: if multiple Markdown utilities are present, use the first one
If two utilities are found, we get commands like

	/usr/bin/markdown /usr/bin/markdown_py README.md > README.html

and that doesn't work.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
v0.5
2018-01-07 20:23:20 -05:00
Zygo Blaxell
dc7360397e README: update the state of bees and the kernel for v4.14
Read-only snapshots have always just worked.  Remove them from the
"untested" list.

nodatasum (and therefore nodatacow) inodes are simply ignored.  This seems
like the right thing to do since deduping a nodatacow extent turns it
into a datacow extent, which seems contrary to administrator wishes
implied by the nodatacow bit.  We probably need an option to
override that assumption.

Clarify why converted ext[234] filesystems may cause problems and
the nature of those problems.

Assorted minor editorial changes.

Discuss calculation of the balance limit parameter when ensuring
sufficient metadata space.

Update kernel version bug/fix/feature lists, including LOGICAL_INO_V2.

Annotate kernel workaround list with known kernel versions that make
the workarounds necessary.

Remove reference to 'DEFRAG_RANGE' as bees requires much more control
over data placement than this interface can offer.  It's easy enough to
create a new ioctl to implement bees requirements once it's known what
those requirements are.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2018-01-07 20:23:20 -05:00
Zygo Blaxell
28f7299f8f Merge branch 'master' into subvol-threads 2018-01-06 22:56:38 -05:00
Zygo Blaxell
305ab5dbfa Merge remote-tracking branch 'nefelim4ag/master' 2018-01-06 22:54:49 -05:00
Timofey Titovets
80e4302958 Update btrfs compression types, add ZSTD, drop LAST
Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2018-01-04 20:32:04 +03:00
Zygo Blaxell
02181956d2 Merge branch 'master' into subvol-threads 2017-12-21 14:01:51 -05:00
Zygo Blaxell
07751885d2 error: drop redundant CHECK_CONSTRAINT
CHECK_CONSTRAINT is just THROW_CHECK1 with an inconsistent name.
Remove it.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 14:00:17 -05:00
Zygo Blaxell
77614a0e99 scan: insert toxic matched extents into hash table as they are discovered
When a toxic extent is discovered, insert the offending hash/address/toxic
entry into the hash table.

When a previously discovered toxic extent is encountered, do nothing,
i.e. allow the offending hash/address/toxic entry in the hash table
to expire.

Previously both inserts were removed from the code, but the former one
is required.  The latter prevents bees from forgiving toxic extents
(or any hash matching one) should they be relocated, deleted, or simply
become non-toxic.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 13:56:15 -05:00
Zygo Blaxell
9b0e8c56c2 Makefile: fix duplicate merge artifact
Keep the first duplicate MARKDOWN stanza.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 13:53:55 -05:00
Zygo Blaxell
649ae5bb40 makeflags: fix missing -D_FILE_OFFSET_BITS=64 in comment
Interesting things happen when blindly swapping the release-build CCFLAGS
with the debug-build commented-out CCFLAGS.  None of these things that
happen are good.

Signed-off-by: Zygo Blaxell <bees@furryterror.org>
2017-12-21 13:28:49 -05:00
Timofey Titovets
40112faf0f Makefile add scripts target for correctly packaging
Currently scheme lead to path like:
/tmp/makepkg/bees-git/pkg/bees-git/usr/lib/bees/bees

While packaging, so allow do:
make
make scripts
make install ...
make install_scripts ...

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
2017-12-21 13:26:45 +03:00
Kai Krakow
b978a5dea6 Fix a fallthrough error in GCC 7+
GCC 7 and higher turn a previous warning into an error for implicit
fallthrough. Let's hint the compiler that this is intentional here.

Signed-off-by: Kai Krakow <kai@kaishome.de>
(cherry picked from commit 270a91cf17783813edaa75fb9766858dcf8e1689)
2017-11-14 11:27:27 -05:00