mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
synced 2024-12-01 16:14:20 +08:00
9870d504d8
To use optimized CRC implementation, the input buffer must be unsigned long aligned. btrfs receive calculates checksum based on read_buf, including btrfs_cmd_header (with zeroed CRC field) and command content. Reorder the buffer to the beginning of the structure and force the alignment to 64, this should be cacheline friendly and could speed up the data transfers. Interesting parts from the report: Sending host: Fedora 33 AMD ThreadRipper 1920X - 128GB RAM 2x10GBit Ethernet, bonded MegaRaid 9270 6x16TB Seagate Exos in RAID5 Receiving host: Fedora 33 Intel i3-7300 - HT enabled - 32GB RAM 10GBit Ethernet, single connection MegaRaid 9260 12x8TB WD NAS drives in RAID5 The 2 hosts are connected to the same 10G switch. The sender could definitely saturate a 10GBit link. The practically achievable writes on the backup host would be lower, but still at least 400MB/s. The file system contains mostly large files of 1GB+, so there is little meta-data. With btrfs send/receive I'm getting a steady transfer rate of 60MB/s. The copy has been running for a little over 5 days now, having only transferred some 25TB. This is way too slow for this setup. Analyzing resource usage, the sender side is fine, both the btrfs send and the corresponding ssh process only use about 10-10% CPU, which on a 24 threaded machine is virtually nothing. However, the receiver is running with a load of ~2.6, with the sshd using 30-50% CPU and the btrfs receive a further 60-70%. The rest of the load comes from IO wait. So the bottleneck is the btrfs receive clearly. Issue: #324 Signed-off-by: Sheng Mao <shngmao@gmail.com> Signed-off-by: David Sterba <dsterba@suse.com> |
||
|---|---|---|
| .. | ||
| box.h | ||
| defs.h | ||
| device-scan.c | ||
| device-scan.h | ||
| device-utils.c | ||
| device-utils.h | ||
| extent-cache.c | ||
| extent-cache.h | ||
| format-output.c | ||
| format-output.h | ||
| fsfeatures.c | ||
| fsfeatures.h | ||
| help.c | ||
| help.h | ||
| internal.h | ||
| messages.c | ||
| messages.h | ||
| path-utils.c | ||
| path-utils.h | ||
| rbtree-utils.c | ||
| rbtree-utils.h | ||
| send-stream.c | ||
| send-stream.h | ||
| send-utils.c | ||
| send-utils.h | ||
| string-table.c | ||
| string-table.h | ||
| task-utils.c | ||
| task-utils.h | ||
| utils-lib.c | ||
| utils.c | ||
| utils.h | ||