-
Notifications
You must be signed in to change notification settings - Fork 8
/
sparse-block.diff
103 lines (91 loc) · 4.22 KB
/
sparse-block.diff
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
This patch adds the --sparse-block option. Andrea Righi writes:
In some filesystems, typically optimized for large I/O throughputs (like
IBM GPFS, IBM SAN FS, or distributed filesystems in general) a lot of
lseek() operations can strongly impact on performances. In this cases it
can be helpful to enlarge the block size used to handle sparse files
directly from a command line parameter.
For example, using a sparse write size of 32KB, I've been able to
increase the transfer rate of an order of magnitude copying the output
files of scientific applications from GPFS to GPFS or GPFS to SAN FS.
-Andrea
To use this patch, run these commands for a successful build:
patch -p1 <patches/sparse-block.diff
./configure (optional if already run)
make
based-on: 6c8ca91c731b7bf2b081694bda85b7dadc2b7aff
diff --git a/fileio.c b/fileio.c
--- a/fileio.c
+++ b/fileio.c
@@ -34,6 +34,7 @@
#define ALIGNED_LENGTH(len) ((((len) - 1) | (ALIGN_BOUNDARY-1)) + 1)
extern int sparse_files;
+extern int sparse_files_block_size;
OFF_T preallocated_len = 0;
@@ -153,7 +154,7 @@ int write_file(int f, int use_seek, OFF_T offset, const char *buf, int len)
while (len > 0) {
int r1;
if (sparse_files > 0) {
- int len1 = MIN(len, SPARSE_WRITE_SIZE);
+ int len1 = MIN(len, sparse_files_block_size ? sparse_files_block_size : SPARSE_WRITE_SIZE);
r1 = write_sparse(f, use_seek, offset, buf, len1);
offset += r1;
} else {
diff --git a/options.c b/options.c
--- a/options.c
+++ b/options.c
@@ -83,6 +83,7 @@ int remove_source_files = 0;
int one_file_system = 0;
int protocol_version = PROTOCOL_VERSION;
int sparse_files = 0;
+long sparse_files_block_size = 0;
int preallocate_files = 0;
int do_compression = 0;
int do_compression_level = CLVL_NOT_SPECIFIED;
@@ -702,6 +703,7 @@ static struct poptOption long_options[] = {
{"sparse", 'S', POPT_ARG_VAL, &sparse_files, 1, 0, 0 },
{"no-sparse", 0, POPT_ARG_VAL, &sparse_files, 0, 0, 0 },
{"no-S", 0, POPT_ARG_VAL, &sparse_files, 0, 0, 0 },
+ {"sparse-block", 0, POPT_ARG_LONG, &sparse_files_block_size, 0, 0, 0 },
{"preallocate", 0, POPT_ARG_NONE, &preallocate_files, 0, 0, 0},
{"inplace", 0, POPT_ARG_VAL, &inplace, 1, 0, 0 },
{"no-inplace", 0, POPT_ARG_VAL, &inplace, 0, 0, 0 },
@@ -2772,6 +2774,12 @@ void server_options(char **args, int *argc_p)
args[ac++] = arg;
}
+ if (sparse_files_block_size) {
+ if (asprintf(&arg, "--sparse-block=%lu", sparse_files_block_size) < 0)
+ goto oom;
+ args[ac++] = arg;
+ }
+
if (io_timeout) {
if (asprintf(&arg, "--timeout=%d", io_timeout) < 0)
goto oom;
diff --git a/rsync.1.md b/rsync.1.md
--- a/rsync.1.md
+++ b/rsync.1.md
@@ -466,6 +466,7 @@ has its own detailed description later in this manpage.
--super receiver attempts super-user activities
--fake-super store/recover privileged attrs using xattrs
--sparse, -S turn sequences of nulls into sparse blocks
+--sparse-block=SIZE set block size used to handle sparse files
--preallocate allocate dest files before writing them
--dry-run, -n perform a trial run with no changes made
--whole-file, -W copy files whole (w/o delta-xfer algorithm)
@@ -1730,6 +1731,18 @@ expand it.
(as opposed to allocated sequences of null bytes) if the kernel version and
filesystem type support creating holes in the allocated data.
+0. `--sparse-block=SIZE`
+
+ Change the block size used to handle sparse files to SIZE bytes. This
+ option only has an effect if the [`--sparse`](#opt) (`-S`) option was also
+ specified. The default block size used by rsync to detect a file hole is
+ 1024 bytes; when the receiver writes data to the destination file and
+ option [`--sparse`](#opt) is used, rsync checks every 1024-bytes chunk to
+ detect if they are actually filled with data or not. With certain
+ filesystems, optimized to receive data streams for example, enlarging this
+ block size can strongly increase performance. The option can be used to
+ tune this block size.
+
0. `--dry-run`, `-n`
This makes rsync perform a trial run that doesn't make any changes (and