-
Notifications
You must be signed in to change notification settings - Fork 2
/
HISTORY
464 lines (421 loc) · 22 KB
/
HISTORY
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
Version 3.01 2011-07-31
* bug fixed: tracker_get_connection_ex and tracker_get_connection_r_ex
connect two times with multi tracker servers
* bug fixed: tracker_mem_check_add_tracker_servers condition not correct
* all logError add source filename and line
* php extension support upload file callback
* php extension support download file callback
Version 3.00 2011-06-19
* mass small files optimization
* add fixed block memory pool: common/fast_mblock.c
* bug fixed: tracker_mem.c do NOT clear g_groups fields
* bug fixed: slave file and appender file download ok
* bug fixed: tracker / storage run by group / user, set file owner
* tracker server support leader
* client support static library
* client_func.h add functions fdfs_tracker_group_equals and
fdfs_get_file_ext_name
* bug fixed: test/dfs_func_pc.c compile ok
* storage server check free space enough when upload a file
Version 2.09 2011-02-19
* bug fixed: write_to_binlog_index then increase g_binlog_index (feedback
by koolcoy)
* disk read / write supports direct mode (avoid caching by the file system)
Version 2.08 2011-01-30
* bug fixed: fdfs_trackerd.c set g_tracker_thread_count to 0
* add cmd TRACKER_PROTO_CMD_SERVER_LIST_ONE_GROUP to support list one group
* support disk recovery automatically
* support total_upload_bytes, success_upload_bytes, total_download_bytes and
success_download_bytes etc. 18 stat fields
* tracker data file storage_groups.dat changes to storage_groups_new.dat, and
storage_servers.dat changes to storage_servers_new.dat
* support file append, add tests: fdfs_appender_test and fdfs_appender_test1
* storage_dio.c: dio_deal_task split to several functions
* tracker http check thread exit normally
* function fdfs_get_file_info_ex changed, add function fdfs_get_file_info_ex1
* fix some type cast error when compile with c++
* client add tools: fdfs_upload_appender and fdfs_append_file
Version 2.07 2011-01-09
* slave file's prefix name can be empty
* FDFS_MAX_GROUPS change from 64 to 512
* file size field in the file id changed: high 32 bits is random integer
when the file size < 2GB and the highest bit set to 1
* tracker_service.c: in function list_group_storages, use strcpy
intead of memcpy
* php extension add function fastdfs_tracker_delete_storage
* client add tool: fdfs_file_info to get file info, including file size,
create timestamp, source storage ip address and crc32 signature
* fdfs_upload_file.c: omit more error info when the local file not exist
Version 2.06 2010-12-26
* sync file op: do not sync the file which exists on dest storage server
and the file size are same
* bug fixed: sync copy file will clear the existed file on dest storage
server (truncate the file size to 0), this bug caused by V2.04
* bug fixed: make temp file discard system function mkstemp,
use file sequence No. with pthread_mutex_lock
* bug fixed: function fastdfs_tracker_list_groups, when parameter group_name
is null or empty string, return all groups info
* bug fixed: upload a file extends 2GB will fail
* bug fixed: tracker to tracker sync system data files, in function:
tracker_mem_get_tracker_server, pTrackerStatus not be set properly
Version 2.05 2010-12-05
* client/fdfs_monitor.c: add sync delay time
* tracker/fast_task_queue.c: pTask->data = pTask->arg + arg_size;
change to: pTask->data = (char *)pTask->arg + arg_size;
* bug fixed: storage_sync.c line 237 cause core dump in Ubuntu 10.04
* upload file test use mmap, support more test_upload processes
* client add three tools: fdfs_upload_file, fdfs_download_file and
fdfs_delete_file
Version 2.04 2010-11-19
* storage.conf: tracker server ip can NOT be 127.0.0.1
* do not catch signal SIGABRT
* strerror change to STRERROR macro
* sync copy file use temp filename first, rename to the correct filename
when sync done
* file id use 4 bytes CRC32 signature instead of random number
* add file: client/fdfs_crc32.c
* one of file hash code signature function change from APHash_ex
to simple_hash_ex
* bug fixed: when fdfs_storaged quit, maybe write to binlog file fail,
the error info is "Bad file descriptor"
Version 2.03 2010-11-08
* bug fixed: core dump when http.need_find_content_type=false and
http.anti_steal.check_token=true
* storage server add join_time field (create timestamp of this storage)
* tracker server fetch system files from other tracker server when
first storage server join in (tracker to tracker sync system files)
* tracker server changes the old ip address to the new address when the
storage server ip address changed
* tracker to tracker sync system data files in some case, multi tracker
server supported well
Version 2.02 2010-10-28
* get parameters function from tracker server changed,
add paramter: storage_sync_file_max_delay
* local ip functions move to common/local_ip_func.c
* when query all storage servers to store, do not increase the current
write server index
* struct FDFSHTTPParams add field: need_find_content_type
* symbol link client library to /usr/lib64 in 64 bits OS
* storage_client.c: deal file extension name correctly
Version 2.01 2010-10-17
* client/fdfs_monitor.c can specify tracker server
* micro STORAGE_STORE_PATH_PREFIX_CHAR change to
FDFS_STORAGE_STORE_PATH_PREFIX_CHAR
* php extension can set log filename
* php extension add function: fastdfs_client_version
* bug fixed: client/tracker_client.c tracker_get_connection_ex NULL pointer
* set max core dump file size to at least 256MB when DEBUG_FLAG is on,
make sure to generate core file when core dump with DEBUG_FLAG on
* upload file can get available storage server list of the group,
add command TRACKER_PROTO_CMD_SERVICE_QUERY_STORE_WITHOUT_GROUP_ALL and
TRACKER_PROTO_CMD_SERVICE_QUERY_STORE_WITH_GROUP_ALL
* bug fixed: storage core dump in some case
Version 2.00 2010-08-22
* tracker network io use libevent instead of traditional io model
* storage network io use libevent instead of traditional io model
* storage disk read/write use separate threads
* tracker_mem.c malloc single group and storage struct, remove referer
* make install copy config files
* tracker.conf add two parameters: storage_sync_file_max_delay and
storage_sync_file_max_time
* client tracker_get_connection increase server_index correctly
* storage sync to storage server adds active test
* test programs compile ok
Version 1.29 2010-06-30
* add files: tracker_dump.h and tracker_dump.c, tracker dump global vars
* add files: storage_dump.h and storage_dump.c, storage dump global vars
* sockopt.c: tcprecvfile and tcpdiscard add parameter total_recv_bytes
* storage server add fields: storage_port and storage_http_port
* auto rename synced remark files when the port of all storage servers
in a group changed to another port
* connect server support timeout, adding connect_timeout parameter in
config file
* log_init set log to cache to false (no cache)
Version 1.28 2010-05-30
* tracker_servive.c: set current_write_group anyway when current group
out of space
* logger support context (multi instance)
* get storage servers by filename: if the file created one day ago (the create
timestamp of the file < current_time - 86400), any active storage server matches
* add files: common/pthread_func.h and common/pthread_func.c
* common/sched_thread.h, remove statement: extern bool g_continue_flag;
* client add libfastcommon
* global variables: g_base_path, g_network_timeout, g_version change to
g_fdfs_base_path, g_fdfs_network_timeout, g_fdfs_version
* common/fdfs_base64.h/c change name to common/base64.h/c
* make.sh use TARGET_PREFIX instead of TARGET_PATH
* protocol add ACTIVE_TEST, tracker and storage both support
* php client, bug fixed: fastdfs_connect_server, the sock must init to -1
* bug fixed: storage status not correct with multi tracker servers
* sync storage mark file and stat file to disk properly
Version 1.27 2010-04-10
* storage.conf: add if_alias_prefix parameter to get the ip address of the
local host
* storage http support domain name
* php extension add some parameters in fastdfs_client.ini
* make.sh compile use debug mode
* type off_t change to int64_t
* redirect stdout and stderr to log file
* php extension list_groups add fields: version and http_domain
Version 1.26 2010-02-28
* remove compile warning of logError
* ini reader support section
* bug fixed: tracker/tracker_mem.c sync storage server status
* use storage server http server port anyway
* bug fixed: ini reader can support relative config filename
* function enhancement: tracker server can check storage HTTP server alive
Version 1.25 2010-02-04
* storage_sync.c if source file not exist when sync a file, change from
logWarning to logDebug
* filename buff size change from 64 to 128
* bug fixed: c client and php client, log not inited cause core dump when
call log functions
* can print stack trace when process core dumped in Linux server
* bug fixed: tracker/tracker_mem.c load storage servers fail with many groups
and storage servers
* common/sockopt.c remove debug info
* storage stat add fields: version
* auto adjust when storage server ip address changed
* bug fixed: when add a new storage server, other storage servers' status keep
the same, not changed
* add macros, compile passed in cygwin, thanks Seapeak
* write to system data file using lock
* common/ini_file_reader.c: use one context parameter, not two parameters
* storage status sync modified (the code of tracker and storage both changed)
* when recv kill signal, worker thread quit more quickly, daemon process
fdfs_trackerd and fdfs_storage quit very quickly when recv kill signal
* remove compile warning info of logError
* tracker server start more quickly with many groups and storage servers
* bug fixed: correct off_t printf format
Version 1.24 2010-01-06
* call php_fdfs_close with TSRMLS_CC as php_fdfs_close(i_obj TSRMLS_CC)
* storage server to storage server report ip address as tracker client
* bug fixed: sendfile exceeds 2GB file in Linux
* bug fixed: delete storage server
* storage stat add fields: up_time and src_ip_addr
* big static or struct memeber char array buffer change to malloc in order to
decrease stack size
* FDFS_WRITE_BUFF_SIZE change from 512KB to 256KB
* bug fixed: client/storage_client.c, meta data miss when upload file
* decrease thread_stack_size default value in config files: tracker.conf
and storage.conf
Version 1.23 2009-11-29
* remove unuseless variable "sleep_secs" in tracker_report_thread_entrance
* storage can bind an address when connect to other servers (as a client)
* common/md5.h fix UINT4 typedef wrong type in 64 bit OS
* client/fdfs_test.c: print the source ip address decoded from the remote
filename
* client add function fdfs_get_file_info
* php extension add functions: fastdfs_http_gen_token and fastdfs_get_file_info
* server process will exit when the http service starts fail
* support file group, a master file with many slave files whose file id can be
combined from master file id and prefix
* php client support uploading slave file
* ip address in filename change from host byte order to network byte order
* storage sync performance enhancement, using read buffer of 64KB to avoid
reading binlog file repeatly
* storage add prototol cmd: STORAGE_PROTO_CMD_QUERY_FILE_INFO
* FDFS_FILE_EXT_NAME_MAX_LEN changed from 5 to 6
* get file info support slave file
* storage server for uploading file support priority
Version 1.22 2009-10-12
* bug fixed: common/shared_func.c allow_hosts array maybe overflow in some case
* tracker/tracker_mem.c: protocol TRACKER_PROTO_CMD_SERVICE_QUERY_FETCH_ALL,
return at least a storage server when active storage
server count of the group > 0
* bug fixed: when client connection disconnected, always log debug or error info
* make.sh: default not install FastDFS services in Linux server
* common/sockopt.c: setsockopt level SOL_TCP only supported in Linux
* common/http_func.c: do not use function strsep because strsep is not portable
* client upload file support callback function
* client support multi tracker groups (multi FastDFS clusters)
* bug fixed: thread_stack_size not correct when the param thread_stack_size
not set in the config file
* supply php extension (directory name: php_client)
* c client reconnect server (tracker or storage) when network IO error
* c client: make tracker server index counter thread safely
Version 1.21 2009-09-19
* bug fixed: when source storage server synced file to new storage server done,
it's status changed to ONLINE (should keep as ACTIVE, report by zhouzezhong)
* add thread_stack_size in config file, default value is 1MB (report by chhxo)
* tracker and storage server use setsockopt to keep alive
(report by zhouzezhong)
* bug fixed: storage server with multi-path, upload file fail when the free
space of each path <= reserved space (the total free space > reserved space,
report by zhouzezhong)
* storage_sync.c: when connect fail, do not change the dest storage server '
status to offline
* tracker_service.c and storage_service.c change log level from WARNING to DEBUG
when client connection disconnected (report by Jney402)
* bug fixed: tracker_client.c correct store_path_index return by tracker server
(report by happy_fastdfs)
* bug fixed: tracker_service.c when store_lookup set to 2 (load balance), use
another pthread lock to avoid long time lock waiting
(report by happy_fastdfs)
* add service shell scripts in directory: init.d
(services will auto installed on Linux, report by hugwww)
Version 1.20 2009-09-05
* base64 use context, functions changed
* common/ini_file_reader.c: fix memory leak
* tracker server support HTTP protocol, one thread mode
* storage server support HTTP protocol, one thread mode
* fix bug: storage server rebuild, auto sync data correctly
* fix bug: sync data fail (correct storage server status)
* when storage server idle time exceeds check_active_interval seconds,
set it's status to offline
* tracker counter thread safely
Version 1.19 2009-07-23
* use poll instead of select in sockopt.c
* hash.c use chain impl by self
* use FastDHT 1.09 client code
* ini reader support HTTP protocol, conf file can be an url
* correct test dir compile error
* use non-block socket to increase network IO performance
* add cmd TRACKER_PROTO_CMD_SERVICE_QUERY_FETCH_ALL: query all storage servers
from which the file can be dowloaded
* while (1) ... break; changed to do ... while (0);
Version 1.18 2009-05-24
* restart.sh only kill the programs match the program name and all parameters
* correct get local ip addresses
* common files do not use global vars like g_network_timeout and g_base_path
* download file support offset and download bytes
* hash function change type from unsigned int to signed int
* file size in file name support 64 bits, old bytes is 4, new bytes is 8
Version 1.17 2009-03-19
* add test programs at sub directory test/
* common/shared_func.c: rindex change to strrchr, add #include <netinet/in.h>
* support SunOS (Solaris), compile passed on SunOS 5.10
* support AIX, compile passed on AIX 5.3
* sys call statfs change to statvfs
* use scheduling thread to sync binlog buff / cache to disk, add parameter
"sync_binlog_buff_interval" to conf file storage.conf
* use FastDHT v1.07 client code
Version 1.16 2009-02-14
* client can specify group name when upload file
* tracker_service.c: cmd dispatch changed to "switch ... case"
not "if ... else if"
* storage_service.c: call fdfs_quit before tracker_disconnect_server
Version 1.15 2009-01-28
* use FastDHT v1.04 client code
* use FastDHT client thread safely
Version 1.14 2009-01-18
* storage/storage_sync.c:
old: if (reader.sync_row_count % 1000 == 0)
new: if (reader.scan_row_count % 2000 == 0)
* little adjustment for common files can be used by FastDHT
* sched_thread.h /.c add global variable g_schedule_flag to quit normally
* shared_func.h / .c add function get_time_item_from_conf
* sched_thread.h /.c support time_base of task
* hash.h / .c add function CRC32, add hash function to support stream hash
* add FastDHT client files in storage/fdht_client/
* create symbol link when the file content is duplicate,
add item "check_file_duplicate" to conf file storage.conf
* use FastDHT v1.02 client code
* auto delete invalid entry in FastDHT when the source file does not exist
Version 1.13 2008-11-29
* re-calculate group 's free space when one of it's storage servers'
free space increase
* add parameters: sync_interval, sync_start_time and sync_end_time to
storage.conf
* performance enhancement: log to buffer, flush to disk every interval seconds
* standard fds closed by daemon_init: 0(stdin), 1(stdout) and 2(stderr)
* fix bug: pthread_kill sometimes cause core dump when program terminated
* fix bug: sync.c open next binlog cause loop call
Version 1.12 2008-11-12
* storage server support multi path (mount point)
* upload file support file ext name, add source storage ip address to filename
* add delete command to delete the invalid storage server
* add client functions which combine group name and filename to file id,
add anothor client test program: fdfs_test1.c to use file id
* client download file support callback function
* add protocol cmd TRACKER_PROTO_CMD_SERVICE_QUERY_UPDATE,
and client API add tracker_query_storage_update
* add protocol cmd TRACKER_PROTO_CMD_STORAGE_SYNC_REPORT to report last
synced timestamp as dest server
* fix sync old data files to new server bug
* fcntl change to pthread_mutex_lock
Version 1.11 2008-10-04
* kill report and sync threads when recv terminate signal
* add item "store_server" in tracker.conf, by default use the first
storage server to store uploaded files
* ini_file_reader.c changed: a conf file can include other conf files
* some adjustment:
some macro name changed
add common_define.h
remove fdfs_define.c
fdfs_os_bits.h change to _os_bits.h
Version 1.10 2008-09-20
* performance optimizing: use thread pool, create all work threads at startup
* trim function op in shared_func.c
* add Makefile template Makefile.in, delete Makefile and Makefile.freebsd
change make.sh to support all unix systems (passed in Linux and FreeBSD)
Version 1.9 2008-09-14
* security enhancement: support allow hosts which can connect to the server
* server can be run by the specified group and user, set by the config file
* change make.sh and add file common/fdfs_os_bits.h,
remove the warning info of printf format for int64_t param in 64 bits system
* storage_client.c changed: auto connect to storage server when not connected
* change some macro name and function name in tracker/tracker_proto.h
Version 1.8 2008-09-07
* communication protocol changed to support large file exceed 2GB:
# all integer field is 8 bytes big-endian
# group name fixed length: FDFS_GROUP_NAME_MAX_LEN bytes
* storage stat numbers (such as total_upload_count, success_upload_count)
use int64_t (8 bytes integer)
* ini_file_reader.c add function iniGetInt64Value
* sockopt.c add function tcpsetnonblockopt
* shared_func.c add function set_nonblock
Version 1.7 2008-08-31
* performance optimizing:
# change fopen to syscall open
# increase the efficiency of socket functions tcpsenddata and tcprecvdata
* change the return value of socket funtions such as tcpsenddata,
tcprecvdata and connectserverbyip
old return value: result=1 for success, result != 1 fail
new return value: result=0 for success, result != 0 fail, return the error code
* log function enhancement:
# support log level
# parameter "log_level" added to server config file
# keep the log file opened to increase performance
* fix log format and parameter mismatched bug (check by printf)
* log CRIT message to log file when program exit unexpectedly
* Makefile add compile flag -D_FILE_OFFSET_BITS=64 to support large files
* change the type of file_size and file_offset to off_t
* change signal to sigaction
* fix client Makefile to compile library correctly
* restart.sh modified: use external command "expr" to replace shell command "let"
Version 1.6 2008-08-24
* add restart daemon shell script: restart.sh
* use setrlimit to increase max open files if necessary
* security enhancement: the format of data filename must be: HH/HH/filename,
eg. B9/F4/SLI2NAAMRPR9r8.d
* fix bug: errno is not correct where the downloaded file does not exist,
communication is broken when the download file is a directory
Version 1.5 2008-08-17
* add client function storage_download_file_to_file
* use pthread_attr_setstacksize to increase thread stack size to 1 MB
* use sendfile syscall to send file in Linux and FreeBSD
* fix bug: add O_TRUNC flag when open file to write
* remove warning info compiled by gcc 4.2
* fcntl set lock.l_len to 0
Version 1.4 2008-08-10
* storage server recv file method change
old method: recv the whole file content/buff before write to file
new method: write to file once recv a certain bytes file buff, eg. 128KB buff size
* storage client and storage server send file method change
old method: get the whole file content/buff, then send to storage server
new method: send file to storage server more times. get a certain bytes file buff, then send to storage server
* upload file package remove the one pad byte field
* remove storage status FDFS_STORAGE_STATUS_DEACTIVE and add FDFS_STORAGE_STATUS_DELETED
Version 1.3 2008-08-03
* fix bug: when meta data is empty, get meta data return error
* support java client
# memset response header to 0
# add group_name to upload file response package
Version 1.2 2008-07-27
* add client function storage_set_metadata to support setting metadata(overwrite or merge)
Version 1.1 2008-07-20
* implement storage disk report
* storing load balance between storage groups(volumes) when set store_lookup to 2
Version 1.0 2008-07-12
* first version