e5b8377202
fix comments and code
2019-02-19 10:32:10 +00:00
4f42f4ab0c
Enable s3fs encoding and decoding in the fuse interface
2019-02-18 13:36:48 +00:00
11b385820d
more robust wtf8 encoding
2019-02-18 12:27:44 +00:00
ca2d1d873d
Adding utility functions to convert invalid utf8 to wtf8 encoding
...
This is to deal with windows clients who pass in cp1252 as if it
were utf8
2019-02-15 15:57:03 +00:00
951761ee2c
Merge pull request #951 from ggtakec/master
...
Added a non-interactive option to utility mode
2019-02-11 03:03:01 +09:00
231fd001d9
Added a non-interactive option to utility mode
...
Rebase for resolving conflicts
Deleted interactive mode on utility mode
Fixed minor mistakes
2019-02-10 17:36:18 +00:00
e00afa8128
Merge pull request #957 from gaul/deps/s3proxy-1.6.1
...
Upgrade to S3Proxy 1.6.1
2019-02-10 23:32:13 +09:00
e9297f39ea
Upgrade to S3Proxy 1.6.1
...
Notably, this improve performance when listing objects using the
filesystem backend, allowing new tests like expanding large tar files.
Release notes:
https://github.com/gaul/s3proxy/releases/tag/s3proxy-1.6.1
2019-02-06 17:06:43 -08:00
314dc5a398
Merge pull request #955 from gaul/clang-tidy/misc
...
Address miscellaneous clang-tidy warnings
2019-02-06 21:21:54 +09:00
e07cb020cc
Merge pull request #954 from gaul/clear-iter
...
Clear containers instead of individual erases
2019-02-06 21:12:36 +09:00
9f79b9e0da
Merge pull request #953 from gaul/readme/s3ql
...
Update s3ql link
2019-02-06 21:03:28 +09:00
e87e40b3b4
Address miscellaneous clang-tidy warnings
...
Also add configuration with suppressions.
2019-02-04 22:58:40 -08:00
f0f95478ec
Update s3ql link
2019-02-04 21:03:46 -08:00
bd66b57ad3
Merge pull request #952 from gaul/multipart/failed
...
Automatically abort failed multipart requests
2019-02-04 22:19:27 +09:00
a1d3ff9766
Automatically abort failed multipart requests
...
This can avoid dangling parts. However, many transfers fail due to
network errors so we still need other mechanisms to handle these
parts.
2019-02-03 10:29:20 -08:00
7f61a947c2
Merge pull request #950 from ggtakec/master
...
Added S3FS_MALLOC_TRIM build switch
2019-02-03 17:23:09 +09:00
4d0bef1e90
Clear containers instead of individual erases
...
This has O(n) runtime instead of O(n log n).
2019-02-02 23:58:43 -08:00
960823fb40
Added S3FS_MALLOC_TRIM build switch
2019-02-03 07:36:17 +00:00
c04e8e7a9d
Merge pull request #949 from gaul/503-slow-down
...
Implement exponential backoff for 503
2019-02-03 15:19:28 +09:00
fb6debd986
Merge pull request #948 from gaul/too-many-parts
...
Add logging for too many parts
2019-02-03 14:55:54 +09:00
d8185a25aa
Merge pull request #946 from gaul/async/completed-tids
...
Simplify async request completion code
2019-02-03 14:08:49 +09:00
53337a0a28
Merge pull request #944 from gaul/utility-mode
...
Repair utility mode
2019-02-03 13:59:56 +09:00
ae51556d04
Merge pull request #943 from gaul/hard-link
...
Return not supported when hard linking
2019-02-03 12:45:21 +09:00
b3de9195a7
Merge pull request #942 from gaul/c++03
...
Ensure s3fs compiles with C++03
2019-02-03 12:35:49 +09:00
055ecf6ea7
Merge pull request #940 from gaul/parallel-multipart-copy
...
Copy parts in parallel
2019-02-03 11:44:15 +09:00
c603680e02
Merge pull request #939 from gaul/stringstream-specific
...
Prefer specific [io]stringstream where possible
2019-02-03 11:06:57 +09:00
814aadd7e3
Merge pull request #938 from gaul/clang-tidy/redundant-void
...
Remove unneeded void parameter
2019-02-03 10:57:56 +09:00
dce63d1529
Merge pull request #937 from gaul/malloc-trim
...
Disable malloc_trim
2019-02-03 10:48:15 +09:00
8ff05d8e38
Implement exponential backoff for 503
...
Amazon returns SlowDown when overloaded. Also return ENOTSUP for 501
and immediately return EIO for 500 instead of retrying. Fixes #603 .
2019-02-02 00:03:50 -08:00
dfa84b82a8
Add logging for too many parts
...
References #610 .
2019-02-01 19:34:26 -08:00
6ac8618381
Ensure s3fs compiles with C++03
2019-02-01 08:33:59 -08:00
8c527c3616
Simplify async request completion code
...
Workers now notify the master thread when they complete, unifying the
Linux and macOS code paths. This also avoids excessive
pthread_tryjoin_np calls. Follows on to
88cd8feb05
2019-01-31 22:55:23 -08:00
54a074647e
Repair utility mode
...
This deinitialized S3fsCurl twice and incorrectly calculated V4
signatures.
2019-01-31 18:45:39 -08:00
c5ebf5d328
Copy parts in parallel
...
S3 can copy multipart much faster than single part due to IO
parallelization. Renaming a 4 GB file reduces from 72 to 20 seconds
with bigger gains with larger files.
2019-01-31 10:21:39 -08:00
43c6ef560e
Return not supported when hard linking
...
This is more correct than permission denied.
2019-01-30 16:43:04 -08:00
3076abc744
Disable malloc_trim
...
This avoids walking the entire heap multiple times for complex
operations like readdir. This does not entirely eliminate the
observed performance regression but does dramatically reduce s3fs CPU
usage. References #935 . Fixes #936 .
2019-01-29 15:29:07 -08:00
07636c8a8d
Prefer specific [io]stringstream where possible
...
These better communicate intent and are slightly more efficient.
2019-01-29 10:44:33 -08:00
35d55ee513
Remove unneeded void parameter
...
This is implicit in C++. Found and fixed via clang-tidy.
2019-01-28 23:22:27 -08:00
a442e843be
Merge pull request #934 from ggtakec/master
...
Checked and corrected all typo
2019-01-27 21:33:16 +09:00
c0cf90cf8b
Checked and corrected all typo
2019-01-27 12:04:29 +00:00
3b1cc3b197
Merge pull request #933 from gaul/cache/remove-mirror-path
...
Remove mirror path when deleting cache
2019-01-27 16:15:49 +09:00
a0c1f30ae7
Merge pull request #932 from gaul/autolock
...
Prefer AutoLock for synchronization
2019-01-27 15:59:18 +09:00
8822a86709
Merge pull request #931 from gaul/doc/typo
...
Correct sigv2 typo
2019-01-27 15:48:24 +09:00
98f397de0e
Merge pull request #930 from gaul/doc/md5-multipart
...
Correct enable_content_md5 docs
2019-01-27 15:36:56 +09:00
fd4d23f8f7
Merge pull request #926 from kzidane/master
...
Accept paths with : in them
2019-01-27 15:23:24 +09:00
4820f0a42b
Merge pull request #925 from gaul/clang-tidy/delete-null
...
Remove redundant null checks before delete
2019-01-27 15:15:31 +09:00
807a618cf7
Merge pull request #924 from gaul/clang-tidy/empty
...
Prefer empty over size checks
2019-01-27 15:04:52 +09:00
a93e500b44
Remove mirror path when deleting cache
...
Fixes #827 .
2019-01-25 18:10:03 -08:00
92d3114584
Prefer AutoLock for synchronization
...
This simplifies the code and fixes an issue with unlocked access.
Also use a recursive lock for StatCache to avoid races between
lock..unlock..lock sequences.
2019-01-25 15:28:41 -08:00
5062d6fbd9
Correct sigv2 typo
2019-01-25 14:30:30 -08:00
