6ca5a24a7f
Fix two inconsistency issues between stat cache and cache file ( #2152 )
...
* Fix inconsistency between stat cache file and cache file
We unlock stat cache file too early in FdEntity::Open(), and would
truncate cache file and update stat cache file, so there's a window that
stat cache doesn't reflect cache file status.
Suggested-by: Takeshi Nakatani <ggtakec@gmail.com >
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com >
* Mark pagelist as unloaded if cache file has been truncated
If cache file size doesn't match object size, the cache file might be
corrupted, so invalidate it and save new cache stat file.
Suggested-by: Takeshi Nakatani <ggtakec@gmail.com >
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com >
---------
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com >
2023-05-30 18:39:50 +09:00
18495c44aa
Added cache clearing on upload error ( #2127 )
2023-03-13 21:22:13 +09:00
f8a825e9b9
multipart upload id is converted by url encode ( #2097 )
2023-02-18 11:40:25 +09:00
0ba49518e9
Make some methods and parameters const ( #2078 )
...
This requires making some locks mutable.
2023-01-04 20:23:39 +09:00
c491fbeabc
Replace uses of lock_already_held flag with AutoLock::Type
2022-07-30 16:20:43 +09:00
48e9e51f4f
Remove more unneeded headers identified by IWYU ( #2011 )
2022-07-30 12:06:47 +09:00
3928a7e359
Remove more uses of const_cast ( #2006 )
...
Follows on to #2004 .
2022-07-28 23:37:15 +09:00
4b2f3fecb5
Set mtime/ctime/atime of all objects as nanosecond
2022-07-28 13:47:03 +09:00
e0655008b3
Protect pending_status in UploadPending ( #1992 )
...
This requires avoiding double-locking in RowFlush. References #1991 .
2022-07-22 23:30:04 +09:00
22f2392fca
Fixed bugs about stream upload
2022-07-19 21:29:56 +09:00
faddb4900f
Merged the code corresponding to the mknod fix( f11eb7d)
2022-07-17 22:20:45 +09:00
b0eeaa6679
Reflected the result of the review in the code
2022-07-17 22:20:45 +09:00
d22e1dc018
Add the stream upload which starts uploading parts before Flush
2022-07-17 22:20:45 +09:00
f11eb7d69b
Fixed a bug that regular files could not be created by mknod
2022-06-29 16:56:19 +09:00
73b49c1038
Fixed a bug that regular files could not be created by mknod
2022-06-29 16:56:19 +09:00
8a5c4306f5
Preserve sub-second precision where possible ( #1915 )
2022-02-23 23:58:51 +09:00
08adffd2fe
Fix typos ( #1916 )
2022-02-23 23:31:52 +09:00
839a33de49
Fixed not to call Flush even if the file size is increased ( #1887 )
...
Changed s3fs_truncate function.
This change reduces the number of file uploads if the file size is changed.
On macOS, I have found that the truncate call when "size=0" cannot reflect the file size.(This reason is not understood...)
To avoid this, only when "size=0", the flush method is called as before.
Other than that, I found a bug in FdEntity::Open() and fixed it.
Fixes #1875 .
2022-02-15 21:29:07 +09:00
1678803566
Added S3fsCred class and moved Credential related processing in it
2022-02-13 21:38:30 +09:00
f6ed972926
Always flush open files with O_CREAT flag ( #1879 )
...
Previously s3fs only created files that had dirty data and not those
with zero-bytes. Regression from
771bbfeac5#1013 . Found
via pjdfstest. References #1589 .
2022-01-30 22:02:37 +09:00
023aaf7dff
Fixed wrong stat of cache after new creation file
...
And added a test for stat of cache after new creation file
2021-10-17 16:10:14 +09:00
2f412804e2
Fixed forgetting to clear the dirty flag for meta information
...
Addressed an error in macos cpp check
2021-10-15 22:54:55 +09:00
d9f2d17040
1. fix RowFlush can not upload last part smaller than 5MB using NoCacheMultipartPost; ( #1753 )
...
2. fix deadlock in UploadPendingMeta
2021-08-31 00:41:47 +09:00
a868c0656e
Changed etaglist_t from string list to new structure etagpairs list
2021-08-16 09:27:12 +09:00
199b3d4709
Fixed a bug in disk free space calculation
2021-07-18 14:52:11 +09:00
945cc2ac54
Added UntreatedParts class instead of untreated upload info members in PseudoFdInfo
2021-07-12 21:45:29 +09:00
c30acbbf90
Splited some methods of FdEntity class by uploading mode
...
- Splited FdEntity::Write method by uploading mode
- Splited FdEntity::RowFlush method by uplading mode
2021-07-10 17:24:34 +09:00
8948eded09
Address clang-tidy warnings ( #1703 )
2021-06-30 08:42:44 +09:00
2f59cb5a0a
Remove unneeded volatile qualifiers ( #1702 )
...
We should use proper locking instead. Specifically for
is_meta_pending this does not do what was intended due to the
read-modify-write of the member.
2021-06-30 08:20:44 +09:00
f505c8224e
Explicitly specify lock type ( #1701 )
...
This makes it more clear and type-safe if the caller already has the
lock. Follows on to 84174c560d
2021-06-27 15:15:48 +09:00
2154e898bc
Fix typos ( #1700 )
2021-06-27 11:22:33 +09:00
f9e80f995d
Fixed a bug about rename existing file of different sizes by mpcopy
2021-06-26 00:04:58 +09:00
6edf3d6427
Updated to clearly output message about the file descriptor type(pseudo/physical)
2021-06-25 19:18:08 +09:00
5b6684ca19
Fixed a bug utimens is calling before flush
2021-06-21 17:20:24 +09:00
d67b83e671
Allow configuration for temporary files directory
2021-06-16 21:29:58 +09:00
4d39ea887e
Protect orgmeta with fdent_lock ( #1678 )
...
Fixes a crash seen with the CentOS 7 builder and when running with
AddressSanitizer. Regression introduced by
ac578d188eFixes #1677 .
2021-06-13 16:14:24 +09:00
c2c56d0263
Added info object about multipart uploading for each pseudo fd
...
(and fixed typo about method name)
2021-06-04 22:42:58 +09:00
ac578d188e
Introduced pseudo fd and separated fd for each file opening
2021-06-04 22:42:58 +09:00
84174c560d
Fix data races caused by incorrect locking ( #1668 )
...
Found via Threadsanitizer. Fixes #1471 .
2021-05-29 00:11:55 +09:00
9bf525ee7a
Ensuring multipart size even when storage is low
...
When the temporary storage was full, the old implementation started an
upload even if there was not enough data to completely fill the minimum
multipart size or fill the user-selected multipart size.
The new implementation ensures the minimum multipart size by forcing
the user-selected multipart size.
Fixes #1591
2021-05-27 22:55:52 +09:00
4b69d4b1bb
Fixed a bug when the disk capacity was insufficient in RowFlush
2021-05-22 23:11:46 +09:00
3694786112
Propagate errno instead of EIO ( #1638 )
...
This improves error fidelity. Follows on to
b70f8db037#1523 .
2021-04-30 07:09:00 +09:00
1838f52e19
Declare undefined symbols for fallocate function
2021-04-25 10:12:25 +09:00
7f3e423bbe
Preserve sub-second time precision ( #1624 )
...
Found via pjdfstests. References #897 . References #1589 .
2021-04-18 13:11:12 +09:00
f6fbd75320
Return EFBIG when file exceeds multipart limit ( #1600 )
...
Found via pjdfstests. References #1589 .
2021-03-28 12:33:01 +09:00
ef079f4e94
Don't ignore nomultipart when storage is low
...
When the temporary storage filled up, the old implementation uploaded
all data with multipart uploads, even if "nomultipart" was set.
The new implementation emits a warning and returns -ENOSPC instead.
Fixes #1595
2021-03-10 08:23:54 +09:00
5b5bc3114a
Fixed a bug that padded null bytes when changing xattr
2021-02-21 12:23:36 +09:00
b0e8758b63
Use result instead of res for consistency ( #1530 )
2021-01-25 07:56:10 +09:00
d9f6469b7b
Fixed flushing dirty data and compressed the cache size
2020-11-14 16:45:37 +09:00
543231c9f2
Address warnings from clang-tidy 11 ( #1470 )
2020-11-09 21:15:20 +09:00
