Newer versions of Windows 10 use several reparse tags for files which
are synchronized to OneDrive cloud storage (0x9000301a, 0x9000601a,
0x9000701a, ...). identify them as IO_REPARSE_TAG_CLOUD and use a
single plugin to process them.
Windows 10 brought a new type of reparse point for directories (0x80000018),
so add opendir() and readdir() to the plugin interface to take directories
into account. The interface for releasedir() is merged with release() as
the plugins can discriminate them if needed.
Windows 10 brought security descriptors which contain ACE's related to
new ways of accessing files or directories. These ACE are now accepted
with minimal consistency checks. They are still ignored for translating
permissions and for Windows-type inheritance.
Windows places filenames with a trailing dot or space in the Win32
namespace and allows setting DOS names on such files. This is true even
though on Windows such filenames can only be created and accessed using
WinNT-style paths and will confuse most Windows software. Regardless,
because libntfs-3g did not allow setting DOS names on such files, in
some cases it was impossible to correctly restore, using libntfs-3g, a
directory structure that was created under Windows.
Update ntfs_set_ntfs_dos_name() to permit operating on a file that has a
long name with a trailing dot or space. But continue to forbid creating
such names on a filesystem FUSE-mounted with the windows_name option.
Additionally, continue to forbid a trailing a dot or space in DOS names;
this matches the Windows behavior.
(contributed by Eric Biggers)
Following a recent patch to the fuse kernel, the Posix ACL checks can
be done within the kernel instead of having to be done in the file
system, provided lowntfs-3g is used.
This mode is still not used by default until the fuse patch is
released by distributions.
Now that the size of the reparse point attribute is no longer used by
the FUSE drivers to populate st_size for symlinks and junctions, it no
longer needs to be returned by ntfs_make_symlink().
(contributed by Eric Biggers)
The validation contained an off-by-one error. The
expression '(u32)(usa_ofs + (usa_count * 2)) > size' used 'usa_count'
after it had been decremented to skip the update sequence number entry.
Consequently, the code could read out of bounds, up to two bytes past the
end of the MST-protected record.
Furthermore, as documented in the comment in layout.h for "NTFS_RECORD"
and also on MSDN for "MULTI_SECTOR_HEADER", the update sequence array
must end before the last le16 in the first logical sector --- not merely
before the end of the record.
Fix the validation and move it into a helper function, as it was done
identically in the read and write paths.
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
Currently, applications that wish to access security descriptors have to
explicitly open the volume's security descriptor index ("$Secure") using
ntfs_open_secure(). Applications are also responsible for closing the
index when done with it. However, the cleanup function for doing,
ntfs_close_secure(), cannot be called easily by all applications because
it requires a SECURITY_CONTEXT argument, not simply the ntfs_volume.
Some applications therefore have to close the inode and index contexts
manually in order to clean up properly.
This proposal updates libntfs-3g to open $Secure unconditonally as part
of ntfs_mount(), so that applications do not have to worry about it.
This proposal updates libntfs-3g to open $Secure unconditonally as part
of ntfs_mount(), so that applications do not have to worry about it.
ntfs_close_secure() is updated to take in a ntfs_volume for internal use,
and ntfs_destroy_security_context() is now the function to call to free
memory associated with a SECURITY_CONTEXT rather than a ntfs_volume.
Some memory leaks in error paths of ntfs_open_secure() are also fixed.
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
A number of functions in libntfs-3g are generally useful, but are tied to
extended attributes support and are not included when the library is
built on platforms without extended attributes support.
This proposal updates libntfs-3g to always include these functions.
The only tricky part is dealing with the XATTR_CREATE and XATTR_REPLACE
flags. These flags are defined in <sys/xattr.h>, so they must be
redefined on platforms without extended attributes support.
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
- Update documentation for COLLATION_RULES
- Document how ntfs_names_full_collate() compares names
- Update comments and DEBUG code to reflect that ntfs_names_full_collate()
always access 'upcase', even in CASE_SENSITIVE mode
- Remove unneeded assignments to 'c1' and 'c2' in IGNORE_CASE mode
Signed-off-by: Eric Biggers <ebiggers3@gmail.com>
When a plugin cannot be initialized a ELIBBAD error is returned on Linux
and OpenIndiana. As this is not a Posix error code, use ENOEXEC on
systems which do not define ELIBBAD.
Prepare merging ntfsrecover.h into logfile.h by adding a usn field to
RESTART_PAGE_HEADER. As this changes the record size, ignore the new
field in existing code.
User extended attributes should only be set on files and directories,
not on symlinks, sockets, devices, etc. For safety they are also
forbidden on metadata files, but should be allowed on the root
directory. For files based on reparse points, requests are made
to the plugin to determine the type.
Kernel cacheing of file attributes is usually not used by ntfs-3g,
because it has defects when dealing with hard linked files and directory
permission checks. Kernel cacheing is however possible when using
lowntfs-3g and not using Posix ACLs.
The new "system compression" files used by Windows 10 make use of reparse
points to record the compression parameters, and a specific named data
stream is used to store the compressed data. With this patch, processing
of reparse points can be done by an external plugin only loaded as needed.
Junctions and symlinks, which are also based on reparse points, are now
processed by "internal plugins".
On the OpenIndiana Hipster distribution, compiling with GCC 4.9 would
fail because __BYTE_ORDER__ was defined but not to any of the values
assumed to be associated with this define (__LITTLE_ENDIAN__ or
__BIG_ENDIAN__). Instead it was defined to either
__ORDER_LITTLE_ENDIAN__ or __ORDER_BIG_ENDIAN__. This caused
compilation to fail.
Fixed by checking that all referenced defines are in fact defined
before using them and adding an additional #elif clause for this newly
discovered condition.
This fixes compiler warnings emitted when you compare an le32 value with
e.g. 'const_cpu_to_le32(-1)' on a little-endian system, because
previously the expansion of the macro expression 'const_cpu_to_le32(-1)'
would be '(-1)' on a little-endian system but '(u32)((((u32)(-1) &
0xff000000u) >> 24) | (((u32)(-1) & 0x00ff0000u) >> 8) | (((u32)(-1) &
0x0000ff00u) << 8) | (((u32)(-1) & 0x000000ffu) << 24))' on a
big-endian system, i.e. the type of the expanded expression would be
'int' (signed) in the little-endian case but 'u32' (unsigned) in the
big-endian case.
With this commit the type of the expanded expression will be 'le32' in
both the little-endian and the big-endian case.
The new compression formats used by Windows 10 uses reparse data, and
a new reparse tag which it is useful to define even though these formats
is not yet supported by ntfs-3g.
On-disk struct definitions used native types (u16/u32/u64/s16/s32/s64),
which doesn't say anything about the intended interpretation of the
data. The intention of having little-endian-specific types and
big-endian-specific types must have been to clarify interpretation of
data and intentions in the code. Therefore it seems reasonable to use
these types in struct definitions to clarify what data represention is
used to encode field data.
Because some struct members in layout.h are big-endian, this change also
means moving the duplicated definitions for big-endian byteswapping
macros and big-endian types found in acls.h and security.h to the
appropriate locations in endians.h and types.h respectively in order to
make them available for the struct definitions in layout.h.
The fuse cache does not handle properly hard-linked files, so ntfs-3g
traditionally disables it by using a null time-out. With an upgrade of
the fuse kernel module on OpenIndiana, cacheing of non hard-linked files
is now possible, so use it.
ntfs_valid_sid() required that the subauthority count be between 1 and 8
inclusively. However, Windows permits more than 8 subauthorities as well
as 0 subauthorities:
- The install.wim file for the latest Windows 10 build contains a file
whose DACL contains a SID with 10 subauthorities.
ntfs_set_ntfs_acl() was failing on this file.
- The IsValidSid() function on Windows returns true for subauthority
less than or equal to 15, including 0.
There was actually already a another SID validation function that had the
Windows-compatible behavior, so I merged the two together.
Contributed by Eric Biggers
Newer versions of Windows use more recent definitions of upper-case
table defined by the Unicode consortium. Now using the same table as
Windows 7, windows 8 and Windows 10. This only has an effect on file
systems newly created by mkntfs.
The support for ioctls has been added to fuse when using protocol 7.18,
and an equivalent upgrade has been done in fuse lite with commit [ae9aee].
For old kernels, a fall back to protocol 7.8 was implemented, but this
appears not to be supported in not-so-old kernels (e.g. 2.6.35).
With this patch, the fall back protocol is set to 7.12 or to the highest
level supported by the kernel.
fstrim(8) discards unused blocks on a mounted filesystem. It is useful for
solid-state drives (SSDs) and thinly-provisioned storage.
Only trimming the full device (with no option) is supported.
Contributed by Richard W.M. Jones
This is backporting code from the full FUSE library in order to support
ioctls. The fuse protocol level negociated is now 7.18 instead of 7.8.
A fallback protocol to 7.8 is provided for compatibility with older kernels.
32-bit ioctls are not supported by a 64-bit library
When Posix ACLs are used, the umask is ignored and the initial permissions
of created files are taken for the parent directory. However the umask
should still be used when the Posix ACLs are not enabled in the mount
options.
chkdsk deletes the ACLs when they are bad or when they are not used any
more. This fixes inserting a new ACL after the previously last ACL (or
even all of them) was deleted.
Windows applies legacy restrictions to file names, so when the option
windows_names is applied, reject the same reserved names, which are
CON, PRN, AUX, NUL, COM1..COM9, and LPT1..LPT9
When the runlist of the data attribute of MFT has to be split across
several extents, the location of each extent has to be known from the
runlist present in previous extents. So, force the first extent into
record 15 to avoid a bad layout.
When ntfsclone'ing to a file, the target file was truncated to the volume
size. This is not useful on file systems which support sparse files. In
the case of ntfs-3g this leads to prevent optimizations specific to
appending data. So when a sparse output file is detected, it is emptied
to benefit from subsequent appending of data.
When calling ntfstruncate() to expand a resident attribute, the function
is called again recursively, losing the requirement for not inserting
holes. This is for forwarding the requirement (used by ntfscp).
When issuing an utimensat as a consequence of utime(2) or utimensat(2),
fuse had temporarily defined a flag utime_omit_ok to identify whether
the file system supports the values UTIME_OMIT and UTIME_NOW to mean
specific timestamp updatings. The flag has been obsoleted and all
file system are now supposed to comply with the convention.
