summaryrefslogtreecommitdiff
path: root/Documentation
diff options
context:
space:
mode:
authorLinus Torvalds <torvalds@linux-foundation.org>2016-08-03 02:47:06 +0300
committerLinus Torvalds <torvalds@linux-foundation.org>2016-08-03 02:47:06 +0300
commit8cbdd85bda499d028b8f128191f392d701e8e41d (patch)
treec7d878c76ba27dc682a42d027cd9a2662418ac2f /Documentation
parent72b5ac54d620b29cae23d25f0405f2765b466f72 (diff)
parent302f0493f0bfaabd6f77ce7bfaa12620abf74948 (diff)
downloadlinux-8cbdd85bda499d028b8f128191f392d701e8e41d.tar.xz
Merge tag 'for-linus-v4.8' of git://github.com/martinbrandenburg/linux
Pull orangefs update from Martin Brandenburg: "Kernel side caching and executable bugfix This allows OrangeFS to utilize the dcache and adds an in kernel attribute cache. We previously used the user side client for this purpose. We see a modest performance increase on small file operations. For example, without the cache, compiling coreutils takes about 17 minutes. With the patch and a 50 millisecond timeout for dcache_timeout_msecs and getattr_timeout_msecs (the default), compiling coreutils takes about 6 minutes 20 seconds. On the same hardware, compiling coreutils on an xfs filesystem takes 90 seconds. We see similar improvements with mdtest and a test involving writing, reading, and deleting a large number of small files. Interested parties can review more data at the following URL. https://docs.google.com/spreadsheets/d/1v4aUeppKexIbRMz_Yn9k4eaM3uy2KCaPoe_93YKWOtA/pubhtml The eventual goal of this is to allow getdents to turn into a readdirplus to the OrangeFS server. The cache will be filled then, which should provide a performance benefit to the common case of readdir followed by getattr on each entry (i.e. ls -l). This also fixes a bug. When orangefs_inode_permission was added, it did not collect i_size from the OrangeFS server, since this presses an unnecessary load on the OrangeFS server. However, it left a case where i_size is never initialized. Then running an executable could fail. With this patch, size is always collected to be inserted into the cache. Thus the bug disappears. If this patch is not accepted during this merge window, we will send a one-line band-aid for this bug instead" * tag 'for-linus-v4.8' of git://github.com/martinbrandenburg/linux: Orangefs: update orangefs.txt orangefs: Account for jiffies wraparound. orangefs: Change default dcache and getattr timeout to 50 msec. orangefs: Allow dcache and getattr cache time to be configured. orangefs: Cache getattr results. orangefs: Use d_time to avoid excessive lookups
Diffstat (limited to 'Documentation')
-rw-r--r--Documentation/filesystems/orangefs.txt50
1 files changed, 46 insertions, 4 deletions
diff --git a/Documentation/filesystems/orangefs.txt b/Documentation/filesystems/orangefs.txt
index e1a0056a365f..1dfdec790946 100644
--- a/Documentation/filesystems/orangefs.txt
+++ b/Documentation/filesystems/orangefs.txt
@@ -281,7 +281,7 @@ on the wait queue and one attempt is made to recycle them. Obviously,
if the client-core stays dead too long, the arbitrary userspace processes
trying to use Orangefs will be negatively affected. Waiting ops
that can't be serviced will be removed from the request list and
-have their states set to "given up". In-progress ops that can't
+have their states set to "given up". In-progress ops that can't
be serviced will be removed from the in_progress hash table and
have their states set to "given up".
@@ -338,7 +338,7 @@ particular response.
PVFS2_VFS_OP_STATFS
fill a pvfs2_statfs_response_t with useless info <g>. It is hard for
us to know, in a timely fashion, these statistics about our
- distributed network filesystem.
+ distributed network filesystem.
PVFS2_VFS_OP_FS_MOUNT
fill a pvfs2_fs_mount_response_t which is just like a PVFS_object_kref
@@ -386,7 +386,7 @@ responses:
io_array[1].iov_base = address of global variable "pdev_magic" (int32_t)
io_array[1].iov_len = sizeof(int32_t)
-
+
io_array[2].iov_base = address of parameter "tag" (PVFS_id_gen_t)
io_array[2].iov_len = sizeof(int64_t)
@@ -402,5 +402,47 @@ Readdir responses initialize the fifth element io_array like this:
io_array[4].iov_len = contents of member trailer_size (PVFS_size)
from out_downcall member of global variable
vfs_request
-
+
+Orangefs exploits the dcache in order to avoid sending redundant
+requests to userspace. We keep object inode attributes up-to-date with
+orangefs_inode_getattr. Orangefs_inode_getattr uses two arguments to
+help it decide whether or not to update an inode: "new" and "bypass".
+Orangefs keeps private data in an object's inode that includes a short
+timeout value, getattr_time, which allows any iteration of
+orangefs_inode_getattr to know how long it has been since the inode was
+updated. When the object is not new (new == 0) and the bypass flag is not
+set (bypass == 0) orangefs_inode_getattr returns without updating the inode
+if getattr_time has not timed out. Getattr_time is updated each time the
+inode is updated.
+
+Creation of a new object (file, dir, sym-link) includes the evaluation of
+its pathname, resulting in a negative directory entry for the object.
+A new inode is allocated and associated with the dentry, turning it from
+a negative dentry into a "productive full member of society". Orangefs
+obtains the new inode from Linux with new_inode() and associates
+the inode with the dentry by sending the pair back to Linux with
+d_instantiate().
+
+The evaluation of a pathname for an object resolves to its corresponding
+dentry. If there is no corresponding dentry, one is created for it in
+the dcache. Whenever a dentry is modified or verified Orangefs stores a
+short timeout value in the dentry's d_time, and the dentry will be trusted
+for that amount of time. Orangefs is a network filesystem, and objects
+can potentially change out-of-band with any particular Orangefs kernel module
+instance, so trusting a dentry is risky. The alternative to trusting
+dentries is to always obtain the needed information from userspace - at
+least a trip to the client-core, maybe to the servers. Obtaining information
+from a dentry is cheap, obtaining it from userspace is relatively expensive,
+hence the motivation to use the dentry when possible.
+
+The timeout values d_time and getattr_time are jiffy based, and the
+code is designed to avoid the jiffy-wrap problem:
+
+"In general, if the clock may have wrapped around more than once, there
+is no way to tell how much time has elapsed. However, if the times t1
+and t2 are known to be fairly close, we can reliably compute the
+difference in a way that takes into account the possibility that the
+clock may have wrapped between times."
+
+ from course notes by instructor Andy Wang