preinit-mounts: init: Enhance handling of overlay errors

There has been a couple instances of overlay errors: duplicated
overlay mounts, and corrupted files that live in a subdirectory.

Add the following enhancements for when an overlay error is detected:
- If the etc umount fails, do not attempt to re-mount the overlay.
  This prevents duplicate mounts.
- Check each file after re-creating the overlay, and if it still has
  errors, delete it from the overlay. This would help also in cases
  like the point above where the umount fails and the overlay was not
  recreated.
- Check all overlay files recursively instead of just the ones in the
  top directory. One instance of this corruption was seen on a file
  in a subdirectory.

Tested: Verified the script would check all files in the overlay.

(From meta-phosphor rev: 50b569209789f824b7dac8c94adb01b78c0bf3f6)

Change-Id: I570a17ec00b7d303abe4654431c0c79d3dd95c1b
Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>

1 files changed, 30 insertions, 15 deletions

diff --git a/meta-phosphor/recipes-phosphor/preinit-mounts/preinit-mounts/init b/meta-phosphor/recipes-phosphor/preinit-mounts/preinit-mounts/init
index 818a32099..ad8132798 100644
--- a/meta-phosphor/recipes-phosphor/preinit-mounts/preinit-mounts/init
+++ b/meta-phosphor/recipes-phosphor/preinit-mounts/preinit-mounts/init
@@ -6,6 +6,21 @@ mount_overlay() {
   fi
 }
 
+recreate_overlay() {
+  # Attempt to re-create the overlay by moving out the overlay contents and
+  # copying them back to /etc, which would create them back in the overlay
+  cd
+  if ! umount /etc; then
+    return
+  fi
+  rm -rf /var/persist/etc-save
+  mv /var/persist/etc /var/persist/etc-save
+  mkdir -p /var/persist/etc
+  mount_overlay
+  cp -rp /var/persist/etc-save/* /etc/
+  rm -rf /var/persist/etc-save
+}
+
 if ! mount ubi0:rwfs /var -t ubifs -o defaults; then
   if ! mount ubi0:rwfs /var -t ubifs -o defaults,ro; then
     mount tmpfs /var -t tmpfs -o defaults
@@ -23,25 +38,25 @@ mount_overlay
 # Check if there are any issues accessing the files in /etc after mounting the
 # overlay by doing an 'ls' command
 error="/var/overlay-error"
+recreate_overlay_done=
 cd /var/persist/etc/
-for i in *; do
-  # Only check regular files at the top of the directory
-  if [[ -f $i ]]; then
+files=$(find . -type f)
+for i in $files; do
+  ls -i /etc/$i >/dev/null 2>${error};
+  if [[ -s ${error} ]]; then
+    # We don't have a way to print this error to the journal, delete it
+    rm -f ${error}
+    if test -n "$recreate_overlay_done"; then
+      recreate_overlay
+      recreate_overlay_done="true"
+    fi
+    # Check file once more
     ls -i /etc/$i >/dev/null 2>${error};
     if [[ -s ${error} ]]; then
-      # We don't have a way to print this error to the journal, delete it
+      # File still corrupted, delete it from the overlay
+      echo "Removing corrupted file from overlay: $i"
       rm -f ${error}
-      # Attempt to re-create the overlay by moving out the overlay contents and
-      # copying them back to /etc, which would create them back in the overlay
-      cd
-      umount /etc
-      rm -rf /var/persist/etc-save
-      mv /var/persist/etc /var/persist/etc-save
-      mkdir -p /var/persist/etc
-      mount_overlay
-      cp -rp /var/persist/etc-save/* /etc/
-      rm -rf /var/persist/etc-save
-      break
+      rm -f /var/persist/etc/$i
     fi
   fi
 done