meta-ampere: Support software workaround for PSU redundancy issue

When system only uses single PSU ( ex : PSU1 ) to power ON normal 12V,
HSC2 will be trigged Fault event (FET health).
At this time, plugging in PSU2 in system, the PSU2 won't deliver power to
+12V_MB because HSC2 is protected by Fault event.
Due to HSC2 protected, the PSU redundancy mechanism can't be created.
Once PSU1 is plugged out at this moment, system will crash ( reset )
because +12V_MB dropped.
Support detecting PSU plug event and reset the Hot-Swap feature by
disabling and then enabling again it through PMBUS command to clear the
event.

Tested:
1. Plug only PSU1 to power ON system (12V)
2. Wait until host is booted, check POUT:
    $ ipmitool  sdr | grep "POUT"
    PSU1_POUT        | 112 Watts         | ok
    PSU2_POUT        | 0 Watts           | ok
3. Plug in PSU2 in system, check POUT:
   $ ipmitool sdr | grep "POUT"
   PSU1_POUT        | 64 Watts          | ok
   PSU2_POUT        | 48 Watts          | ok
4. Unplug AC power PSU1, check POUT:
   $ ipmitool sdr | grep "POUT"
   PSU1_POUT        | 0 Watts           | ok
   PSU2_POUT        | 128 Watts         | ok
5. Plug in AC power PSU1, check POUT:
   $ ipmitool sdr | grep "POUT"
   PSU1_POUT        | 80 Watts          | ok
   PSU2_POUT        | 64 Watts          | ok
6. Repeat 1. to 5. with PSU2.

Signed-off-by: Chanh Nguyen <chanh@os.amperecomputing.com>
Change-Id: Ia6e00cd7b08de48059f2450e7eaf108418d0a026

5 files changed, 94 insertions, 0 deletions

diff --git a/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/ampere_psu_reset_hotswap.sh b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/ampere_psu_reset_hotswap.sh
new file mode 100644
index 0000000000..65bff9dc16
--- /dev/null
+++ b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/ampere_psu_reset_hotswap.sh
@@ -0,0 +1,74 @@
+#!/bin/sh
+
+# When system only use single PSU ( ex : PSU1 ) to power ON normal 12V,
+# HSC2 will be trigged Fault event (FET health).
+# At this time, to plug-in PSU2 in system, PSU2 won't deliver power to
+# +12V_MB because HSC2 is protected by Fault event.
+# Due to HSC2 protected, the PSU redundancy mechanism can't be created.
+# Once PSU1 is plugged out at this moment, system will crash ( reset )
+# because +12V_MB dropped.
+
+# BMC SW work-around solution:
+# - When BMC detect event PSU is plugged in system, BMC will reset HSC
+# by disbale HOT SWAP and then enable HOT SWAP through pmbus command to clear
+# Fault event.
+
+# Note:
+# In case hot swap occurs during BMC reset, BMC still not in operation state,
+# BMC can't detect PSU plug/unplug, then the work-around won't be executed
+
+# Author: Chanh Nguyen <chnguyen@amperecomputing.com>
+
+HSC1_PMBUS_NUM=10
+HSC2_PMBUS_NUM=10
+HSC1_SLAVE_ADDR=0x10
+HSC2_SLAVE_ADDR=0x11
+OPERATION=0x01
+STATUS_MFR_SPECIFIC=0x80
+
+# $1 will be the name of the psu
+PSU=$1
+
+if [[ $PSU == 1 ]]; then
+	HSC_PMBUS_NUM=$HSC1_PMBUS_NUM
+	HSC_SLAVE_ADDR=$HSC1_SLAVE_ADDR
+elif [[ $PSU == 2 ]]; then
+	HSC_PMBUS_NUM=$HSC2_PMBUS_NUM
+	HSC_SLAVE_ADDR=$HSC2_SLAVE_ADDR
+else
+	echo "Please choose PSU1 (1) or PSU2 (2)"
+	echo "Ex: ampere_psu_reset_hotswap.sh 1"
+	exit 0
+fi
+
+# Check HOST state
+chassisstate=$(obmcutil chassisstate | awk -F. '{print $NF}')
+if [[ "$chassisstate" == 'Off' ]]; then
+	echo "HOST is being OFF, so can't access the i2c $HSC_PMBUS_NUM. Please Turn ON HOST !"
+	exit 1
+fi
+
+# Check FET health problems
+data=$(i2cget -f -y $HSC_PMBUS_NUM $HSC_SLAVE_ADDR $STATUS_MFR_SPECIFIC)
+
+if [[ $? -ne 0 ]]; then
+	echo "ERROR: Can't access the i2c. Please check /dev/i2c-$HSC_PMBUS_NUM"
+	exit 1
+fi
+
+psu_sts=$(((data & 0x80) != 0))
+
+if [[ $psu_sts == 1 ]]; then
+	echo "PSU $PSU: FET health problems have been detected"
+	echo "Reset Hot swap output on PSU $PSU"
+	# Disable Hot swap output
+	write_data=0x00
+	i2cset -f -y $HSC_PMBUS_NUM $HSC_SLAVE_ADDR $OPERATION $write_data b
+
+	# Enable Hot swap output
+	write_data=0x80;
+	i2cset -f -y $HSC_PMBUS_NUM $HSC_SLAVE_ADDR $OPERATION $write_data b
+
+else
+	echo "PSU $PSU: FET health problems have not been detected"
+fi
\ No newline at end of file
diff --git a/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/obmc/gpio/PSU1_B25 b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/obmc/gpio/PSU1_B25
new file mode 100644
index 0000000000..b0ce1a83e2
--- /dev/null
+++ b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/obmc/gpio/PSU1_B25
@@ -0,0 +1,4 @@
+DEVPATH=/dev/input/by-path/platform-gpio-keys-event
+KEY=60
+POLARITY=0
+TARGET=psu1_hotswap_reset.service
diff --git a/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/obmc/gpio/PSU2_B25 b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/obmc/gpio/PSU2_B25
new file mode 100644
index 0000000000..ac50f53753
--- /dev/null
+++ b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/obmc/gpio/PSU2_B25
@@ -0,0 +1,4 @@
+DEVPATH=/dev/input/by-path/platform-gpio-keys-event
+KEY=61
+POLARITY=0
+TARGET=psu2_hotswap_reset.service
diff --git a/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/psu1_hotswap_reset.service b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/psu1_hotswap_reset.service
new file mode 100644
index 0000000000..fec3ad6a2b
--- /dev/null
+++ b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/psu1_hotswap_reset.service
@@ -0,0 +1,6 @@
+[Unit]
+Description=Ampere Computing LLC PSU1 reset hotswap
+
+[Service]
+Restart=no
+ExecStart=ampere_psu_reset_hotswap.sh 1
diff --git a/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/psu2_hotswap_reset.service b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/psu2_hotswap_reset.service
new file mode 100644
index 0000000000..fdb8692a5b
--- /dev/null
+++ b/meta-ampere/meta-jade/recipes-phosphor/gpio/psu-hotswap-reset/psu2_hotswap_reset.service
@@ -0,0 +1,6 @@
+[Unit]
+Description=Ampere Computing LLC PSU2 reset hotswap
+
+[Service]
+Restart=no
+ExecStart=ampere_psu_reset_hotswap.sh 2