subtree updates

meta-openembedded: f22bf6efaa..6792ebdd96:
  Armin Kuster (1):
        Revert "c-ares: Add fix for CVE-2021-3672"

  Hitendra Prajapati (1):
        python3-lxml: CVE-2022-2309 NULL Pointer Dereference allows attackers to cause a denial of service

  Khem Raj (1):
        c-ares: Upgrade to 1.17.1 release

  Ranjitsinh Rathod (1):
        nodejs: Upgrade to 12.22.12

  Sinan Kaya (1):
        c-ares: remove custom patches

  Yi Zhao (1):
        cryptsetup: upgrade 2.3.2 -> 2.3.7

  wangmy (2):
        c-ares: upgrade 1.17.1 -> 1.17.2
        c-ares: upgrade 1.17.2 -> 1.18.1

poky: 4aad5914ef..90a6f6a110:
  Alexander Kanavin (3):
        mobile-broadband-provider-info: upgrade 20220511 -> 20220725
        tzdata: upgrade 2022a -> 2022b
        wireless-regdb: upgrade 2022.06.06 -> 2022.08.12

  Andrei Gherzan (1):
        qemu: Define libnfs PACKAGECONFIG

  Anuj Mittal (1):
        cryptodev-module: fix build with 5.11+ kernels

  Aryaman Gupta (2):
        bitbake: bitbake: runqueue: add cpu/io pressure regulation
        bitbake: bitbake: runqueue: add memory pressure regulation

  Bruce Ashfield (3):
        linux-yocto/5.4: update to v5.4.210
        linux-yocto/5.4: update to v5.4.212
        linux-yocto/5.4: update to v5.4.213

  Chee Yang Lee (6):
        connman: fix CVE-2022-32292
        gnutls: fix CVE-2021-4209
        virglrenderer: fix CVE-2022-0135
        gst-plugins-good: fix several CVE
        go: fix and ignore several CVEs
        qemu: fix and ignore several CVEs

  Dmitry Baryshkov (3):
        linux-firmware: upgrade 20220708 -> 20220913
        linux-firmware: package new Qualcomm firmware
        linux-firmware: package new Qualcomm firmware

  Ernst Sjöstrand (1):
        cve-check: Don't use f-strings

  Florin Diaconescu (1):
        binutils : CVE-2022-38533

  Hitendra Prajapati (9):
        libtiff: CVE-2022-34526 A stack overflow was discovered
        golang: fix CVE-2022-30629 and CVE-2022-30631
        golang: fix CVE-2022-30632 and CVE-2022-30633
        golang: fix CVE-2022-30635 and CVE-2022-32148
        golang: CVE-2022-32189 a denial of service
        sqlite: CVE-2022-35737 assertion failure
        connman: CVE-2022-32293 man-in-the-middle attack against a WISPR HTTP
        bluez: CVE-2022-39176 BlueZ allows physically proximate attackers
        golang: CVE-2022-27664 net/http: handle server errors after sending GOAWAY

  Jon Mason (1):
        ref-manual: add numa to machine features

  Joshua Watt (2):
        bitbake: utils: Pass lock argument in fileslocked
        classes: cve-check: Get shared database lock

  Khan@kpit.com (1):
        python3: Fix CVE-2021-28861 for python3

  Lee Chee Yang (1):
        subversion: fix CVE-2021-28544

  Martin Jansa (1):
        create-pull-request: don't switch the git remote protocol to git://

  Mathieu Dubois-Briand (1):
        bind: Fix CVEs 2022-2795, 2022-38177, 2022-38178

  Michael Opdenacker (1):
        dev-manual: fix reference to BitBake user manual

  Minjae Kim (1):
        inetutils: CVE-2022-39028 - fix remote DoS vulnerability in inetutils-telnetd

  Paul Barker (1):
        licenses: Handle newer SPDX license names

  Paul Eggleton (1):
        relocate_sdk.py: ensure interpreter size error causes relocation to fail

  Pawan Badganchi (1):
        libxml2: Add fix for CVE-2016-3709

  Rajesh Dangi (1):
        linux-yocto/5.4: update genericx86* machines to v5.4.205

  Ranjitsinh Rathod (2):
        libarchive: Fix CVE-2021-23177 issue
        libarchive: Fix CVE-2021-31566 issue

  Richard Purdie (8):
        bitbake: runqueue: Change pressure file warning to a note
        vim: Upgrade 9.0.0115 -> 9.0.0242
        vim: Upgrade 9.0.0242 -> 9.0.0341
        vim: Upgrade 9.0.0341 -> 9.0.0453
        qemu: Add PACKAGECONFIG for brlapi
        vim: Upgrade 9.0.453 -> 9.0.541
        vim: Upgrade 9.0.0541 -> 9.0.0598
        build-appliance-image: Update to dunfell head revision

  Robert Joslyn (2):
        curl: Backport patch for CVE-2022-35252
        tzdata: Update from 2022b to 2022c

  Ross Burton (1):
        cve-check: close cursors as soon as possible

  Sana Kazi (1):
        sqlite3: Fix CVE-2021-20223

  Shubham Kulkarni (1):
        go: Add fix for CVE-2022-32190

  Steve Sakoman (2):
        documentation: update for 3.1.20
        poky.conf: bump version for 3.1.20 release

  Virendra Thakur (4):
        tiff: Fix for CVE-2022-2867/8/9
        sqlite3: Fix CVE-2020-35525
        sqlite3: Fix CVE-2020-35527
        expat: Fix CVE-2022-40674

  Yi Zhao (1):
        tiff: Security fixes CVE-2022-1354 and CVE-2022-1355

  niko.mauno@vaisala.com (2):
        systemd: Fix unwritable /var/lock when no sysvinit handling
        systemd: Add 'no-dns-fallback' PACKAGECONFIG option

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I90e4979b6917a013eca89c594677a940ccfac5fc

1 files changed, 82 insertions, 0 deletions

diff --git a/poky/bitbake/lib/bb/runqueue.py b/poky/bitbake/lib/bb/runqueue.py
index a513b0983b..6cdc72a85b 100644
--- a/poky/bitbake/lib/bb/runqueue.py
+++ b/poky/bitbake/lib/bb/runqueue.py
@@ -24,6 +24,7 @@ import pickle
 from multiprocessing import Process
 import shlex
 import pprint
+import time
 
 bblogger = logging.getLogger("BitBake")
 logger = logging.getLogger("BitBake.RunQueue")
@@ -142,6 +143,55 @@ class RunQueueScheduler(object):
                 self.buildable.append(tid)
 
         self.rev_prio_map = None
+        self.is_pressure_usable()
+
+    def is_pressure_usable(self):
+        """
+        If monitoring pressure, return True if pressure files can be open and read. For example
+        openSUSE /proc/pressure/* files have readable file permissions but when read the error EOPNOTSUPP (Operation not supported)
+        is returned.
+        """
+        if self.rq.max_cpu_pressure or self.rq.max_io_pressure or self.rq.max_memory_pressure:
+            try:
+                with open("/proc/pressure/cpu") as cpu_pressure_fds, \
+                    open("/proc/pressure/io") as io_pressure_fds, \
+                    open("/proc/pressure/memory") as memory_pressure_fds:
+
+                    self.prev_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1]
+                    self.prev_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1]
+                    self.prev_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1]
+                    self.prev_pressure_time = time.time()
+                self.check_pressure = True
+            except:
+                bb.note("The /proc/pressure files can't be read. Continuing build without monitoring pressure")
+                self.check_pressure = False
+        else:
+            self.check_pressure = False
+
+    def exceeds_max_pressure(self):
+        """
+        Monitor the difference in total pressure at least once per second, if
+        BB_PRESSURE_MAX_{CPU|IO|MEMORY} are set, return True if above threshold.
+        """
+        if self.check_pressure:
+            with open("/proc/pressure/cpu") as cpu_pressure_fds, \
+                open("/proc/pressure/io") as io_pressure_fds, \
+                open("/proc/pressure/memory") as memory_pressure_fds:
+                # extract "total" from /proc/pressure/{cpu|io}
+                curr_cpu_pressure = cpu_pressure_fds.readline().split()[4].split("=")[1]
+                curr_io_pressure = io_pressure_fds.readline().split()[4].split("=")[1]
+                curr_memory_pressure = memory_pressure_fds.readline().split()[4].split("=")[1]
+                exceeds_cpu_pressure =  self.rq.max_cpu_pressure and (float(curr_cpu_pressure) - float(self.prev_cpu_pressure)) > self.rq.max_cpu_pressure
+                exceeds_io_pressure =  self.rq.max_io_pressure and (float(curr_io_pressure) - float(self.prev_io_pressure)) > self.rq.max_io_pressure
+                exceeds_memory_pressure = self.rq.max_memory_pressure and (float(curr_memory_pressure) - float(self.prev_memory_pressure)) > self.rq.max_memory_pressure
+                now = time.time()
+                if now - self.prev_pressure_time > 1.0:
+                    self.prev_cpu_pressure = curr_cpu_pressure
+                    self.prev_io_pressure = curr_io_pressure
+                    self.prev_memory_pressure = curr_memory_pressure
+                    self.prev_pressure_time = now
+            return (exceeds_cpu_pressure or exceeds_io_pressure or exceeds_memory_pressure)
+        return False
 
     def next_buildable_task(self):
         """
@@ -155,6 +205,12 @@ class RunQueueScheduler(object):
         if not buildable:
             return None
 
+        # Bitbake requires that at least one task be active. Only check for pressure if
+        # this is the case, otherwise the pressure limitation could result in no tasks
+        # being active and no new tasks started thereby, at times, breaking the scheduler.
+        if self.rq.stats.active and self.exceeds_max_pressure():
+            return None
+
         # Filter out tasks that have a max number of threads that have been exceeded
         skip_buildable = {}
         for running in self.rq.runq_running.difference(self.rq.runq_complete):
@@ -1700,6 +1756,9 @@ class RunQueueExecute:
 
         self.number_tasks = int(self.cfgData.getVar("BB_NUMBER_THREADS") or 1)
         self.scheduler = self.cfgData.getVar("BB_SCHEDULER") or "speed"
+        self.max_cpu_pressure = self.cfgData.getVar("BB_PRESSURE_MAX_CPU")
+        self.max_io_pressure = self.cfgData.getVar("BB_PRESSURE_MAX_IO")
+        self.max_memory_pressure = self.cfgData.getVar("BB_PRESSURE_MAX_MEMORY")
 
         self.sq_buildable = set()
         self.sq_running = set()
@@ -1735,6 +1794,29 @@ class RunQueueExecute:
         if self.number_tasks <= 0:
              bb.fatal("Invalid BB_NUMBER_THREADS %s" % self.number_tasks)
 
+        lower_limit = 1.0
+        upper_limit = 1000000.0
+        if self.max_cpu_pressure:
+            self.max_cpu_pressure = float(self.max_cpu_pressure)
+            if self.max_cpu_pressure < lower_limit:
+                bb.fatal("Invalid BB_PRESSURE_MAX_CPU %s, minimum value is %s." % (self.max_cpu_pressure, lower_limit))
+            if self.max_cpu_pressure > upper_limit:
+                bb.warn("Your build will be largely unregulated since BB_PRESSURE_MAX_CPU is set to %s. It is very unlikely that such high pressure will be experienced." % (self.max_cpu_pressure))
+
+        if self.max_io_pressure:
+            self.max_io_pressure = float(self.max_io_pressure)
+            if self.max_io_pressure < lower_limit:
+                bb.fatal("Invalid BB_PRESSURE_MAX_IO %s, minimum value is %s." % (self.max_io_pressure, lower_limit))
+            if self.max_io_pressure > upper_limit:
+                bb.warn("Your build will be largely unregulated since BB_PRESSURE_MAX_IO is set to %s. It is very unlikely that such high pressure will be experienced." % (self.max_io_pressure))
+
+        if self.max_memory_pressure:
+            self.max_memory_pressure = float(self.max_memory_pressure)
+            if self.max_memory_pressure < lower_limit:
+                bb.fatal("Invalid BB_PRESSURE_MAX_MEMORY %s, minimum value is %s." % (self.max_memory_pressure, lower_limit))
+            if self.max_memory_pressure > upper_limit:
+                bb.warn("Your build will be largely unregulated since BB_PRESSURE_MAX_MEMORY is set to %s. It is very unlikely that such high pressure will be experienced." % (self.max_io_pressure))
+            
         # List of setscene tasks which we've covered
         self.scenequeue_covered = set()
         # List of tasks which are covered (including setscene ones)