summaryrefslogtreecommitdiff
path: root/arch/mips/include/asm/div64.h
AgeCommit message (Collapse)AuthorFilesLines
2021-04-23MIPS: Avoid handcoded DIVU in `__div64_32' altogetherMaciej W. Rozycki1-6/+2
Remove the inline asm with a DIVU instruction from `__div64_32' and use plain C code for the intended DIVMOD calculation instead. GCC is smart enough to know that both the quotient and the remainder are calculated with single DIVU, so with ISAs up to R5 the same instruction is actually produced with overall similar code. For R6 compiled code will work, but separate DIVU and MODU instructions will be produced, which are also interlocked, so scalar implementations will likely not perform as well as older ISAs with their asynchronous MD unit. Likely still faster then the generic algorithm though. This removes a compilation error for R6 however where the original DIVU instruction is not supported anymore and the MDU accumulator registers have been removed and consequently GCC complains as to a constraint it cannot find a register for: In file included from ./include/linux/math.h:5, from ./include/linux/kernel.h:13, from mm/page-writeback.c:15: ./include/linux/math64.h: In function 'div_u64_rem': ./arch/mips/include/asm/div64.h:76:17: error: inconsistent operand constraints in an 'asm' 76 | __asm__("divu $0, %z1, %z2" \ | ^~~~~~~ ./include/asm-generic/div64.h:245:25: note: in expansion of macro '__div64_32' 245 | __rem = __div64_32(&(n), __base); \ | ^~~~~~~~~~ ./include/linux/math64.h:91:22: note: in expansion of macro 'do_div' 91 | *remainder = do_div(dividend, divisor); | ^~~~~~ This has passed correctness verification with test_div64 and reduced the module's average execution time down to 1.0404s from 1.0445s with R3400 @40MHz. The module's MIPS I machine code has also shrunk by 12 bytes or 3 instructions. Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk> Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
2021-04-21MIPS: Avoid DIVU in `__div64_32' is result would be zeroMaciej W. Rozycki1-2/+4
We already check the high part of the divident against zero to avoid the costly DIVU instruction in that case, needed to reduce the high part of the divident, so we may well check against the divisor instead and set the high part of the quotient to zero right away. We need to treat the high part the divident in that case though as the remainder that would be calculated by the DIVU instruction we avoided. This has passed correctness verification with test_div64 and reduced the module's average execution time down to 1.0445s and 0.2619s from 1.0668s and 0.2629s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz. Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk> Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
2021-04-21MIPS: Reinstate platform `__div64_32' handlerMaciej W. Rozycki1-16/+41
Our current MIPS platform `__div64_32' handler is inactive, because it is incorrectly only enabled for 64-bit configurations, for which generic `do_div' code does not call it anyway. The handler is not suitable for being called from there though as it only calculates 32 bits of the quotient under the assumption the 64-bit divident has been suitably reduced. Code for such reduction used to be there, however it has been incorrectly removed with commit c21004cd5b4c ("MIPS: Rewrite <asm/div64.h> to work with gcc 4.4.0."), which should have only updated an obsoleted constraint for an inline asm involving $hi and $lo register outputs, while possibly wiring the original MIPS variant of the `do_div' macro as `__div64_32' handler for the generic `do_div' implementation Correct the handler as follows then: - Revert most of the commit referred, however retaining the current formatting, except for the final two instructions of the inline asm sequence, which the original commit missed. Omit the original 64-bit parts though. - Rename the original `do_div' macro to `__div64_32'. Use the combined `x' constraint referring to the MD accumulator as a whole, replacing the original individual `h' and `l' constraints used for $hi and $lo registers respectively, of which `h' has been obsoleted with GCC 4.4. Update surrounding code accordingly. We have since removed support for GCC versions before 4.9, so no need for a special arrangement here; GCC has supported the `x' constraint since forever anyway, or at least going back to 1991. - Rename the `__base' local variable in `__div64_32' to `__radix' to avoid a conflict with a local variable in `do_div'. - Actually enable this code for 32-bit rather than 64-bit configurations by qualifying it with BITS_PER_LONG being 32 instead of 64. Include <asm/bitsperlong.h> for this macro rather than <linux/types.h> as we don't need anything else. - Finally include <asm-generic/div64.h> last rather than first. This has passed correctness verification with test_div64 and reduced the module's average execution time down to 1.0668s and 0.2629s from 2.1529s and 0.5647s respectively for an R3400 CPU @40MHz and a 5Kc CPU @160MHz. For a reference 64-bit `do_div' code where we have the DDIVU instruction available to do the whole calculation right away averages at 0.0660s for the latter CPU. Fixes: c21004cd5b4c ("MIPS: Rewrite <asm/div64.h> to work with gcc 4.4.0.") Reported-by: Huacai Chen <chenhuacai@kernel.org> Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk> Cc: stable@vger.kernel.org # v2.6.30+ Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
2009-05-14MIPS: Rewrite <asm/div64.h> to work with gcc 4.4.0.Ralf Baechle1-92/+50
The inline assembler used on 32-bit kernels was using the "h" constraint which was considered dangerous and removed for gcc 4.4.0. Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2008-10-11MIPS: Move headfiles to new location below arch/mips/includeRalf Baechle1-0/+110
Signed-off-by: Ralf Baechle <ralf@linux-mips.org>