# git rev-parse -q --verify e1c10e7c7d8e863689e0239ca027b2a2e1a5f39a^{commit} e1c10e7c7d8e863689e0239ca027b2a2e1a5f39a already have revision, skipping fetch # git checkout -q -f -B kisskb e1c10e7c7d8e863689e0239ca027b2a2e1a5f39a # git clean -qxdf # git log -1 commit e1c10e7c7d8e863689e0239ca027b2a2e1a5f39a Author: Madhavan Srinivasan Date: Sun Aug 20 23:28:31 2017 +0530 powerpc: rewrite local_t using soft_irq Local atomic operations are fast and highly reentrant per CPU counters. Used for percpu variable updates. Local atomic operations only guarantee variable modification atomicity wrt the CPU which owns the data and these needs to be executed in a preemption safe way. Here is the design of this patch. Since local_* operations are only need to be atomic to interrupts (IIUC), we have two options. Either replay the "op" if interrupted or replay the interrupt after the "op". Initial patchset posted was based on implementing local_* operation based on CR5 which replay's the "op". Patchset had issues in case of rewinding the address pointor from an array. This make the slow path really slow. Since CR5 based implementation proposed using __ex_table to find the rewind addressr, this rasied concerns about size of __ex_table and vmlinux. https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-December/123115.html But this patch uses, powerpc_local_irq_pmu_save to soft_disable interrupts (including PMIs). After finishing the "op", powerpc_local_irq_pmu_restore() called and correspondingly interrupts are replayed if any occured. patch re-write the current local_* functions to use arch_local_irq_disbale. Base flow for each function is { powerpc_local_irq_pmu_save(flags) load .. store powerpc_local_irq_pmu_restore(flags) } Reason for the approach is that, currently l[w/d]arx/st[w/d]cx. instruction pair is used for local_* operations, which are heavy on cycle count and they dont support a local variant. So to see whether the new implementation helps, used a modified version of Rusty's benchmark code on local_t. https://lkml.org/lkml/2008/12/16/450 Modifications to Rusty's benchmark code: - Executed only local_t test Here are the values with the patch. Time in ns per iteration Local_t Without Patch With Patch _inc 38 10 _add 38 10 _read 4 4 _add_return 38 10 Currently only asm/local.h has been rewritten, and also the entire change is tested only in PPC64 (pseries guest) and PPC64 host (LE) Patch convert the inline asm implementation of the local_t to C. Also removed the local_dec_if_positive since there are no users for this. Reviewed-by: Nicholas Piggin Signed-off-by: Madhavan Srinivasan Signed-off-by: Michael Ellerman # < /opt/cross/kisskb/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux-gcc --version # < git log --format=%s --max-count=1 e1c10e7c7d8e863689e0239ca027b2a2e1a5f39a # < make -j 48 ARCH=powerpc O=/home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc CROSS_COMPILE=/opt/cross/kisskb/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux- randconfig KCONFIG_SEED=0xF4684890 # Added to kconfig CONFIG_STANDALONE=y # Added to kconfig CONFIG_BUILD_DOCSRC=n # Added to kconfig CONFIG_MODULE_SIG=n # Added to kconfig CONFIG_CPU_BIG_ENDIAN=y # Added to kconfig CONFIG_PPC64=y # Added to kconfig CONFIG_PPC_DISABLE_WERROR=y # Added to kconfig CONFIG_SECTION_MISMATCH_WARN_ONLY=y # Added to kconfig CONFIG_PREVENT_FIRMWARE_BUILD=y # Added to kconfig CONFIG_CC_STACKPROTECTOR_STRONG=n # Added to kconfig CONFIG_GCC_PLUGINS=n # Added to kconfig CONFIG_LD_HEAD_STUB_CATCH=y # Added to kconfig # yes \n | make -j 48 ARCH=powerpc O=/home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc CROSS_COMPILE=/opt/cross/kisskb/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux- oldconfig yes: standard output: Broken pipe yes: write error # make -j 48 ARCH=powerpc O=/home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc CROSS_COMPILE=/opt/cross/kisskb/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux- make[1]: Entering directory '/home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc' GEN ./Makefile scripts/kconfig/conf --silentoldconfig Kconfig CHK include/config/kernel.release GEN ./Makefile WRAP arch/powerpc/include/generated/uapi/asm/param.h WRAP arch/powerpc/include/generated/uapi/asm/poll.h WRAP arch/powerpc/include/generated/uapi/asm/resource.h WRAP arch/powerpc/include/generated/uapi/asm/sockios.h WRAP arch/powerpc/include/generated/uapi/asm/statfs.h CHK include/generated/uapi/linux/version.h UPD include/generated/uapi/linux/version.h HOSTCC scripts/basic/bin2c UPD include/config/kernel.release Using /home/kisskb/slave/src as source for kernel WRAP arch/powerpc/include/generated/asm/clkdev.h WRAP arch/powerpc/include/generated/asm/div64.h WRAP arch/powerpc/include/generated/asm/export.h WRAP arch/powerpc/include/generated/asm/irq_regs.h WRAP arch/powerpc/include/generated/asm/irq_work.h WRAP arch/powerpc/include/generated/asm/local64.h WRAP arch/powerpc/include/generated/asm/mcs_spinlock.h WRAP arch/powerpc/include/generated/asm/preempt.h WRAP arch/powerpc/include/generated/asm/rwsem.h WRAP arch/powerpc/include/generated/asm/vtime.h WRAP arch/powerpc/include/generated/asm/msi.h CHK include/generated/utsrelease.h UPD include/generated/utsrelease.h HOSTCC scripts/kallsyms HOSTCC scripts/sortextable HOSTCC scripts/extract-cert HOSTCC scripts/insert-sys-cert HOSTCC scripts/asn1_compiler HOSTCC scripts/genksyms/genksyms.o SHIPPED scripts/genksyms/parse.tab.c HOSTCC scripts/dtc/dtc.o SHIPPED scripts/genksyms/lex.lex.c SHIPPED scripts/genksyms/keywords.hash.c CC scripts/mod/empty.o HOSTCC scripts/mod/mk_elfconfig HOSTCC scripts/genksyms/parse.tab.o HOSTCC scripts/dtc/flattree.o HOSTCC scripts/dtc/treesource.o CC scripts/mod/devicetable-offsets.s HOSTCC scripts/dtc/srcpos.o SHIPPED scripts/genksyms/parse.tab.h HOSTCC scripts/dtc/fstree.o HOSTCC scripts/dtc/checks.o HOSTCC scripts/dtc/livetree.o SHIPPED scripts/dtc/dtc-lexer.lex.c HOSTCC scripts/genksyms/lex.lex.o HOSTCC scripts/dtc/util.o SHIPPED scripts/dtc/dtc-parser.tab.h HOSTCC scripts/dtc/data.o HOSTCC scripts/dtc/dtc-lexer.lex.o SHIPPED scripts/dtc/dtc-parser.tab.c HOSTCC scripts/dtc/dtc-parser.tab.o MKELF scripts/mod/elfconfig.h HOSTCC scripts/mod/sumversion.o HOSTCC scripts/mod/modpost.o CHK include/generated/timeconst.h UPD include/generated/timeconst.h CC kernel/bounds.s CHK scripts/mod/devicetable-offsets.h UPD scripts/mod/devicetable-offsets.h HOSTCC scripts/mod/file2alias.o CHK include/generated/bounds.h UPD include/generated/bounds.h CC arch/powerpc/kernel/asm-offsets.s HOSTLD scripts/dtc/dtc HOSTLD scripts/genksyms/genksyms CHK include/generated/asm-offsets.h UPD include/generated/asm-offsets.h CALL /home/kisskb/slave/src/scripts/checksyscalls.sh HOSTLD scripts/mod/modpost CC init/main.o HOSTCC usr/gen_init_cpio CHK include/generated/compile.h CC init/do_mounts.o CC init/do_mounts_initrd.o CC init/init_task.o CC init/initramfs.o UPD include/generated/compile.h CC init/version.o CC arch/powerpc/mm/mem.o CC arch/powerpc/mm/fault.o CC arch/powerpc/mm/mmap.o CC arch/powerpc/mm/pgtable.o CC arch/powerpc/lib/alloc.o AS arch/powerpc/lib/string.o CC arch/powerpc/mm/pgtable-hash64.o CC arch/powerpc/lib/feature-fixups.o CC arch/powerpc/mm/init-common.o AS arch/powerpc/mm/slb_low.o CC arch/powerpc/mm/slb.o AS arch/powerpc/lib/feature-fixups-test.o CC arch/powerpc/mm/hash_native_64.o CC arch/powerpc/mm/mmu_context_book3s64.o CC arch/powerpc/mm/hash_utils_64.o CC arch/powerpc/mm/pgtable_64.o AS arch/powerpc/lib/checksum_64.o CC arch/powerpc/mm/init_64.o CC arch/powerpc/lib/code-patching.o CC arch/powerpc/lib/sstep.o CC arch/powerpc/lib/checksum_wrappers.o AS arch/powerpc/lib/ldstfp.o CC arch/powerpc/mm/hash64_4k.o CC arch/powerpc/mm/slice.o CC arch/powerpc/mm/copro_fault.o CC arch/powerpc/mm/hugetlbpage-hash64.o CC arch/powerpc/mm/tlb_hash64.o CC arch/powerpc/mm/pgtable-book3s64.o CC arch/powerpc/mm/hugetlbpage.o CC arch/powerpc/mm/hugepage-hash64.o CC arch/powerpc/mm/mmu_context_iommu.o AR arch/powerpc/math-emu/built-in.o CC arch/powerpc/sysdev/mpic.o CC arch/powerpc/sysdev/mpic_msi.o CC arch/powerpc/platforms/maple/setup.o CC arch/powerpc/sysdev/mpic_u3msi.o CC arch/powerpc/platforms/cell/spu_callbacks.o CC arch/powerpc/platforms/pasemi/setup.o AS arch/powerpc/crypto/md5-asm.o CC arch/powerpc/sysdev/mpic_msgr.o CC arch/powerpc/sysdev/msi_bitmap.o CC arch/powerpc/platforms/maple/pci.o CC arch/powerpc/platforms/maple/time.o AR arch/powerpc/net/built-in.o CC arch/powerpc/crypto/md5-glue.o GEN usr/initramfs_data.cpio CC arch/powerpc/platforms/ps3/setup.o CC arch/powerpc/platforms/ps3/mm.o CC arch/powerpc/platforms/pseries/lpar.o AS arch/powerpc/platforms/pseries/hvCall.o CC arch/powerpc/lib/xor_vmx.o CC arch/powerpc/xmon/xmon.o CC arch/powerpc/xmon/nonstdio.o CC arch/powerpc/platforms/ps3/time.o AS arch/powerpc/lib/copypage_64.o CC arch/powerpc/lib/xor_vmx_glue.o AS usr/initramfs_data.o CC arch/powerpc/kernel/cputable.o CC arch/powerpc/platforms/pseries/nvram.o CC arch/powerpc/platforms/powernv/setup.o AS arch/powerpc/lib/copyuser_64.o CC arch/powerpc/platforms/pseries/reconfig.o CC arch/powerpc/kernel/ptrace.o CC arch/powerpc/platforms/pseries/of_helpers.o AR usr/built-in.o AS arch/powerpc/lib/mem_64.o AS arch/powerpc/xmon/spr_access.o AS arch/powerpc/lib/hweight_64.o /home/kisskb/slave/src/arch/powerpc/crypto/md5-glue.c: In function 'ppc_md5_final': /home/kisskb/slave/src/arch/powerpc/crypto/md5-glue.c:111:1: internal compiler error: in change_address_1, at emit-rtl.c:1933 Please submit a full bug report, with preprocessed source if appropriate. See for instructions. /home/kisskb/slave/src/scripts/Makefile.build:302: recipe for target 'arch/powerpc/crypto/md5-glue.o' failed make[2]: *** [arch/powerpc/crypto/md5-glue.o] Error 1 /home/kisskb/slave/src/Makefile:1019: recipe for target 'arch/powerpc/crypto' failed make[1]: *** [arch/powerpc/crypto] Error 2 make[1]: *** Waiting for unfinished jobs.... AS arch/powerpc/lib/string_64.o AS arch/powerpc/lib/copyuser_power7.o CC arch/powerpc/kernel/syscalls.o AS arch/powerpc/lib/copypage_power7.o AS arch/powerpc/platforms/ps3/hvcall.o CC arch/powerpc/kvm/book3s_exports.o CC arch/powerpc/kvm/../../../virt/kvm/kvm_main.o CC arch/powerpc/platforms/pseries/setup.o CC arch/powerpc/kvm/../../../virt/kvm/eventfd.o CC arch/powerpc/platforms/ps3/htab.o AS arch/powerpc/lib/memcpy_power7.o AR init/mounts.o CC arch/powerpc/platforms/pseries/iommu.o CC arch/powerpc/kvm/../../../virt/kvm/vfio.o CC arch/powerpc/platforms/pseries/event_sources.o CC arch/powerpc/platforms/pseries/ras.o CC arch/powerpc/platforms/pseries/firmware.o AS arch/powerpc/lib/memcpy_64.o CC arch/powerpc/platforms/pasemi/pci.o AS arch/powerpc/lib/memcmp_64.o CC arch/powerpc/lib/locks.o CC arch/powerpc/platforms/cell/spu_base.o CC arch/powerpc/lib/vmx-helper.o CC arch/powerpc/platforms/pasemi/time.o AS arch/powerpc/lib/crtsavres.o CC arch/powerpc/platforms/pasemi/idle.o AS arch/powerpc/platforms/powernv/opal-wrappers.o CC arch/powerpc/platforms/powernv/opal.o CC arch/powerpc/platforms/pseries/power.o CC arch/powerpc/platforms/powernv/opal-async.o CC arch/powerpc/platforms/pseries/dlpar.o CC arch/powerpc/platforms/powernv/idle.o CC arch/powerpc/platforms/pseries/mobility.o CC arch/powerpc/platforms/powernv/opal-rtc.o CC arch/powerpc/platforms/powernv/opal-lpc.o CC arch/powerpc/platforms/powernv/opal-nvram.o AS arch/powerpc/platforms/pasemi/powersave.o CC arch/powerpc/platforms/pasemi/iommu.o CC arch/powerpc/platforms/powernv/opal-flash.o CC arch/powerpc/sysdev/dart_iommu.o CC arch/powerpc/kvm/../../../virt/kvm/coalesced_mmio.o AR init/built-in.o CC arch/powerpc/platforms/pseries/rng.o CC arch/powerpc/platforms/pseries/pci.o CC arch/powerpc/platforms/powernv/opal-elog.o CC arch/powerpc/platforms/pseries/pci_dlpar.o CC arch/powerpc/platforms/powernv/rng.o CC arch/powerpc/platforms/pseries/eeh_pseries.o CC arch/powerpc/sysdev/mmio_nvram.o CC arch/powerpc/platforms/pasemi/dma_lib.o CC arch/powerpc/platforms/pasemi/misc.o CC arch/powerpc/platforms/pasemi/msi.o CC arch/powerpc/kernel/irq.o CC arch/powerpc/platforms/pseries/msi.o CC arch/powerpc/platforms/cell/spu_notify.o CC arch/powerpc/platforms/cell/spu_syscalls.o CC arch/powerpc/platforms/pseries/smp.o CC arch/powerpc/platforms/pseries/kexec.o CC arch/powerpc/platforms/ps3/repository.o AR arch/powerpc/platforms/maple/built-in.o CC arch/powerpc/platforms/ps3/interrupt.o CC [M] arch/powerpc/platforms/cell/cpufreq_spudemand.o CC arch/powerpc/kvm/powerpc.o CC arch/powerpc/platforms/pseries/pseries_energy.o CC arch/powerpc/platforms/powernv/opal-dump.o CC arch/powerpc/sysdev/fsl_lbc.o CC arch/powerpc/platforms/pseries/hotplug-cpu.o CC arch/powerpc/platforms/powernv/opal-sysparam.o CC arch/powerpc/platforms/pseries/lparcfg.o CC arch/powerpc/platforms/pseries/vio.o AR arch/powerpc/mm/built-in.o CC arch/powerpc/platforms/pseries/ibmebus.o CC arch/powerpc/sysdev/rtc_cmos_setup.o CC arch/powerpc/sysdev/simple_gpio.o CC arch/powerpc/platforms/cell/spufs/file.o CC arch/powerpc/platforms/cell/spufs/inode.o CC arch/powerpc/platforms/ps3/exports.o CC arch/powerpc/platforms/cell/spufs/context.o CC arch/powerpc/platforms/powernv/opal-sensor.o CC arch/powerpc/platforms/powernv/opal-msglog.o /home/kisskb/slave/src/arch/powerpc/kvm/powerpc.c: In function 'kvmppc_swab_shared': /home/kisskb/slave/src/arch/powerpc/kvm/powerpc.c:155:1: internal compiler error: in change_address_1, at emit-rtl.c:1933 Please submit a full bug report, with preprocessed source if appropriate. See for instructions. CC arch/powerpc/kernel/align.o /home/kisskb/slave/src/scripts/Makefile.build:302: recipe for target 'arch/powerpc/kvm/powerpc.o' failed make[2]: *** [arch/powerpc/kvm/powerpc.o] Error 1 make[2]: *** Waiting for unfinished jobs.... CC arch/powerpc/sysdev/i8259.o CC arch/powerpc/kernel/signal_32.o CC arch/powerpc/sysdev/scom.o CC arch/powerpc/platforms/cell/spufs/syscalls.o CC arch/powerpc/platforms/powernv/opal-hmi.o CC arch/powerpc/platforms/cell/spufs/sched.o CC arch/powerpc/platforms/cell/spufs/backing_ops.o CC arch/powerpc/platforms/cell/spufs/hw_ops.o CC arch/powerpc/platforms/powernv/opal-power.o CC arch/powerpc/sysdev/xics/xics-common.o /home/kisskb/slave/src/arch/powerpc/lib/sstep.c: In function 'analyse_instr': /home/kisskb/slave/src/arch/powerpc/lib/sstep.c:1828:1: internal compiler error: in change_address_1, at emit-rtl.c:1933 Please submit a full bug report, with preprocessed source if appropriate. See for instructions. /home/kisskb/slave/src/scripts/Makefile.build:302: recipe for target 'arch/powerpc/lib/sstep.o' failed make[2]: *** [arch/powerpc/lib/sstep.o] Error 1 /home/kisskb/slave/src/Makefile:1019: recipe for target 'arch/powerpc/lib' failed make[1]: *** [arch/powerpc/lib] Error 2 CC arch/powerpc/sysdev/xics/icp-native.o CC arch/powerpc/sysdev/xics/icp-hv.o CC arch/powerpc/sysdev/xive/common.o CC arch/powerpc/sysdev/xive/native.o CC arch/powerpc/platforms/powernv/opal-irqchip.o CC arch/powerpc/sysdev/xics/ics-rtas.o CC arch/powerpc/platforms/powernv/opal-kmsg.o CC arch/powerpc/platforms/powernv/opal-powercap.o CC arch/powerpc/platforms/cell/spufs/run.o CC arch/powerpc/platforms/ps3/os-area.o CC arch/powerpc/platforms/powernv/opal-psr.o CC arch/powerpc/platforms/cell/spufs/gang.o SHIPPED arch/powerpc/platforms/cell/spufs/spu_save_dump.h SHIPPED arch/powerpc/platforms/cell/spufs/spu_restore_dump.h CC arch/powerpc/platforms/powernv/opal-sensor-groups.o CC arch/powerpc/platforms/cell/spufs/fault.o CC arch/powerpc/platforms/powernv/subcore.o CC arch/powerpc/platforms/ps3/system-bus.o CC arch/powerpc/platforms/powernv/smp.o AS arch/powerpc/platforms/powernv/subcore-asm.o CC arch/powerpc/sysdev/xics/ics-opal.o CC arch/powerpc/sysdev/xics/icp-opal.o CC arch/powerpc/platforms/cell/spufs/lscsa_alloc.o AR arch/powerpc/platforms/pasemi/built-in.o CC arch/powerpc/platforms/powernv/pci.o CC arch/powerpc/kernel/pmc.o CC arch/powerpc/platforms/cell/spufs/coredump.o CC arch/powerpc/platforms/powernv/pci-ioda.o CC arch/powerpc/kernel/vdso.o CC arch/powerpc/platforms/powernv/npu-dma.o CC arch/powerpc/kernel/process.o CPP arch/powerpc/kernel/systbl_chk.i CC arch/powerpc/platforms/cell/spufs/switch.o /home/kisskb/slave/src/Makefile:1019: recipe for target 'arch/powerpc/kvm' failed make[1]: *** [arch/powerpc/kvm] Error 2 CC arch/powerpc/platforms/powernv/pci-cxl.o CC arch/powerpc/platforms/ps3/smp.o CC arch/powerpc/platforms/powernv/eeh-powernv.o CC arch/powerpc/platforms/ps3/spu.o AR arch/powerpc/xmon/built-in.o CC arch/powerpc/kernel/idle.o /home/kisskb/slave/src/arch/powerpc/kernel/align.c: In function 'fix_alignment': /home/kisskb/slave/src/arch/powerpc/kernel/align.c:1068:1: internal compiler error: in change_address_1, at emit-rtl.c:1933 Please submit a full bug report, with preprocessed source if appropriate. See for instructions. CC arch/powerpc/platforms/ps3/device-init.o CC arch/powerpc/kernel/signal.o /home/kisskb/slave/src/scripts/Makefile.build:302: recipe for target 'arch/powerpc/kernel/align.o' failed make[2]: *** [arch/powerpc/kernel/align.o] Error 1 make[2]: *** Waiting for unfinished jobs.... CC arch/powerpc/platforms/powernv/opal-xscom.o CC arch/powerpc/platforms/powernv/opal-imc.o CC arch/powerpc/platforms/powernv/opal-prd.o AR arch/powerpc/platforms/pseries/built-in.o /home/kisskb/slave/src/arch/powerpc/platforms/powernv/opal-imc.c: In function 'disable_nest_pmu_counters': /home/kisskb/slave/src/arch/powerpc/platforms/powernv/opal-imc.c:130:13: warning: assignment discards 'const' qualifier from pointer target type [enabled by default] AR arch/powerpc/sysdev/xics/built-in.o AR arch/powerpc/platforms/ps3/built-in.o AR arch/powerpc/sysdev/xive/built-in.o AR arch/powerpc/sysdev/built-in.o /home/kisskb/slave/src/Makefile:1019: recipe for target 'arch/powerpc/kernel' failed make[1]: *** [arch/powerpc/kernel] Error 2 AR arch/powerpc/platforms/cell/spufs/spufs.o AR arch/powerpc/platforms/cell/spufs/built-in.o AR arch/powerpc/platforms/cell/built-in.o AR arch/powerpc/platforms/powernv/built-in.o AR arch/powerpc/platforms/built-in.o make[1]: Leaving directory '/home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc' Makefile:145: recipe for target 'sub-make' failed make: *** [sub-make] Error 2 Command 'make -j 48 ARCH=powerpc O=/home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc CROSS_COMPILE=/opt/cross/kisskb/gcc-4.6.3-nolibc/powerpc-linux/bin/powerpc-linux- ' returned non-zero exit status 2 # rm -rf /home/kisskb/slave/build/powerpc-next_powerpc-randconfig_powerpc # Build took: 0:00:20.642702