renegade-project/linux-next
2
0
Fork 0
mirror of https://github.com/edk2-porting/linux-next.git synced 2024-12-22 12:14:01 +08:00
Code
e56b3bc794
linux-next/arch/x86/kernel
History
Linus Torvalds e56b3bc794 cpu masks: optimize and clean up cpumask_of_cpu() 
Clean up and optimize cpumask_of_cpu(), by sharing all the zero words.

Instead of stupidly generating all possible i=0...NR_CPUS 2^i patterns
creating a huge array of constant bitmasks, realize that the zero words
can be shared.

In other words, on a 64-bit architecture, we only ever need 64 of these
arrays - with a different bit set in one single world (with enough zero
words around it so that we can create any bitmask by just offsetting in
that big array). And then we just put enough zeroes around it that we
can point every single cpumask to be one of those things.

So when we have 4k CPU's, instead of having 4k arrays (of 4k bits each,
with one bit set in each array - 2MB memory total), we have exactly 64
arrays instead, each 8k bits in size (64kB total).

And then we just point cpumask(n) to the right position (which we can
calculate dynamically). Once we have the right arrays, getting
"cpumask(n)" ends up being:

  static inline const cpumask_t *get_cpu_mask(unsigned int cpu)
  {
          const unsigned long *p = cpu_bit_bitmap[1 + cpu % BITS_PER_LONG];
          p -= cpu / BITS_PER_LONG;
          return (const cpumask_t *)p;
  }

This brings other advantages and simplifications as well:

 - we are not wasting memory that is just filled with a single bit in
   various different places

 - we don't need all those games to re-create the arrays in some dense
   format, because they're already going to be dense enough.

if we compile a kernel for up to 4k CPU's, "wasting" that 64kB of memory
is a non-issue (especially since by doing this "overlapping" trick we
probably get better cache behaviour anyway).

[ mingo@elte.hu:

  Converted Linus's mails into a commit. See:

     http://lkml.org/lkml/2008/7/27/156
     http://lkml.org/lkml/2008/7/28/320

  Also applied a family filter - which also has the side-effect of leaving
  out the bits where Linus calls me an idio... Oh, never mind ;-)
]

Signed-off-by: Ingo Molnar <mingo@elte.hu>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Cc: Mike Travis <travis@sgi.com>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
2008-07-28 22:20:41 +02:00
..
acpi cpumask: change cpumask_of_cpu_ptr to use new cpumask_of_cpu 2008-07-26 16:40:33 +02:00
cpu cpumask: change cpumask_of_cpu_ptr to use new cpumask_of_cpu 2008-07-26 16:40:33 +02:00
.gitignore arch/x86/kernel/.gitignore: Added vmlinux.lds to .gitignore file because it shouldn't be tracked. 2008-07-10 10:13:51 -07:00
alternative.c x86: fix SMP alternatives: use mutex instead of spinlock, text_poke is sleepable 2008-05-23 21:56:52 +02:00
amd_iommu_init.c x86, AMD IOMMU: include amd_iommu_last_bdf in device initialization 2008-07-26 15:45:57 +02:00
amd_iommu.c x86, AMD IOMMU: include amd_iommu_last_bdf in device initialization 2008-07-26 15:45:57 +02:00
aperture_64.c x86: make only GART code include gart.h 2008-07-11 11:00:54 +02:00
apic_32.c Merge branches 'x86/urgent', 'x86/amd-iommu', 'x86/apic', 'x86/cleanups', 'x86/core', 'x86/cpu', 'x86/fixmap', 'x86/gart', 'x86/kprobes', 'x86/memtest', 'x86/modules', 'x86/nmi', 'x86/pat', 'x86/reboot', 'x86/setup', 'x86/step', 'x86/unify-pci', 'x86/uv', 'x86/xen' and 'xen-64bit' into x86/for-linus 2008-07-21 16:37:17 +02:00
apic_64.c Merge branches 'x86/urgent', 'x86/amd-iommu', 'x86/apic', 'x86/cleanups', 'x86/core', 'x86/cpu', 'x86/fixmap', 'x86/gart', 'x86/kprobes', 'x86/memtest', 'x86/modules', 'x86/nmi', 'x86/pat', 'x86/reboot', 'x86/setup', 'x86/step', 'x86/unify-pci', 'x86/uv', 'x86/xen' and 'xen-64bit' into x86/for-linus 2008-07-21 16:37:17 +02:00
apm_32.c remove include/linux/pm_legacy.h 2008-07-24 10:47:22 -07:00
asm-offsets_32.c x86/paravirt: split sysret and sysexit 2008-07-08 13:13:15 +02:00
asm-offsets_64.c xen64: add asm-offsets 2008-07-16 10:58:55 +02:00
asm-offsets.c
audit_64.c
bios_uv.c x86 BIOS interface for RTC on SGI UV 2008-07-18 14:35:14 +02:00
bootflag.c x86: coding style cleanup for kernel/bootflag.c 2008-01-30 13:32:31 +01:00
cpuid.c device create: x86: convert device_create to device_create_drvdata 2008-07-21 21:54:46 -07:00
crash_dump_32.c
crash_dump_64.c
crash.c x86: allow machine_crash_shutdown to be replaced 2008-04-27 12:00:29 +03:00
doublefault_32.c x86: unify tss_struct 2008-01-30 13:31:31 +01:00
ds.c x86: debug Store - call kfree if only we really need it 2008-04-17 17:41:34 +02:00
e820.c x86: introduce x86_quirks 2008-07-20 09:18:17 +02:00
early_printk.c x86, generic: mark early_printk as asmlinkage 2008-06-18 13:11:01 +02:00
early-quirks.c Merge branches 'x86/urgent', 'x86/amd-iommu', 'x86/apic', 'x86/cleanups', 'x86/core', 'x86/cpu', 'x86/fixmap', 'x86/gart', 'x86/kprobes', 'x86/memtest', 'x86/modules', 'x86/nmi', 'x86/pat', 'x86/reboot', 'x86/setup', 'x86/step', 'x86/unify-pci', 'x86/uv', 'x86/xen' and 'xen-64bit' into x86/for-linus 2008-07-21 16:37:17 +02:00
efi_32.c x86: fix Intel Mac booting with EFI 2008-07-03 08:19:18 +02:00
efi_64.c x86: add flags parameter to reserve_bootmem_generic() 2008-07-08 11:49:49 +02:00
efi_stub_32.S
efi_stub_64.S x86: EFI runtime service support 2008-01-30 13:31:19 +01:00
efi.c x86: introduce max_low_pfn_mapped for 64-bit 2008-07-11 10:24:04 +02:00
entry_32.S i386 syscall audit fast-path 2008-07-23 18:00:30 -07:00
entry_64.S x86_64 ia32 syscall audit fast-path 2008-07-23 17:55:22 -07:00
ftrace.c ftrace: store mcount address in rec->ip 2008-06-23 22:10:56 +02:00
genapic_64.c x86: introduce max_physical_apicid for bigsmp switching 2008-06-10 11:32:09 +02:00
genapic_flat_64.c NR_CPUS: Replace NR_CPUS in arch/x86/kernel/genapic_flat_64.c 2008-07-20 10:21:10 +02:00
genx2apic_uv_x.c Merge branch 'cpus4096-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-23 18:37:44 -07:00
geode_32.c x86, geode: add a VSA2 ID for General Software 2008-06-19 14:19:03 +02:00
head32.c x86: extract common part of head32.c and head64.c into head.c 2008-06-05 15:10:02 +02:00
head64.c x86_64: add workaround for no %gs-based percpu 2008-07-16 10:58:13 +02:00
head_32.S x86: fix cpu hotplug on 32bit 2008-07-27 21:43:11 +02:00
head_64.S Revert "x86_64: there's no need to preallocate level1_fixmap_pgt" 2008-07-16 11:07:30 +02:00
head.c x86: move reserve_setup_data to setup.c 2008-07-08 13:16:14 +02:00
hpet.c rtc-cmos: avoid spurious irqs 2008-07-24 10:47:34 -07:00
i386_ksyms_32.c ftrace: store mcount address in rec->ip 2008-06-23 22:10:56 +02:00
i387.c x86 ptrace: fix PTRACE_GETFPXREGS error 2008-07-01 11:03:31 +02:00
i8237.c Driver core: change sysdev classes to use dynamic kobject names 2008-01-24 20:40:40 -08:00
i8253.c x86: cleanup div_sc() usage 2008-04-26 17:35:47 +02:00
i8259.c i8259: fix final ugliness 2008-06-02 11:55:52 +02:00
init_task.c [PATCH] take init_files to fs/file.c 2008-05-16 17:22:20 -04:00
io_apic_32.c x86: I/O APIC: Always report how the timer has been set up 2008-07-18 14:27:47 +02:00
io_apic_64.c Merge branch 'cpus4096-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-23 18:37:44 -07:00
io_delay.c x86: io delay - add checking for NULL early param 2008-07-09 13:57:54 +02:00
ioport.c x86: refactor ioport unification 2008-01-30 13:33:10 +01:00
ipi.c x86: APIC: remove apic_write_around(); use alternatives 2008-07-18 12:51:21 +02:00
irq_32.c x86: use __page_aligned_data/bss 2008-07-16 10:54:39 +02:00
irq_64.c x86: make /proc/stat account for all interrupts 2008-05-25 07:11:49 +02:00
irqinit_32.c x86: rename the i8259_32/64.c leftovers to irqinit_32/64.c 2008-05-24 16:47:39 +02:00
irqinit_64.c x86-64: make BUILD_IRQ() also reset section back 2008-07-24 12:49:26 -07:00
k8.c
kdebugfs.c x86: Add a arch directory for x86 under debugfs 2008-07-18 17:22:04 -07:00
kgdb.c x86: KGDB build fix 2008-04-19 19:19:54 +02:00
kprobes.c kprobes: improve kretprobe scalability with hashed locking 2008-07-25 10:53:30 -07:00
kvm.c x86: KVM guest: hypercall batching 2008-04-27 12:00:28 +03:00
kvmclock.c x86: KVM guest: make kvm_smp_prepare_boot_cpu() static 2008-07-20 12:42:37 +03:00
ldt.c cpumask: change cpumask_of_cpu_ptr to use new cpumask_of_cpu 2008-07-26 16:40:33 +02:00
machine_kexec_32.c kexec jump: save/restore device state 2008-07-26 12:00:04 -07:00
machine_kexec_64.c kexec jump 2008-07-26 12:00:04 -07:00
Makefile Merge branch 'x86/paravirt-spinlocks' into x86/for-linus 2008-07-21 16:45:56 +02:00
mca_32.c x86: coding style fixes to arch/x86/kernel/mca_32.c 2008-04-17 17:40:49 +02:00
mfgpt_32.c geode: fix modular build 2008-06-12 21:25:51 +02:00
microcode.c cpumask: change cpumask_of_cpu_ptr to use new cpumask_of_cpu 2008-07-26 16:40:33 +02:00
mmconf-fam10h_64.c x86: move mmconfig declarations to header 2008-05-30 15:45:17 -07:00
module_32.c
module_64.c PAGE_ALIGN(): correctly handle 64-bit values on 32-bit architectures 2008-07-24 10:47:21 -07:00
mpparse.c x86: extend and use x86_quirks to clean up NUMAQ code 2008-07-20 09:25:52 +02:00
msr.c device create: x86: convert device_create to device_create_drvdata 2008-07-21 21:54:46 -07:00
nmi.c Merge branches 'x86/urgent', 'x86/amd-iommu', 'x86/apic', 'x86/cleanups', 'x86/core', 'x86/cpu', 'x86/fixmap', 'x86/gart', 'x86/kprobes', 'x86/memtest', 'x86/modules', 'x86/nmi', 'x86/pat', 'x86/reboot', 'x86/setup', 'x86/step', 'x86/unify-pci', 'x86/uv', 'x86/xen' and 'xen-64bit' into x86/for-linus 2008-07-21 16:37:17 +02:00
numaq_32.c x86: add ->pre_time_init to x86_quirks 2008-07-20 09:25:52 +02:00
olpc.c x86: olpc: add One Laptop Per Child architecture support 2008-04-29 08:06:07 -07:00
paravirt_patch_32.c x86/paravirt: split sysret and sysexit 2008-07-08 13:13:15 +02:00
paravirt_patch_64.c x86/paravirt: add sysret/sysexit pvops for returning to 32-bit compatibility userspace 2008-07-08 13:15:52 +02:00
paravirt.c x86: fix pte_flags() to only return flags, fix lguest (updated) 2008-07-22 10:41:18 +02:00
pci-calgary_64.c x86 calgary: fix handling of devices that aren't behind the Calgary 2008-07-26 12:00:03 -07:00
pci-dma.c dma-mapping: add the device argument to dma_mapping_error() 2008-07-26 12:00:03 -07:00
pci-gart_64.c dma-mapping: add the device argument to dma_mapping_error() 2008-07-26 12:00:03 -07:00
pci-nommu.c dma-mapping: add the device argument to dma_mapping_error() 2008-07-26 12:00:03 -07:00
pci-swiotlb_64.c dma-mapping: add the device argument to dma_mapping_error() 2008-07-26 12:00:03 -07:00
pcspeaker.c
pmtimer_64.c x86: move pmtmr related declarations 2008-01-30 13:30:18 +01:00
probe_roms_32.c x86: seperate probe_roms into another file 2008-07-08 12:50:05 +02:00
process_32.c Merge branch 'linus' into timers/nohz 2008-07-18 19:53:16 +02:00
process_64.c Merge branch 'timers-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-24 12:55:01 -07:00
process.c Merge branches 'x86/urgent', 'x86/amd-iommu', 'x86/apic', 'x86/cleanups', 'x86/core', 'x86/cpu', 'x86/fixmap', 'x86/gart', 'x86/kprobes', 'x86/memtest', 'x86/modules', 'x86/nmi', 'x86/pat', 'x86/reboot', 'x86/setup', 'x86/step', 'x86/unify-pci', 'x86/uv', 'x86/xen' and 'xen-64bit' into x86/for-linus 2008-07-21 16:37:17 +02:00
ptrace.c x86 ptrace: user-sets-TF nits 2008-07-16 12:15:17 -07:00
pvclock.c x86: Add structs and functions for paravirt clocksource 2008-06-24 21:02:31 +03:00
quirks.c Merge branch 'timers/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-15 10:39:57 -07:00
reboot_fixups_32.c x86: constify data in reboot.c 2008-05-25 08:58:30 +02:00
reboot.c cpumask: change cpumask_of_cpu_ptr to use new cpumask_of_cpu 2008-07-26 16:40:33 +02:00
relocate_kernel_32.S kexec jump 2008-07-26 12:00:04 -07:00
relocate_kernel_64.S x86: relocate_kernel - use predefined macroses for page attributes 2008-04-17 17:41:29 +02:00
rtc.c provide rtc_cmos platform device 2008-06-12 18:05:42 -07:00
scx200_32.c x86: fix sparse warning in kernel/scx200_32.c 2008-01-31 22:05:45 +01:00
setup_percpu.c cpu masks: optimize and clean up cpumask_of_cpu() 2008-07-28 22:20:41 +02:00
setup.c x86: call early_cpu_init at the same point 2008-07-24 12:28:11 +02:00
sigframe.h x86: move struct definitions to unifed sigframe.h 2008-04-17 17:40:46 +02:00
signal_32.c Merge branch 'sched/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-23 19:36:53 -07:00
signal_64.c x86-64: Clean up 'save/restore_i387()' usage 2008-07-24 16:12:40 -07:00
smp.c generic-ipi: fixlet 2008-07-06 14:01:50 +02:00
smpboot.c Merge branch 'cpus4096-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-23 18:37:44 -07:00
smpcommon.c x86: convert to generic helpers for IPI function calls 2008-06-26 11:21:54 +02:00
stacktrace.c stacktrace: fix modular build, export print_stack_trace and save_stack_trace 2008-06-30 09:20:55 +02:00
step.c x86 ptrace: user-sets-TF nits 2008-07-16 12:15:17 -07:00
summit_32.c x86: make generic arch support NUMAQ 2008-06-10 11:34:42 +02:00
sys_i386_32.c x86: coding style fixes to arch/x86/kernel/sys_i386_32.c 2008-06-10 12:34:54 +02:00
sys_x86_64.c unified (weak) sys_pipe implementation 2008-05-03 13:50:33 -07:00
syscall_64.c x86: coding style fixes to arch/x86/kernel/syscall_64.c 2008-04-17 17:40:48 +02:00
syscall_table_32.S flag parameters add-on: remove epoll_create size param 2008-07-24 10:47:29 -07:00
tce_64.c
test_nx.c x86: Explicitly include required header files. 2008-04-17 17:41:15 +02:00
test_rodata.c x86: include proper prototypes for rodata_test 2008-02-14 23:30:20 +01:00
time_32.c x86: add ->pre_time_init to x86_quirks 2008-07-20 09:25:52 +02:00
time_64.c x86: merge tsc_init and clocksource code 2008-07-09 07:43:27 +02:00
tlb_32.c on_each_cpu(): kill unused 'retry' parameter 2008-06-26 11:24:38 +02:00
tlb_64.c Merge branch 'generic-ipi' into generic-ipi-for-linus 2008-07-15 21:55:59 +02:00
tlb_uv.c x86, SGI UV: uv_ptc_proc_write fix 2008-07-08 12:23:31 +02:00
tls.c asmlinkage_protect replaces prevent_tail_call 2008-04-10 17:28:26 -07:00
tls.h x86: x86 user_regset TLS 2008-01-30 13:31:52 +01:00
topology.c x86: fix section mismatch warning in topology.c:arch_register_cpu 2008-02-19 16:18:30 +01:00
trampoline_32.S x86: trampoline_32.S - switch to .cpuinit.data 2008-04-26 17:35:47 +02:00
trampoline_64.S x86: move suspend wakeup code to C 2008-04-17 17:41:37 +02:00
trampoline.c x86: extend e820 ealy_res support 32bit 2008-05-25 10:55:11 +02:00
traps_32.c x86: introducing asm-x86/traps.h 2008-07-18 18:51:57 +02:00
traps_64.c x86: introducing asm-x86/traps.h 2008-07-18 18:51:57 +02:00
tsc_sync.c x86: add warning to check_tsc_warp() 2008-01-30 13:33:24 +01:00
tsc.c x86: fix TSC build error on 32bit 2008-07-15 22:46:47 +02:00
verify_cpu_64.S
visws_quirks.c x86: introduce x86_quirks 2008-07-20 09:18:17 +02:00
vm86_32.c x86: replace most VM86 flags with flags from processor-flags.h 2008-04-17 17:41:33 +02:00
vmi_32.c x86: APIC: remove apic_write_around(); use alternatives 2008-07-18 12:51:21 +02:00
vmiclock_32.c x86: rename paravirtualized TSC functions 2008-07-09 07:43:28 +02:00
vmlinux_32.lds.S Merge branch 'core/rodata' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-14 15:28:10 -07:00
vmlinux_64.lds.S Merge branch 'core/rodata' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2008-07-14 15:28:10 -07:00
vmlinux.lds.S
vsmp_64.c x86: vsmp_64 add missing includes 2008-05-25 08:58:24 +02:00
vsyscall_64.c Merge branch 'generic-ipi' into generic-ipi-for-linus 2008-07-15 21:55:59 +02:00
x8664_ksyms_64.c Merge branch 'auto-ftrace-next' into tracing/for-linus 2008-07-14 16:11:52 +02:00