The hugepage arch code provides a number of hook functions/macros
which mirror the functionality of various normal page pte access
functions. Various changes in the normal page accessors (in
particular BenH's recent changes to the handling of lazy icache
flushing and PAGE_EXEC) have caused the hugepage versions to get out
of sync with the originals. In some cases, this is a bug, at least on
some MMU types.
One of the reasons that some hooks were not identical to the normal
page versions, is that the fact we're dealing with a hugepage needed
to be passed down do use the correct dcache-icache flush function.
This patch makes the main flush_dcache_icache_page() function hugepage
aware (by checking for the PageCompound flag). That in turn means we
can make set_huge_pte_at() just a call to set_pte_at() bringing it
back into sync. As a bonus, this lets us remove the
hash_huge_page_do_lazy_icache() function, replacing it with a call to
the hash_page_do_lazy_icache() function it was based on.
Some other hugepage pte access hooks - huge_ptep_get_and_clear() and
huge_ptep_clear_flush() - are not so easily unified, but this patch at
least brings them back into sync with the current versions of the
corresponding normal page functions.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
This patch separates the parts of hugetlbpage.c which are inherently
specific to the hash MMU into a new hugelbpage-hash64.c file.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
This patch simplifies the logic used to initialize hugepages on
powerpc. The somewhat oddly named set_huge_psize() is renamed to
add_huge_page_size() and now does all necessary verification of
whether it's given a valid hugepage sizes (instead of just some) and
instantiates the generic hstate structure (but no more).
hugetlbpage_init() now steps through the available pagesizes, checks
if they're valid for hugepages by calling add_huge_page_size() and
initializes the kmem_caches for the hugepage pagetables. This means
we can now eliminate the mmu_huge_psizes array, since we no longer
need to pass the sizing information for the pagetable caches from
set_huge_psize() into hugetlbpage_init()
Determination of the default huge page size is also moved from the
hash code into the general hugepage code.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Currently each available hugepage size uses a slightly different
pagetable layout: that is, the bottem level table of pointers to
hugepages is a different size, and may branch off from the normal page
tables at a different level. Every hugepage aware path that needs to
walk the pagetables must therefore look up the hugepage size from the
slice info first, and work out the correct way to walk the pagetables
accordingly. Future hardware is likely to add more possible hugepage
sizes, more layout options and more mess.
This patch, therefore reworks the handling of hugepage pagetables to
reduce this complexity. In the new scheme, instead of having to
consult the slice mask, pagetable walking code can check a flag in the
PGD/PUD/PMD entries to see where to branch off to hugepage pagetables,
and the entry also contains the information (eseentially hugepage
shift) necessary to then interpret that table without recourse to the
slice mask. This scheme can be extended neatly to handle multiple
levels of self-describing "special" hugepage pagetables, although for
now we assume only one level exists.
This approach means that only the pagetable allocation path needs to
know how the pagetables should be set out. All other (hugepage)
pagetable walking paths can just interpret the structure as they go.
There already was a flag bit in PGD/PUD/PMD entries for hugepage
directory pointers, but it was only used for debug. We alter that
flag bit to instead be a 0 in the MSB to indicate a hugepage pagetable
pointer (normally it would be 1 since the pointer lies in the linear
mapping). This means that asm pagetable walking can test for (and
punt on) hugepage pointers with the same test that checks for
unpopulated page directory entries (beq becomes bge), since hugepage
pointers will always be positive, and normal pointers always negative.
While we're at it, we get rid of the confusing (and grep defeating)
#defining of hugepte_shift to be the same thing as mmu_huge_psizes.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Currently we have a fair bit of rather fiddly code to manage the
various kmem_caches used to store page tables of various levels. We
generally have two caches holding some combination of PGD, PUD and PMD
tables, plus several more for the special hugepage pagetables.
This patch cleans this all up by taking a different approach. Rather
than the caches being designated as for PUDs or for hugeptes for 16M
pages, the caches are simply allocated to be a specific size. Thus
sharing of caches between different types/levels of pagetables happens
naturally. The pagetable size, where needed, is passed around encoded
in the same way as {PGD,PUD,PMD}_INDEX_SIZE; that is n where the
pagetable contains 2^n pointers.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Currently, hpte_need_flush() only correctly flushes the given address
for normal pages. Callers for hugepages are required to mask the
address themselves.
But hpte_need_flush() already looks up the page sizes for its own
reasons, so this is a rather silly imposition on the callers. This
patch alters it to mask based on the pagesize it has looked up itself,
and removes the awkward masking code in the hugepage caller.
Signed-off-by: David Gibson <dwg@au1.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
When running Active Memory Sharing, the Collaborative Memory Manager (CMM)
may mark some pages as "loaned" with the hypervisor. Periodically, the
CMM will query the hypervisor for a loan request, which is a single signed
value. When kexec'ing into a kdump kernel, the CMM driver in the kdump
kernel is not aware of the pages the previous kernel had marked as "loaned",
so the hypervisor and the CMM driver are out of sync. Fix the CMM driver
to handle this scenario by ignoring requests to decrease the number of loaned
pages if we don't think we have any pages loaned. Pages that are marked as
"loaned" which are not in the balloon will automatically get switched to "active"
the next time we touch the page. This also fixes the case where totalram_pages
is smaller than min_mem_mb, which can occur during kdump.
Signed-off-by: Brian King <brking@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
get_irq_desc() is a powerpc-specific version of irq_to_desc(). That
is reason enough to remove it, but it also doesn't know about sparse
irq_desc support which irq_to_desc() does (when we enable it).
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Rather than open-coding our own check, use irq_has_action()
to check if an irq has an action - ie. is "in use".
irq_has_action() doesn't take the descriptor lock, but it
shouldn't matter - we're just using it as an indicator
that the irq is in use. disable_irq_nosync() will take
the descriptor lock before doing anything also.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
The irq_desc array consumes quite a lot of space, and for systems
that don't need or can't have 512 irqs it's just wasted space.
The first 16 are reserved for ISA, so the minimum of 32 is really
16 - and no one has asked for more than 512 so leave that as the
maximum.
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
The CHRP code has some fishy timer based code to scan the RTAS event
log, which uses a 1KB stack buffer and doesn't even use the results.
The pSeries code as a nicer daemon that allows userspace to read the
event log and basically uses the same RTAS interface
This patch moves rtasd.c out of platform/pseries and makes it usable
by CHRP, after removing the old crufty event log mechanism in there.
The nvram logging part of the daemon is still only available on 64-bit
since the underlying nvram management routines aren't currently shared.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Some of the stuff in /proc/ppc64 such as the RTAS bits are actually
useful to some 32-bit platforms. Rename the file, and create a
symlink on 64-bit for backward compatibility
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Just as with kexec, hibernation may fail even on well-tested platforms:
some PCI device, a driver of which doesn't play well with hibernation,
is enough to break resuming.
Hibernation code is not much platform dependent, and hiding features only
because these were not verified on a particular hardware is
counterproductive: we just prevent the features from being widely tested.
For example, with this patch I just tested hibernation on a MPC83xx
board, and it works quite well, modulo a few drivers that need some
fixing.
So, let's make it possible to select hibernation support for all
PowerPCs, then let's wait for any possible bug reports, and actually fix
(or just collect ;-) the bugs instead of hiding them. If some platforms
really can't stand hibernation, we can make a blacklist, with proper
comments why exactly hibernation doesn't work, whether it is possible to
fix, and what needs to be done to fix it.
CONFIG_HIBERNATION is still =n by default, so the commit doesn't change
anything apart from ability to set it to =y.
I'm not sure if EXPERIMENTAL dependency is needed, I'd rather not add it
for a few reasons:
1) It doesn't matter much, for distro kernels user has no clue that some
feature is experimental. Majority of defconfigs enable EXPERIMENTAL
anyway (90 vs. 4, which, btw, means that EXPERIMENTAL is overused
in Kconfigs);
2) EXPERIMENTAL is a good thing for features that change default
behaviour of a kernel, while for hibernation user has to explicitly
issue 'echo disk > /sys/power/state' to trigger any hibernation bugs;
3) Per init/Kconfig, EXPERIMENTAL is a good thing to scare and discourage
users from 'widespread use of a feature', while we want to encourage
that use.
Signed-off-by: Anton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
The non-debug case in ps3/mm.c uses pr_debug(), so that the compiler
still does type checks etc. and doesn't complain about unused
variables in the non-debug case.
However with DEBUG=n and CONFIG_DYNAMIC_DEBUG=y there's still code
generated for those pr_debugs().
size before:
text data bss dec hex filename
17553 4112 88 21753 54f9 arch/powerpc/platforms/ps3/mm.o
size after:
text data bss dec hex filename
7377 776 88 8241 2031 arch/powerpc/platforms/ps3/mm.o
Signed-off-by: Michael Ellerman <michael@ellerman.id.au>
Acked-by: Geoff Levand <geoffrey.levand@am.sony.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Add R-standby specific bits to the SuperH Mobile sleep code.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Use RSMEM instead of ILMEM for sleep mode code storage on SH7724.
This allows us to use R-standby mode on SH7724.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Add MMU and cache handling functionality to the SuperH Mobile
sleep code. The MMU and cache registers are saved and restored.
The MMU is disabled and the cache is flushed and disabled before
entering sleep modes if the SUSP_SH_MMU flag is set. This flag
should be set in the case of R-standby and most likely for future
U-standby support as well.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Add code to keep track of supported sleep modes. This to
only export cpuidle modes that are backed by board support
code. Also, do not allow suspend-to-ram if sdram board code
is missing.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Rework the SuperH Mobile sleep code from including
board specific code to allowing each board to provide
pre/post code snippets. These snippets should contain
sdram management code to enter and leave self-refresh.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Move the AP325RXA board code from a single board file
to a separate directory. This to make it easy to add
support for sdram sleep mode code.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Add code to allow boards registering self-contained
functions for going to/from self-refresh. At this
point the board code is unused. When all supported
boards have been converted then the new sleep code
will make use of these functions.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Make use of the recently added notifier chains for sh7724
r-standby register save/restore handling. At this point
only the BSC and INTC are handled.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
This patch adds atomic notifier chains for pre/post
sleep events. Useful for cpu code and boards that
need to save and restore register state before and
after entering a sleep mode.
Signed-off-by: Magnus Damm <damm@opensource.se>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
The variable 'phys' already contains the physical address to flush. It
is not a virtual address and should not be passed to virt_to_phys().
Signed-off-by: Matt Fleming <matt@console-pimps.org>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Errata 411920 indicates that any "invalidate entire instruction cache"
operation can fail if the right conditions are present. This is not
limited just to those operations in flush.c, but elsewhere. Place the
workaround in the already existing __flush_icache_all() function
instead.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
This adds a better sched_clock() to the IOP platform,
implemented using its new clocksource support.
Tested on n2100, compile-tested for all plat-iop machines.
[dan.j.williams@intel.com: allow early cp6 access]
Signed-off-by: Mikael Pettersson <mikpe@it.uu.se>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This updates the IOP platform to use the kernel's generic time
framework. With clockevent support in place, this reduces to
selecting GENERIC_TIME and removing the platform's private timer
->offset() operation (iop_gettimeoffset).
Tested on n2100, compile-tested for all plat-iop machines.
Signed-off-by: Mikael Pettersson <mikpe@it.uu.se>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This updates the IOP platform to expose the interrupting
timer 0 as a clockevent object. The timer interrupt handler
is changed to call the clockevent ->event_handler() instead
of timer_tick(), and ->set_next_event() and ->set_mode()
operations are added to allow the mode of the timer to be
updated (required for ONESHOT/NOHZ mode).
Timer 0 must now be properly initialised, which requires
a new write_tcr0() function from the mach-specific code.
The mode of timer 0 must be read at the start of ->set_mode(),
which requires a new read_tmr0() function from the mach-
specific code.
Initial setup of timer 0 is also rewritten to be more robust.
Tested on n2100, compile-tested for all plat-iop machines.
Signed-off-by: Mikael Pettersson <mikpe@it.uu.se>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
This updates the IOP platform to expose the free-running
timer 1 as a clocksource object. This timer is now also
properly initialised, which requires a new write_tcr1()
function from the mach-specific code. Apart from the
explicit initialisation, there is no functional change
in how timer 1 is programmed.
Tested on n2100, compile-tested for all plat-iop machines.
Signed-off-by: Mikael Pettersson <mikpe@it.uu.se>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
When SPARSEMEM_EXTREME is enabled, memory_present() wants to use bootmem
to allocate data structures. However, we call memory_present() after
declaring memory to bootmem, but before we've reserved areas.
This leads to sparsemem data structures being overwritten later in the
kernel's initialization (when slab initializes.)
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* 'merge' of git://git.kernel.org/pub/scm/linux/kernel/git/benh/powerpc:
powerpc/ppc64: Use preempt_schedule_irq instead of preempt_schedule
powerpc: Minor cleanup to lib/Kconfig.debug
powerpc: Minor cleanup to sound/ppc/Kconfig
powerpc: Minor cleanup to init/Kconfig
powerpc: Limit memory hotplug support to PPC64 Book-3S machines
powerpc: Limit hugetlbfs support to PPC64 Book-3S machines
powerpc: Fix compile errors found by new ppc64e_defconfig
powerpc: Add a Book-3E 64-bit defconfig
powerpc/booke: Fix xmon single step on PowerPC Book-E
powerpc: Align vDSO base address
powerpc: Fix segment mapping in vdso32
powerpc/iseries: Remove compiler version dependent hack
powerpc/perf_events: Fix priority of MSR HV vs PR bits
powerpc/5200: Update defconfigs
drivers/serial/mpc52xx_uart.c: Use UPIO_MEM rather than SERIAL_IO_MEM
powerpc/boot/dts: drop obsolete 'fsl5200-clocking'
of: Remove nested function
mpc5200: support for the MAN mpc5200 based board mucmc52
mpc5200: support for the MAN mpc5200 based board uc101
* 'omap-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap-2.6:
omap4: Fix UART4 platform data on omap4
omap4: Allow omap_serial_early_init() for OMAP4430 board
omap3: PM: enable UART3 module wakeups
omap2: Fix console serial port number for n8x0
omap2: Fix detection of n8x0
omap1: Fix DSP public peripherals support for ams-delta
omap1: Fix redundant UARTs pin muxing that can break other hardware support
omap: iommu: fix wrong condition check for SUPERSECTION
omap: SDMA: Fix omap_stop_dma() API for channel linking
omap: Fix omap-keypad by restoring old keypad.h without breaking omap2 boards that use matrix_keypad
* 'sched-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
sched: Do less agressive buddy clearing
sched: Disable SD_PREFER_LOCAL for MC/CPU domains
* 'x86-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
x86, UV: Set DELIVERY_MODE=4 for vector=NMI_VECTOR in uv_hub_send_ipi()
x86, UV: Fix and clean up bau code to use uv_gpa_to_pnode()
x86: Don't print number of MCE banks for every CPU
x86, UV: Fix information in __uv_hub_info structure
x86: Document linker script ASSERT() quirk
sigp sense only returns the status of a cpu if it is non zero. If the
status of the sensed cpu is all zeros condition code 0 (accpeted) is
set and no status bits are returned.
The current code however assumes that a status was returned and tests
bits in it. This means uninitalized data is accessed with random
results.
Worst case is that the code that checks if cpu is offline on cpu
hotplug assumes that the target cpu is offline while it is still
running. This leads potentially to memory corruption since resources
that are still needed by the target cpu will be freed and could be
resused while still in use.
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
According to the architecture a cpu must not necessarily enter stopped
state after completion of a sigp instruction with "stop" order code.
So remove the BUG() statement after self sending sigp stop to avoid
that it ever gets reached.
Also add a sigp busy check to make sure that the order gets delivered.
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Offlined cpus still have valid prefix register contents. Dumpers
will store the register contents of a cpu to the location where its
prefix register points to.
For offlined cpus the area (lowcore) has been freed and the dumper
would write the uninteresting contents of the offline cpu to a memory
location which might be in use by some other component and destroy
valueable information.
To fix this set the prefix register of offline cpus to absolute
address zero again. This prevents the current dumpers to write to
random memory locations.
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Now that the return from alloc_percpu is compatible with the address
of per-cpu vars, it makes sense to hand around the address of per-cpu
variables. To make this sane, we remove the per_cpu__ prefix we used
created to stop people accidentally using these vars directly.
Now we have sparse, we can use that (next patch).
tj: * Updated to convert stuff which were missed by or added after the
original patch.
* Kill per_cpu_var() macro.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Christoph Lameter <cl@linux-foundation.org>
