Lai Jiangshan | 11 May 07:46 2012

[PATCH] memory: add kernelcore_max_addr boot option

Current ZONE_MOVABLE (kernelcore=) setting policy with boot option doesn't meet
our requirement. We need something like kernelcore_max_addr= boot option
to limit the kernelcore upper address.

The memory with higher address will be migratable(movable) and they
are easier to be offline(always ready to be offline when the system don't require
so much memory).

All kernelcore_max_addr=, kernelcore= and movablecore= can be safely specified
at the same time(or any 2 of them).

Signed-off-by: Lai Jiangshan <laijs <at> cn.fujitsu.com>
---
 Documentation/kernel-parameters.txt |    9 +++++++++
 mm/page_alloc.c                     |   27 ++++++++++++++++++++++++++-
 2 files changed, 35 insertions(+), 1 deletions(-)
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index c1601e5..9f42787 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
 <at>  <at>  -1184,6 +1184,15  <at>  <at>  bytes respectively. Such letter suffixes can also be entirely omitted.
 			use the HighMem zone if it exists, and the Normal
 			zone if it does not.

+	kernelcore_max_addr=nn[KMG]	[KNL,X86,IA-64,PPC] This parameter
+			is the same effect as kernelcore parameter, except it
+			specifies the up physical address of memory range
+			usable by the kernel for non-movable allocations.
+			If both kernelcore and kernelcore_max_addr are
+			specified, this requested's priority is higher than
(Continue reading)

Andrew Morton | 11 May 22:39 2012

Re: [PATCH] memory: add kernelcore_max_addr boot option

On Fri, 11 May 2012 13:46:04 +0800
Lai Jiangshan <laijs <at> cn.fujitsu.com> wrote:

> Current ZONE_MOVABLE (kernelcore=) setting policy with boot option doesn't meet
> our requirement. We need something like kernelcore_max_addr= boot option
> to limit the kernelcore upper address.

Why do you need this?  Please fully describe the requirement/use case.

> The memory with higher address will be migratable(movable) and they
> are easier to be offline(always ready to be offline when the system don't require
> so much memory).
> 
> All kernelcore_max_addr=, kernelcore= and movablecore= can be safely specified
> at the same time(or any 2 of them).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo <at> kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont <at> kvack.org"> email <at> kvack.org </a>

Yasuaki Ishimatsu | 14 May 04:33 2012

Re: [PATCH] memory: add kernelcore_max_addr boot option

Hi Andrew,

2012/05/12 5:39, Andrew Morton wrote:
> On Fri, 11 May 2012 13:46:04 +0800
> Lai Jiangshan<laijs <at> cn.fujitsu.com>  wrote:
>
>> Current ZONE_MOVABLE (kernelcore=) setting policy with boot option doesn't meet
>> our requirement. We need something like kernelcore_max_addr= boot option
>> to limit the kernelcore upper address.
>
> Why do you need this?  Please fully describe the requirement/use case.

We want to create removable node for removing a system board
which equip CPU and memory at runtime. To do this, all memory
of a node on system board must be allocated as ZONE_MOVABLE.
But current linux cannot do it.
So we create removable node by limiting the memory address of
the kernelcore by the boot option.

Thanks,
Yasuaki Ishimatsu

>> The memory with higher address will be migratable(movable) and they
>> are easier to be offline(always ready to be offline when the system don't require
>> so much memory).
>>
>> All kernelcore_max_addr=, kernelcore= and movablecore= can be safely specified
>> at the same time(or any 2 of them).
>
> --
(Continue reading)

Yasuaki Ishimatsu | 14 May 13:50 2012

Re: [PATCH] memory: add kernelcore_max_addr boot option

Hi Lai,

Your patch does not consider allocated memory from  memblock.
Thus even if I set the kernelcore_max_addr boot option, movable
node cannot be created.

I made sample patches that limited the memory from memblock.

[Patch 1/4] x86: get pg_data_t's memory from other node
[Patch 2/4] x86: use memblock_set_current_limit() to set memblock.current_limit
[Patch 3/4] memblock: limit memory address from memblock
[Patch 4/4] memblock: compare current_limit with end variable at memblock_find_in_range_node()

System seems to be able to create movable node by applying these
patches.

But there are two problems.
   - When online memory of movable zone is under 512MB by offlining
     memory, system cannot create new process.
   - When all memory of movable zone is offlined, "kernel BUG at
     mm/slub.c:3587!" message is shown.

I have not understood the root cause of the problems.

Thanks,
Yasuaki Ishimatsu

2012/05/11 14:46, Lai Jiangshan wrote:
> Current ZONE_MOVABLE (kernelcore=) setting policy with boot option doesn't meet
> our requirement. We need something like kernelcore_max_addr= boot option
(Continue reading)

Yasuaki Ishimatsu | 14 May 13:58 2012

[Patch 1/4] x86: get pg_data_t's memory from other node

If system can create movable node which all memory of the
node is allocated as ZONE_MOVABLE, setup_node_data() cannot
allocate memory for the node's pg_data_t.
So when memblock_alloc_nid() fails, setup_node_data() retries
memblock_alloc().

---
  arch/x86/mm/numa.c |    8 ++++++--
  1 file changed, 6 insertions(+), 2 deletions(-)

Index: linux-3.4-rc6/arch/x86/mm/numa.c
===================================================================
--- linux-3.4-rc6.orig/arch/x86/mm/numa.c	2012-05-15 06:43:38.887962970 +0900
+++ linux-3.4-rc6/arch/x86/mm/numa.c	2012-05-15 06:43:42.422918776 +0900
 <at>  <at>  -223,9 +223,13  <at>  <at>  static void __init setup_node_data(int n
  		remapped = true;
  	} else {
  		nd_pa = memblock_alloc_nid(nd_size, SMP_CACHE_BYTES, nid);
-		if (!nd_pa) {
-			pr_err("Cannot find %zu bytes in node %d\n",
+		if (!nd_pa)
+			printk(KERN_WARNING "Cannot find %zu bytes in node %d\n",
  			       nd_size, nid);
+		nd_pa = memblock_alloc(nd_size, SMP_CACHE_BYTES);
+		if (!nd_pa) {
+			pr_err("Cannot find %zu bytes in other node\n",
+			       nd_size);
  			return;
  		}
  		nd = __va(nd_pa);
(Continue reading)

Yasuaki Ishimatsu | 14 May 13:58 2012

[Patch 2/4] x86: use memblock_set_current_limit() to set memblock.current_limit

memblock.current_limit is set directly though memblock_set_current_limit()
is prepared. So fix it.

Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki <at> jp.fujitsu.com>
---
  arch/x86/kernel/setup.c |    4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)

Index: linux-3.4-rc6/arch/x86/kernel/setup.c
===================================================================
--- linux-3.4-rc6.orig/arch/x86/kernel/setup.c	2012-05-15 04:43:11.862313172 +0900
+++ linux-3.4-rc6/arch/x86/kernel/setup.c	2012-05-15 06:44:53.504030089 +0900
 <at>  <at>  -897,7 +897,7  <at>  <at>  void __init setup_arch(char **cmdline_p)

  	cleanup_highmap();

-	memblock.current_limit = get_max_mapped();
+	memblock_set_current_limit(get_max_mapped());
  	memblock_x86_fill();

  	/*
 <at>  <at>  -933,7 +933,7  <at>  <at>  void __init setup_arch(char **cmdline_p)
  		max_low_pfn = max_pfn;
  	}
  #endif
-	memblock.current_limit = get_max_mapped();
+	memblock_set_current_limit(get_max_mapped());

  	/*
  	 * NOTE: On x86-32, only from this point on, fixmaps are ready for use.
(Continue reading)

Yasuaki Ishimatsu | 14 May 13:58 2012

[Patch 3/4] memblock: limit memory address from memblock

Setting kernelcore_max_pfn means all memory which is bigger than
the boot parameter is allocated as ZONE_MOVABLE. So memory which
is allocated by memblock also should be limited by the parameter.

The patch limits memory from memblock.

---
  include/linux/memblock.h |    1 +
  mm/memblock.c            |    5 ++++-
  mm/page_alloc.c          |    6 +++++-
  3 files changed, 10 insertions(+), 2 deletions(-)

Index: linux-3.4-rc6/include/linux/memblock.h
===================================================================
--- linux-3.4-rc6.orig/include/linux/memblock.h	2012-05-15 03:17:33.180555589 +0900
+++ linux-3.4-rc6/include/linux/memblock.h	2012-05-15 03:51:25.102153084 +0900
 <at>  <at>  -42,6 +42,7  <at>  <at>  struct memblock {

  extern struct memblock memblock;
  extern int memblock_debug;
+extern phys_addr_t memblock_limit;

  #define memblock_dbg(fmt, ...) \
  	if (memblock_debug) printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
Index: linux-3.4-rc6/mm/memblock.c
===================================================================
--- linux-3.4-rc6.orig/mm/memblock.c	2012-05-15 03:17:33.180555589 +0900
+++ linux-3.4-rc6/mm/memblock.c	2012-05-15 03:51:25.104153055 +0900
 <at>  <at>  -876,7 +876,10  <at>  <at>  int __init_memblock memblock_is_region_r

(Continue reading)

Sam Ravnborg | 14 May 18:52 2012

Re: [Patch 3/4] memblock: limit memory address from memblock

On Mon, May 14, 2012 at 08:58:54PM +0900, Yasuaki Ishimatsu wrote:
> Setting kernelcore_max_pfn means all memory which is bigger than
> the boot parameter is allocated as ZONE_MOVABLE. So memory which
> is allocated by memblock also should be limited by the parameter.
>
> The patch limits memory from memblock.

I see no reason why we need two limits for memblock.
And if we really require two limits then please use a function
to set it.
All other setup/etc. towoards memblock is via function,
and starting to introduce magic variables is confusing.

Also new stuff in memblock shal have a nice comment
describing the usage. that we in the past has failed to
do so is no excuse.

	Sam

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo <at> kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont <at> kvack.org"> email <at> kvack.org </a>

Yasuaki Ishimatsu | 14 May 13:59 2012

[Patch 4/4] memblock: compare current_limit with end variable at memblock_find_in_range_node()

memblock_find_in_range_node() does not compare memblock.current_limit
with end variable. Thus even if memblock.current_limit is smaller than
end variable, the function allocates memory address that is bigger than
memblock.current_limit.

The patch adds the check to "memblock_find_in_range_node()"

Signed-off-by: Yasuaki Ishimatsu <isimatu.yasuaki <at> jp.fujitsu.com>
---
  mm/memblock.c |    5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)

Index: linux-3.4-rc6/mm/memblock.c
===================================================================
--- linux-3.4-rc6.orig/mm/memblock.c	2012-05-15 03:51:25.104153055 +0900
+++ linux-3.4-rc6/mm/memblock.c	2012-05-15 04:16:49.468094485 +0900
 <at>  <at>  -97,11 +97,12  <at>  <at>  phys_addr_t __init_memblock memblock_fin
  					phys_addr_t align, int nid)
  {
  	phys_addr_t this_start, this_end, cand;
+	phys_addr_t current_limit = memblock.current_limit;
  	u64 i;

  	/* pump up  <at> end */
-	if (end == MEMBLOCK_ALLOC_ACCESSIBLE)
-		end = memblock.current_limit;
+	if ((end == MEMBLOCK_ALLOC_ACCESSIBLE) || (end > current_limit))
+		end = current_limit;

  	/* avoid allocating the first page */
(Continue reading)

Yasuaki Ishimatsu | 14 May 14:01 2012

Re: [PATCH] memory: add kernelcore_max_addr boot option

Hi Lai,

2012/05/14 20:50, Yasuaki Ishimatsu wrote:
> Hi Lai,
>
> Your patch does not consider allocated memory from memblock.
> Thus even if I set the kernelcore_max_addr boot option, movable
> node cannot be created.
>
> I made sample patches that limited the memory from memblock.
>
> [Patch 1/4] x86: get pg_data_t's memory from other node
> [Patch 2/4] x86: use memblock_set_current_limit() to set memblock.current_limit
> [Patch 3/4] memblock: limit memory address from memblock
> [Patch 4/4] memblock: compare current_limit with end variable at memblock_find_in_range_node()
>
> System seems to be able to create movable node by applying these
> patches.
>

> But there are two problems.
> - When online memory of movable zone is under 512MB by offlining
> memory, system cannot create new process.
> - When all memory of movable zone is offlined, "kernel BUG at
> mm/slub.c:3587!" message is shown.

There are typos.
s/zone/node/

Thanks,
(Continue reading)


Gmane