21 Jun 2012 14:27
21 Jun 2012 15:35
Re: Kernel Memory
HowKernel StuffWorks <howkernelstuffworks <at> gmail.com>
2012-06-21 13:35:17 GMT
2012-06-21 13:35:17 GMT
Hi,
I am a kernel newbie too, Kernel memory is not pagable, because kernel itself is responsible for paging. See this discussion,
http://kerneltrap.org/node/6404
Paging happens for regular processes, i.e each process memory is divided into a page of certain size(4kb in Linux), so it can swap for another page that might be needed for that particular moment, it usually replaces an page that have not been used for long time(see LRU,internal fragmentation). Kernel is a process too, just like anyother process, but it differs from others because it directly talks to the hardware and also it is the one which takes care of paging and LRU algo's. So it does not make much sense, for a kernel memory to be pageable, because, if it removes the page that contains that has the paging algorithm, then there is no way to come back, you want to retrieve a page from disk_swap_area, but you cannot, because the page that contains code to retrieve the instructions are paged :)... This is not just for paging, since kernel controls everything, it is not advisable to put its own code in swap area(see, virtual memory, virtual memory = physical memory + swap space).
BTW, i am a kernel newbie too, this is my basic understanding, please feel free to correct it, if I am wrong....
On Thu, Jun 21, 2012 at 8:27 AM, Vijay Chauhan <kernel.vijay <at> gmail.com> wrote:
Hello,
I am newbie.
It has been said "kernel memory is not pageable"
What does it mean? There is no concept of kernel virtual address?
Any simple explanation will help me to udnerstand.
Thanks,
Vijay
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies <at> kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
_______________________________________________ Kernelnewbies mailing list Kernelnewbies <at> kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
21 Jun 2012 15:40
Re: Kernel Memory
kishore sheik ahamed <linuxinme <at> gmail.com>
2012-06-21 13:40:13 GMT
2012-06-21 13:40:13 GMT
Hey Vijay
I am a newbie too. Just sharing what I could go through.
It is said that Kernel or atleast a part of kernel needs to be non paged for fast interrupt access etc as pinned memory
Wiki says
Pinned/Locked/Fixed pages
Operating systems have memory areas that are pinned (never swapped to secondary storage). For example, interrupt mechanisms rely on an array of pointers to their handlers, such as I/O completion and page fault. If the pages containing these pointers or the code that they invoke were pageable, interrupt-handling would become far more complex and time-consuming, particularly in the case of page fault interrupts. Hence, some part of the page table structures is not pageable.
Some pages may be pinned for short periods of time, others may be pinned for long periods of time, and still others may need to be permanently pinned. For example:
- The paging supervisor code and drivers for secondary storage devices on which pages reside must be permanently pinned, as otherwise paging wouldn't even work because the necessary code wouldn't be available.
- Timing-dependent components may be pinned to avoid variable paging delays.
- Data buffers that are accessed directly by peripheral devices that use direct memory access or I/O channels must reside in pinned pages while the I/O operation is in progress because such devices and the buses to which they are attached expect to find data buffers located at physical memory addresses; regardless of whether the bus has a memory management unit for I/O, transfers cannot be stopped if a page fault occurs and then restarted when the page fault has been processed.
There are other two discussion thread which say kernel is non-pageable and now due to growing kernel Data structures it is allowed
http://kerneltrap.org/node/6404
http://kerneltrap.org/node/8206
Regards
Kishore
On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan <kernel.vijay <at> gmail.com> wrote:
Hello,
I am newbie.
It has been said "kernel memory is not pageable"
What does it mean? There is no concept of kernel virtual address?
Any simple explanation will help me to udnerstand.
Thanks,
Vijay
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies <at> kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
_______________________________________________ Kernelnewbies mailing list Kernelnewbies <at> kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
22 Jun 2012 20:23
Re: Kernel Memory
Vijay Chauhan <kernel.vijay <at> gmail.com>
2012-06-22 18:23:30 GMT
2012-06-22 18:23:30 GMT
Thanks everyone. I think i got enough information for further study. Thanks, Vijay On Thu, Jun 21, 2012 at 7:10 PM, kishore sheik ahamed <linuxinme <at> gmail.com> wrote: > Hey Vijay > > I am a newbie too. Just sharing what I could go through. > > It is said that Kernel or atleast a part of kernel needs to be non paged for > fast interrupt access etc as pinned memory > Wiki says > > Pinned/Locked/Fixed pages > > Operating systems have memory areas that are pinned (never swapped to > secondary storage). For example, interrupt mechanisms rely on an array of > pointers to their handlers, such as I/O completion and page fault. If the > pages containing these pointers or the code that they invoke were pageable, > interrupt-handling would become far more complex and time-consuming, > particularly in the case of page fault interrupts. Hence, some part of the > page table structures is not pageable. > > Some pages may be pinned for short periods of time, others may be pinned for > long periods of time, and still others may need to be permanently pinned. > For example: > > The paging supervisor code and drivers for secondary storage devices on > which pages reside must be permanently pinned, as otherwise paging wouldn't > even work because the necessary code wouldn't be available. > Timing-dependent components may be pinned to avoid variable paging delays. > Data buffers that are accessed directly by peripheral devices that use > direct memory access or I/O channels must reside in pinned pages while the > I/O operation is in progress because such devices and the buses to which > they are attached expect to find data buffers located at physical memory > addresses; regardless of whether the bus has a memory management unit for > I/O, transfers cannot be stopped if a page fault occurs and then restarted > when the page fault has been processed. > > There are other two discussion thread which say kernel is non-pageable and > now due to growing kernel Data structures it is allowed > > http://kerneltrap.org/node/6404 > > http://kerneltrap.org/node/8206 > > > Regards > > Kishore > > > > On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan <kernel.vijay <at> gmail.com> > wrote: >> >> Hello, >> >> I am newbie. >> It has been said "kernel memory is not pageable" >> What does it mean? There is no concept of kernel virtual address? >> >> Any simple explanation will help me to udnerstand. >> >> Thanks, >> Vijay >> >> _______________________________________________ >> Kernelnewbies mailing list >> Kernelnewbies <at> kernelnewbies.org >> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies > >
21 Jun 2012 15:44
Re: Kernel Memory
Arun KS <getarunks <at> gmail.com>
2012-06-21 13:44:29 GMT
2012-06-21 13:44:29 GMT
Hello Vijay, On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan <kernel.vijay <at> gmail.com> wrote: > Hello, > > I am newbie. > It has been said "kernel memory is not pageable" > What does it mean? There is no concept of kernel virtual address? You might have heard about 3G/1G split. This 1GB is the virtual address of the kernel. And whenever kernel try to access any address in this range(for ARM architecture it is 0xC0000000 - 0xFFFFFFFF), there should not be any page fault. That means MMU should be able to convert your virtual address to physical. This 1GB contains your IO address, RAM address. Paging is a mechanism OS uses to pull the data(in pagesizes) to and fro between system RAM and secondary memory. Kernel memory is not pageable. This means memory allocated for the kernel will not be pagged out. If you try to access any memory in kernel with out creating page tables(this can be done by ioremap) you will end up in OOPS. The main reason of kernel not being swapable or pageable is as follows. Think this way. What will happen if we have paged out that portion of the logic which decides what to do when a page fault occurs? Who will take care of the page fault then? But if a user program hit a page fault(ie accessed address is not in main memory), kernel will load the page from secondary memory if it is a valid address. And if the address accesses is illegal, kernel kill the user application(Segmentation fault). Thanks, Arun > > Any simple explanation will help me to udnerstand. > > Thanks, > Vijay > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies <at> kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
21 Jun 2012 16:02
Re: Kernel Memory
AFT <aftnix <at> gmail.com>
2012-06-21 14:02:55 GMT
2012-06-21 14:02:55 GMT
Vijay Chauhan <kernel.vijay <at> gmail.com> writes: > Hello, > > I am newbie. > It has been said "kernel memory is not pageable" > What does it mean? There is no concept of kernel virtual address? > Yes. Kernel works on static adress space. > Any simple explanation will help me to udnerstand. > I'm not sure if you want to understand "how kernel manages memory for its internal DS". If its the case you should read the following documents. 1) Read the chapter 8 of Linux Device driver 3rd edition. http://lwn.net/Kernel/LDD3/ 2) To understand slab allocator read the following papers by bonwick a) 94 paper describing slab allocator: http://static.usenix.org/publications/library/proceedings/bos94/full_papers/bonwick.a b) Its followup in 2001 http://static.usenix.org/event/usenix01/full_papers/bonwick/bonwick_html/index.html 3) These should be enough. But if you want to know detailed architecture of how virtual memory manager work you should read Gorman's book on Linux virtual memory manager. Its a free pdf. can be found here: http://ptgmedia.pearsoncmg.com/images/0131453483/downloads/gorman_book.pdf Happy hacking. Cheers aft > Thanks, > Vijay > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies <at> kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
21 Jun 2012 17:48
Re: Kernel Memory
<michi1 <at> michaelblizek.twilightparadox.com>
2012-06-21 15:48:25 GMT
2012-06-21 15:48:25 GMT
Hi! On 17:57 Thu 21 Jun , Vijay Chauhan wrote: > Hello, > > I am newbie. > It has been said "kernel memory is not pageable" > What does it mean? There is no concept of kernel virtual address? > > Any simple explanation will help me to udnerstand. The right term is actually "kernel memory is not swappable". Swapping means writing inactive memory to disk and then using it for something else. Kernel memory not being swappable is a design decicion made in the early linux days. Operating systems which swap kernel memory need to isolate everything which should not be swappd out (e.g. things needed for swap-in, realtime stuff, security sensitive data, ...). This is quite a bit of work. I also guess it is pretty pointless nowadays. Installed memory and is getting so huge that virtual memory developers have a hard time trying to keep cpu-usage overhead for swapping user space memory low. > There is no concept of kernel virtual address? Kernel memory uses virtual addresses as well. However, these the entire system memory is continuously mapped somewhere in the virtual address space. The drawback is that fragmentation turns allocation of large continuous memory regions into a game of luck. There is also an virtual address area (vmalloc) which is used to dynamically map multiple scattered pages to a continuous region. But this is rather slow and rarely used. You might want to take a look at: http://lwn.net/Kernel/LDD3/ -Michi -- -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
RSS Feed