headers
Konrad Rzeszutek Wilk | 3 Jan 2012 21:33
Picon
Favicon

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
> Dear all,
> 
> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
> packet data,  we have statically allocated huge DMA memory buffer ring
> at once instead of calling dma_map_single() per packet.  Considering
> to further reduce the copy overhead between different NIC(port) ring
> while forwarding, one packet from a input NIC(port) will be
> transferred to output NIC(port) with no any copy action.
> 
> To satisfy this requirement, the packet memory should be mapped into
> input port and unmapped when leaving input port, then mapped into
> output port and unmapped later.
> 
> Whether it's legal to map the same DMA memory into input and output
> port simultaneously? If it's not, then the zero-copy for packet
> forwarding is not feasible?
> 

Did you ever a get a response about this?

Is the output/input port on a seperate device function? Or is it
just a specific MMIO BAR in your PCI device?

> Hope PCI expert to post your suggestion.
> 
> Thanks!
> 
> --Yanfei
>
(Continue reading)

Permalink | Reply |
headers
Yanfei Wang | 4 Jan 2012 03:44
Picon

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
<konrad.wilk <at> oracle.com> wrote:
> On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
>> Dear all,
>>
>> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
>> packet data,  we have statically allocated huge DMA memory buffer ring
>> at once instead of calling dma_map_single() per packet.  Considering
>> to further reduce the copy overhead between different NIC(port) ring
>> while forwarding, one packet from a input NIC(port) will be
>> transferred to output NIC(port) with no any copy action.
>>
>> To satisfy this requirement, the packet memory should be mapped into
>> input port and unmapped when leaving input port, then mapped into
>> output port and unmapped later.
>>
>> Whether it's legal to map the same DMA memory into input and output
>> port simultaneously? If it's not, then the zero-copy for packet
>> forwarding is not feasible?
>>
>
> Did you ever a get a response about this?
No.
>
> Is the output/input port on a seperate device function? Or is it
> just a specific MMIO BAR in your PCI device?
>
Platform: x86, intel nehalem 8Core NUMA, linux 2.6.39, 10G
82599NIC(two ports per NIC card);
Function: Forwarding packets between different ports.
(Continue reading)

Permalink | Reply |
headers
James Bottomley | 4 Jan 2012 16:59

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
> <konrad.wilk <at> oracle.com> wrote:
> > On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
> >> Dear all,
> >>
> >> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
> >> packet data,  we have statically allocated huge DMA memory buffer ring
> >> at once instead of calling dma_map_single() per packet.  Considering
> >> to further reduce the copy overhead between different NIC(port) ring
> >> while forwarding, one packet from a input NIC(port) will be
> >> transferred to output NIC(port) with no any copy action.
> >>
> >> To satisfy this requirement, the packet memory should be mapped into
> >> input port and unmapped when leaving input port, then mapped into
> >> output port and unmapped later.
> >>
> >> Whether it's legal to map the same DMA memory into input and output
> >> port simultaneously? If it's not, then the zero-copy for packet
> >> forwarding is not feasible?
> >>
> >
> > Did you ever a get a response about this?
> No.

This is probably because no-one really understands what you're asking.
As far as mapping memory to PCI devices goes, it's the job of the bridge
(or the iommu which may or may not be part of the bridge).  A standard
iommu tends not to care about devices and functions, so a range once
mapped is available to everything behind the bridge.  A more secure
(Continue reading)

Permalink | Reply |
headers
Yanfei Wang | 5 Jan 2012 13:40
Picon

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Wed, Jan 4, 2012 at 11:59 PM, James Bottomley
<James.Bottomley <at> hansenpartnership.com> wrote:
> On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
>> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
>> <konrad.wilk <at> oracle.com> wrote:
>> > On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
>> >> Dear all,
>> >>
>> >> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
>> >> packet data,  we have statically allocated huge DMA memory buffer ring
>> >> at once instead of calling dma_map_single() per packet.  Considering
>> >> to further reduce the copy overhead between different NIC(port) ring
>> >> while forwarding, one packet from a input NIC(port) will be
>> >> transferred to output NIC(port) with no any copy action.
>> >>
>> >> To satisfy this requirement, the packet memory should be mapped into
>> >> input port and unmapped when leaving input port, then mapped into
>> >> output port and unmapped later.
>> >>
>> >> Whether it's legal to map the same DMA memory into input and output
>> >> port simultaneously? If it's not, then the zero-copy for packet
>> >> forwarding is not feasible?
>> >>
>> >
>> > Did you ever a get a response about this?
>> No.
>
> This is probably because no-one really understands what you're asking.
> As far as mapping memory to PCI devices goes, it's the job of the bridge
> (or the iommu which may or may not be part of the bridge).  A standard
(Continue reading)

Permalink | Reply |
headers
James Bottomley | 5 Jan 2012 17:20

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Thu, 2012-01-05 at 20:40 +0800, Yanfei Wang wrote:
> On Wed, Jan 4, 2012 at 11:59 PM, James Bottomley
> <James.Bottomley <at> hansenpartnership.com> wrote:
> > On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
> >> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
> >> <konrad.wilk <at> oracle.com> wrote:
> >> > On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
> >> >> Dear all,
> >> >>
> >> >> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
> >> >> packet data,  we have statically allocated huge DMA memory buffer ring
> >> >> at once instead of calling dma_map_single() per packet.  Considering
> >> >> to further reduce the copy overhead between different NIC(port) ring
> >> >> while forwarding, one packet from a input NIC(port) will be
> >> >> transferred to output NIC(port) with no any copy action.
> >> >>
> >> >> To satisfy this requirement, the packet memory should be mapped into
> >> >> input port and unmapped when leaving input port, then mapped into
> >> >> output port and unmapped later.
> >> >>
> >> >> Whether it's legal to map the same DMA memory into input and output
> >> >> port simultaneously? If it's not, then the zero-copy for packet
> >> >> forwarding is not feasible?
> >> >>
> >> >
> >> > Did you ever a get a response about this?
> >> No.
> >
> > This is probably because no-one really understands what you're asking.
> > As far as mapping memory to PCI devices goes, it's the job of the bridge
(Continue reading)

Permalink | Reply |
headers
Yanfei Wang | 6 Jan 2012 03:05
Picon

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Fri, Jan 6, 2012 at 12:20 AM, James Bottomley
<James.Bottomley <at> hansenpartnership.com> wrote:
> On Thu, 2012-01-05 at 20:40 +0800, Yanfei Wang wrote:
>> On Wed, Jan 4, 2012 at 11:59 PM, James Bottomley
>> <James.Bottomley <at> hansenpartnership.com> wrote:
>> > On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
>> >> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
>> >> <konrad.wilk <at> oracle.com> wrote:
>> >> > On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
>> >> >> Dear all,
>> >> >>
>> >> >> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
>> >> >> packet data,  we have statically allocated huge DMA memory buffer ring
>> >> >> at once instead of calling dma_map_single() per packet.  Considering
>> >> >> to further reduce the copy overhead between different NIC(port) ring
>> >> >> while forwarding, one packet from a input NIC(port) will be
>> >> >> transferred to output NIC(port) with no any copy action.
>> >> >>
>> >> >> To satisfy this requirement, the packet memory should be mapped into
>> >> >> input port and unmapped when leaving input port, then mapped into
>> >> >> output port and unmapped later.
>> >> >>
>> >> >> Whether it's legal to map the same DMA memory into input and output
>> >> >> port simultaneously? If it's not, then the zero-copy for packet
>> >> >> forwarding is not feasible?
>> >> >>
>> >> >
>> >> > Did you ever a get a response about this?
>> >> No.
>> >
(Continue reading)

Permalink | Reply |
headers
James Bottomley | 6 Jan 2012 17:00

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Fri, 2012-01-06 at 10:05 +0800, Yanfei Wang wrote:
> On Fri, Jan 6, 2012 at 12:20 AM, James Bottomley
> <James.Bottomley <at> hansenpartnership.com> wrote:
> > On Thu, 2012-01-05 at 20:40 +0800, Yanfei Wang wrote:
> >> On Wed, Jan 4, 2012 at 11:59 PM, James Bottomley
> >> <James.Bottomley <at> hansenpartnership.com> wrote:
> >> > On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
> >> >> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
> >> >> <konrad.wilk <at> oracle.com> wrote:
> >> >> > On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
> >> >> >> Dear all,
> >> >> >>
> >> >> >> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
> >> >> >> packet data,  we have statically allocated huge DMA memory buffer ring
> >> >> >> at once instead of calling dma_map_single() per packet.  Considering
> >> >> >> to further reduce the copy overhead between different NIC(port) ring
> >> >> >> while forwarding, one packet from a input NIC(port) will be
> >> >> >> transferred to output NIC(port) with no any copy action.
> >> >> >>
> >> >> >> To satisfy this requirement, the packet memory should be mapped into
> >> >> >> input port and unmapped when leaving input port, then mapped into
> >> >> >> output port and unmapped later.
> >> >> >>
> >> >> >> Whether it's legal to map the same DMA memory into input and output
> >> >> >> port simultaneously? If it's not, then the zero-copy for packet
> >> >> >> forwarding is not feasible?
> >> >> >>
> >> >> >
> >> >> > Did you ever a get a response about this?
> >> >> No.
(Continue reading)

Permalink | Reply |
headers
Don Dutile | 5 Jan 2012 19:48
Picon
Favicon

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On 01/05/2012 07:40 AM, Yanfei Wang wrote:
> On Wed, Jan 4, 2012 at 11:59 PM, James Bottomley
> <James.Bottomley <at> hansenpartnership.com>  wrote:
>> On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
>>> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
>>> <konrad.wilk <at> oracle.com>  wrote:
>>>> On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
>>>>> Dear all,
>>>>>
>>>>> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
>>>>> packet data,  we have statically allocated huge DMA memory buffer ring
>>>>> at once instead of calling dma_map_single() per packet.  Considering
>>>>> to further reduce the copy overhead between different NIC(port) ring
>>>>> while forwarding, one packet from a input NIC(port) will be
>>>>> transferred to output NIC(port) with no any copy action.
>>>>>
>>>>> To satisfy this requirement, the packet memory should be mapped into
>>>>> input port and unmapped when leaving input port, then mapped into
>>>>> output port and unmapped later.
>>>>>
>>>>> Whether it's legal to map the same DMA memory into input and output
>>>>> port simultaneously? If it's not, then the zero-copy for packet
>>>>> forwarding is not feasible?
>>>>>
>>>>
>>>> Did you ever a get a response about this?
>>> No.
>>
>> This is probably because no-one really understands what you're asking.
>> As far as mapping memory to PCI devices goes, it's the job of the bridge
(Continue reading)

Permalink | Reply |
headers
Yanfei Wang | 6 Jan 2012 03:11
Picon

Re: 【Question】Whether it's legal to enable same physical DMA memory mapped for different NIC device?

On Fri, Jan 6, 2012 at 2:48 AM, Don Dutile <ddutile <at> redhat.com> wrote:
> On 01/05/2012 07:40 AM, Yanfei Wang wrote:
>>
>> On Wed, Jan 4, 2012 at 11:59 PM, James Bottomley
>> <James.Bottomley <at> hansenpartnership.com>  wrote:
>>>
>>> On Wed, 2012-01-04 at 10:44 +0800, Yanfei Wang wrote:
>>>>
>>>> On Wed, Jan 4, 2012 at 4:33 AM, Konrad Rzeszutek Wilk
>>>> <konrad.wilk <at> oracle.com>  wrote:
>>>>>
>>>>> On Wed, Dec 07, 2011 at 10:16:40PM +0800, ustc.mail wrote:
>>>>>>
>>>>>> Dear all,
>>>>>>
>>>>>> In NIC driver, to eliminate the  overhead of dma_map_single() for DMA
>>>>>> packet data,  we have statically allocated huge DMA memory buffer ring
>>>>>> at once instead of calling dma_map_single() per packet.  Considering
>>>>>> to further reduce the copy overhead between different NIC(port) ring
>>>>>> while forwarding, one packet from a input NIC(port) will be
>>>>>> transferred to output NIC(port) with no any copy action.
>>>>>>
>>>>>> To satisfy this requirement, the packet memory should be mapped into
>>>>>> input port and unmapped when leaving input port, then mapped into
>>>>>> output port and unmapped later.
>>>>>>
>>>>>> Whether it's legal to map the same DMA memory into input and output
>>>>>> port simultaneously? If it's not, then the zero-copy for packet
>>>>>> forwarding is not feasible?
>>>>>>
(Continue reading)

Permalink | Reply |

Gmane