Hi,

I wanted to clearly understand the fundamental difference, if any,
between
TCP Multidata Transmit (MDT) which was available in some later
versions of Solaris 9
and LSO, which was recently added to Solaris 10 Update 4.

Is there a good document somewhere, or some portions of the source
code,
(as available in the ON src in OpenSolaris) that I can refer to for
this ?

Thanks a lot,
--Debashis

Debashis Dutt <debdut@gmail.com> writes:
> I wanted to clearly understand the fundamental difference, if any,
> between
> TCP Multidata Transmit (MDT) which was available in some later
> versions of Solaris 9
> and LSO, which was recently added to Solaris 10 Update 4.

MDT creates specially-formatted STREAMS messages that have multiple
TCP/IP packets within them, so that the driver gets a single put(9E)
call and can just send the packets normally. LSO fakes a large MTU
and has the driver resegment things.

MDT has the advantage that all of the header manipulation is done up
in the IP level rather than the driver, so the manipulation is known
to be correct and equivalent to "normal" operation. This means that
things like IPsec work fine with it.

LSO has the advantage that there are more devices out there designed
to work with it, but because the driver is playing a more active role,
it _may_ have an impact on correctness.

Frankly, if it were my network, I wouldn't do either. Complexity is
the enemy. I'm not wild about performance enhancements that add
complicated new execution paths.

> Is there a good document somewhere, or some portions of the source
> code,
> (as available in the ON src in OpenSolaris) that I can refer to for
> this ?

Searching for routines with "mdt" in the name might help.

--
James Carlson, Solaris Networking <james.d.carlson@sun.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

On Oct 25, 6:01 am, James Carlson <james.d.carl...@sun.com> wrote:
> Debashis Dutt <deb...@gmail.com> writes:
> > I wanted to clearly understand the fundamental difference, if any,
> > between
> > TCP Multidata Transmit (MDT) which was available in some later
> > versions of Solaris 9
> > and LSO, which was recently added to Solaris 10 Update 4.
>
> MDT creates specially-formatted STREAMS messages that have multiple
> TCP/IP packets within them, so that the driver gets a single put(9E)
> call and can just send the packets normally. LSO fakes a large MTU
> and has the driver resegment things.
>
> MDT has the advantage that all of the header manipulation is done up
> in the IP level rather than the driver, so the manipulation is known
> to be correct and equivalent to "normal" operation. This means that
> things like IPsec work fine with it.
>
> LSO has the advantage that there are more devices out there designed
> to work with it, but because the driver is playing a more active role,
> it _may_ have an impact on correctness.
>
> Frankly, if it were my network, I wouldn't do either. Complexity is
> the enemy. I'm not wild about performance enhancements that add
> complicated new execution paths.
>
> > Is there a good document somewhere, or some portions of the source
> > code,
> > (as available in the ON src in OpenSolaris) that I can refer to for
> > this ?
>
> Searching for routines with "mdt" in the name might help.
>
> --
> James Carlson, Solaris Networking <james.d.carl...@sun.com>
> Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
> MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

Thanks a lot James for clearing this up.

So from what I understand, MDT is completely internal to the stack,
and a NIC driver
has nothing else to do, other than reading multiple TCP segments from
the same STREAMS
message and send it on its way to the hardware. TCP header
manipulation has already been
completed in the stack.

Is this correct ?

Thanks
--Debashis

On Oct 25, 6:01 am, James Carlson <james.d.carl...@sun.com> wrote:
> Debashis Dutt <deb...@gmail.com> writes:
> > I wanted to clearly understand the fundamental difference, if any,
> > between
> > TCP Multidata Transmit (MDT) which was available in some later
> > versions of Solaris 9
> > and LSO, which was recently added to Solaris 10 Update 4.
>
> MDT creates specially-formatted STREAMS messages that have multiple
> TCP/IP packets within them, so that the driver gets a single put(9E)
> call and can just send the packets normally. LSO fakes a large MTU
> and has the driver resegment things.
>
> MDT has the advantage that all of the header manipulation is done up
> in the IP level rather than the driver, so the manipulation is known
> to be correct and equivalent to "normal" operation. This means that
> things like IPsec work fine with it.
>
> LSO has the advantage that there are more devices out there designed
> to work with it, but because the driver is playing a more active role,
> it _may_ have an impact on correctness.
>
> Frankly, if it were my network, I wouldn't do either. Complexity is
> the enemy. I'm not wild about performance enhancements that add
> complicated new execution paths.
>
> > Is there a good document somewhere, or some portions of the source
> > code,
> > (as available in the ON src in OpenSolaris) that I can refer to for
> > this ?
>
> Searching for routines with "mdt" in the name might help.
>
> --
> James Carlson, Solaris Networking <james.d.carl...@sun.com>
> Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
> MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

Also is there any time frame when the GLDv3 interface will be made
public ?

There are some features such as Link Aggregation which GLDv2 does not
provide,
but all the new NIC drivers want to support it.

Thanks
--Debashis

Debashis Dutt <debdut@gmail.com> writes:
> Thanks a lot James for clearing this up.
>
> So from what I understand, MDT is completely internal to the stack,
> and a NIC driver
> has nothing else to do, other than reading multiple TCP segments from
> the same STREAMS
> message and send it on its way to the hardware.

Right. The driver *does* need to be changed to support M_MULTIDATA,
but that's basically it.

> TCP header
> manipulation has already been
> completed in the stack.
>
> Is this correct ?

Yes.

--
James Carlson, Solaris Networking <james.d.carlson@sun.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

Debashis Dutt <debdut@gmail.com> writes:
> Also is there any time frame when the GLDv3 interface will be made
> public ?

I don't know. networking-discuss@opensolaris.org would be the right
place to ask.

> There are some features such as Link Aggregation which GLDv2 does not
> provide,
> but all the new NIC drivers want to support it.

Right. You might also want to check out OpenSolaris project
"Clearview," which is extending many of the GLDv3 features out to
older drivers.

Plus, there are several folks (Garrett D'Amore in particular) who are
converting old drivers over. Visit www.opensolaris.org to find out
what what's going on.

--
James Carlson, Solaris Networking <james.d.carlson@sun.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

On Oct 26, 3:15 pm, James Carlson <james.d.carl...@sun.com> wrote:
> Debashis Dutt <deb...@gmail.com> writes:
> > Also is there any time frame when the GLDv3 interface will be made
> > public ?
>
> I don't know. networking-disc...@opensolaris.org would be the right
> place to ask.
>
> > There are some features such as Link Aggregation which GLDv2 does not
> > provide,
> > but all the new NIC drivers want to support it.
>
> Right. You might also want to check out OpenSolaris project
> "Clearview," which is extending many of the GLDv3 features out to
> older drivers.
>
> Plus, there are several folks (Garrett D'Amore in particular) who are
> converting old drivers over. Visitwww.opensolaris.orgto find out
> what what's going on.
>
> --
> James Carlson, Solaris Networking <james.d.carl...@sun.com>
> Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
> MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

Thanks so much for your input, James.

On Oct 26, 3:48 pm, Debashis Dutt <deb...@gmail.com> wrote:
> On Oct 26, 3:15 pm, James Carlson <james.d.carl...@sun.com> wrote:
>
>
>
> > Debashis Dutt <deb...@gmail.com> writes:
> > > Also is there any time frame when the GLDv3 interface will be made
> > > public ?
>
> > I don't know. networking-disc...@opensolaris.org would be the right
> > place to ask.
>
> > > There are some features such as Link Aggregation which GLDv2 does not
> > > provide,
> > > but all the new NIC drivers want to support it.
>
> > Right. You might also want to check out OpenSolaris project
> > "Clearview," which is extending many of the GLDv3 features out to
> > older drivers.
>
> > Plus, there are several folks (Garrett D'Amore in particular) who are
> > converting old drivers over. Visitwww.opensolaris.orgtofind out
> > what what's going on.
>
> > --
> > James Carlson, Solaris Networking <james.d.carl...@sun.com>
> > Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
> > MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
>
> Thanks so much for your input, James.

I am not sure that I can mail to networking-discuss@opensolaris.org.
Looks like any external post to this group, has to be approved by
the moderator. and my mail has not yet been approved.

Any other alternatives where I can get some answers ?

Thanks
--Debashis

In article <1193769164.788176.146380@t8g2000prg.googlegroups. com>,
Debashis Dutt <debdut@gmail.com> wrote:
>I am not sure that I can mail to networking-discuss@opensolaris.org.
>Looks like any external post to this group, has to be approved by
>the moderator. and my mail has not yet been approved.

Its a mailing list.

Did you subscribe?

Append -request to the alias.

John
groenveld@acm.org

On Oct 30, 11:52 am, groen...@cse.psu.edu (John Groenveld) wrote:
> In article <1193769164.788176.146...@t8g2000prg.googlegroups. com>,
> Debashis Dutt <deb...@gmail.com> wrote:
>
> >I am not sure that I can mail to networking-disc...@opensolaris.org.
> >Looks like any external post to this group, has to be approved by
> >the moderator. and my mail has not yet been approved.
>
> Its a mailing list.
>
> Did you subscribe?
>
> Append -request to the alias.
>
> John
> groenv...@acm.org

I did send in a request to subscribe.
Thanks