Hello,

I am looking to add a couple of hooks into OpenBSD to enable automated
install and configuration.
The existing install.site/siteXY.tgz are great, and I use that hook
extensively, but I would like to automate
getting my machines to the point where install.site can be run.

My overall goal is that various attributes of a machine should be
specified in a configuration management system, and the key identifier
of
a machine is its mac address, which should be used to look up
everything about the machine.

I've given this some thought, and here are the minimum two hooks in
OpenBSD that I think are required:

1) pxeboot should request a series of specific "boot.conf" files:
pxeboot should be modified to first request boot.conf.MAC-ADDRESS
(the mac address of the machine), if that file is found via tftp, it
is used.
If not, pxeboot should then request
boot.conf.OpenBSD-Major.Minor-machine (eg boot.conf.OpenBSD-4.3-amd64
), if found, use that, if not
pxeboot should finally request boot.conf (as it does today). The
advantage of this is that one can specify very specific information
about
which version of bsd.rd should be used to install the machine,
and this can be detailed down to the individual machine via the mac
address.
If you don't need that much granularity, the
boot.conf.OpenBSD-Major.Minor-machine allows proper installation for a
broader class of machines
and releases than is supported today. BTW, ISC-DHCP already
provides the ability to specify a unique pxeboot file for a given mac
address, so
that part is already complete, the snag is that all OpenBSD
versions of pxeboot request the same etc/boot.conf file!!!

2) bsd.rd should be able to request (over the network) the install
script to be used.
Currently, bsd.rd executes .profile
(src/distrib/miniroot/dot.profile) to ask the user what to do
(install, upgrade, sh).
Before that, I'd like dot.profile to attempt to download an
install/upgrade script from the network, if it finds such a script, it
automatically
would use that, and begin executing it. I am not sure exactly how
to specify the name of this script, ideally via a config line in the
boot.conf file
that pxeboot obtains, but if that is not easy to do, I am open to
other ideas. We could hard code this URL, but ideally we'd allow some
granularity
like in the boot.conf case above. (e.g.
http://openbsd-install/mac-address/custom-install.sh,
http://openbsd-install/4.3/amd64/custom-install.sh )

That's it. With these two hooks, it would be relatively
straightforward to pxeboot a new machine, and the configuration
management system
would interact with the local infrastructure (DHCP, tftp,
configuration web server, etc) to bring a machine up, on the network,
install the OS, and
presumably finally executing the install/upgrade.site script at the end.

I'm not a hard core systems programmer, so I'm looking to hire someone
to develop the above.
Ideally we could implement it so that current behavior is preserved (I
believe my proposal above falls back to existing behavior), and
hopefully we could get these additions accepted for some future
OpenBSD release.

If you might be interested in implementing the proposal above, please
contact me.

If you have a better idea of how to implement what I have proposed
above, I'd love to hear your ideas also.

Best regards,

Don

On 3 July 2008 at 13:17, "Don Jackson" <don.jackson@gmail.com> wrote:

> Hello,
>
> I am looking to add a couple of hooks into OpenBSD to enable automated
> install and configuration.

[snip]

> My overall goal is that various attributes of a machine should be
> specified in a configuration management system, and the key identifier
> of
> a machine is its mac address, which should be used to look up
> everything about the machine.
>
> I've given this some thought, and here are the minimum two hooks in
> OpenBSD that I think are required:
>
> 1) pxeboot should request a series of specific "boot.conf" files:
> pxeboot should be modified to first request boot.conf.MAC-ADDRESS

> (or)

> boot.conf.OpenBSD-Major.Minor-machine (eg boot.conf.OpenBSD-4.3-amd64

> (or)

> pxeboot should finally request boot.conf (as it does today). The

> (or)

> boot.conf.OpenBSD-Major.Minor-machine

Such a thing would make upgrades a lot easier. However you can
workaround this using dhcpd.conf. (I've played with this idea for years
now.)

For instance I have .../$OSREV/$ARCH/tftpboot/$MODEL/boot.conf

($MODEL is my own name for a given system, e.g. 'x4100m2'. This is
required since some machines want console on com0, some on com1... and
others need a kernel with devices disabled, etc.)

My boot server has multiple network aliases (IP addresses) bound to
it, and each IP is in inetd.conf, for tftpd. Each tftpd line points
to different tftpboot directories. Inside each directory is a custom
boot.conf for that machine.

When it's time to change out the OS on a system (I frequently
switch between i386 and amd64 on my x4100m2's) you just change the
'next-server' dhcp option and reboot the box.

It's a lot of infrastructure to work around a simple problem... but it
works.

The custom bsd.rd is chosen from that boot.conf
(e.g. 'boot amd64-x4100m2.rd').

> 2) bsd.rd should be able to request (over the network) the install
> script to be used.

I have a custom ramdisk image to do that, and the install script to do
it as well. Machine boots, identifies itself, and installs in less than
10 minutes (a good 5-6 minutes of that is due to newfs). Works on i386,
amd64, and sparc64.

In the last two weeks I ripped that install script apart, pulling out
all the extra things it required on the ramdisk, and kind-of dumped the
idea of the custom ramdisk. Why? It is too much trouble to maintain
for 3 (soon maybe 4) architectures, and tracking -current, and -stable.
I've simplified it all to this:

- pxeboot/netboot and type what I need at the boot> prompt
- because it netbooted, the network is active so I just type
'ftp http://install-server/autoinstall'
- ./autoinstall

Done. It takes about a minute to do this and saves hours of work
building and maintaining a custom ramdisk.

All that said... it would be a fantastic thing to add to the stock
ramdisk image and I agree with you completely :-)

After having implemented it, IMO it's just too much trouble to do a
custom one.

* Don Jackson <don.jackson@gmail.com> [2008-07-03 22:22]:
> 1) pxeboot should request a series of specific "boot.conf" files:
> pxeboot should be modified to first request boot.conf.MAC-ADDRESS
> (the mac address of the machine), if that file is found via tftp, it
> is used.

we discussed that before and I even think there was a diff around
implementing this. it was held back because of other issues in pxeboot
which have been resolved since then afaicr... I would really love to
see that.

--
Henning Brauer, hb@bsws.de, henning@openbsd.org
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting - Hamburg & Amsterdam

On Fri, 2008-07-04 at 02:37 -0400, Tom McLaughlin wrote:
> On Thu, 2008-07-03 at 13:17 -0700, Don Jackson wrote:
> > Hello,
> >
> > I am looking to add a couple of hooks into OpenBSD to enable automated
> > install and configuration.
> > The existing install.site/siteXY.tgz are great, and I use that hook
> > extensively, but I would like to automate
> > getting my machines to the point where install.site can be run.
> >
> > My overall goal is that various attributes of a machine should be
> > specified in a configuration management system, and the key identifier
> > of
> > a machine is its mac address, which should be used to look up
> > everything about the machine.
> >
> > I've given this some thought, and here are the minimum two hooks in
> > OpenBSD that I think are required:
> >
> > 1) pxeboot should request a series of specific "boot.conf" files:
> > pxeboot should be modified to first request boot.conf.MAC-ADDRESS
> > (the mac address of the machine), if that file is found via tftp, it
> > is used.
> > If not, pxeboot should then request
> > boot.conf.OpenBSD-Major.Minor-machine (eg boot.conf.OpenBSD-4.3-amd64
> > ), if found, use that, if not
> > pxeboot should finally request boot.conf (as it does today). The
> > advantage of this is that one can specify very specific information
> > about
> > which version of bsd.rd should be used to install the machine,
> > and this can be detailed down to the individual machine via the mac
> > address.
> > If you don't need that much granularity, the
> > boot.conf.OpenBSD-Major.Minor-machine allows proper installation for a
> > broader class of machines
> > and releases than is supported today. BTW, ISC-DHCP already
> > provides the ability to specify a unique pxeboot file for a given mac
> > address, so
> > that part is already complete, the snag is that all OpenBSD
> > versions of pxeboot request the same etc/boot.conf file!!!
> >
>
> I use pxelinux so when stray hardware follows me home I can just boot it
> and pick a random OS from a menu to install. I ran across a slightly
> different idea in a blog post below while trying to figure out how to
> make multiple version and arch combinations of Open play nice with
> pxelinux.
>
> http://www.thegibson.org/blog/archives/10
> http://www.thegibson.org/blog/files/...oot_i386.patch
> http://www.thegibson.org/blog/files/...ot_amd64.patch
>
> tom
>

And after I hit send I realize all that patch does is make pxeboot play
nicer with pxelinux when the original idea would solve my problems
without needing pxelinux. Never mind the noise. It's late here.

tom

--
| tmclaugh at sdf.lonestar.org tmclaugh at FreeBSD.org |
| FreeBSD http://www.FreeBSD.org |

On Thu, 3 Jul 2008, Don Jackson wrote:

> Hello,
>
> I am looking to add a couple of hooks into OpenBSD to enable automated
> install and configuration.
> The existing install.site/siteXY.tgz are great, and I use that hook
> extensively, but I would like to automate
> getting my machines to the point where install.site can be run.
>
> My overall goal is that various attributes of a machine should be
> specified in a configuration management system, and the key identifier
> of
> a machine is its mac address, which should be used to look up
> everything about the machine.
>
> I've given this some thought, and here are the minimum two hooks in
> OpenBSD that I think are required:
>
> 1) pxeboot should request a series of specific "boot.conf" files:
> pxeboot should be modified to first request boot.conf.MAC-ADDRESS
> (the mac address of the machine), if that file is found via tftp, it
> is used.

Another way would be to allow specification of the boot.conf path using
a DHCP option. GRUB uses option 150 for this, pxelinux (which needs to be
different of course) uses option 209.

-d

Don Jackson <don.jackson@gmail.com> wrote:
> 1) pxeboot should request a series of specific "boot.conf"
> files: pxeboot should be modified to first request
> boot.conf.MAC-ADDRESS (the mac address of the machine), if that
> file is found via tftp, it is used.

We use pxelinux to display a friendly menu of different PXE boot
options such as Linux, BIOS Flash, MEM Test and OpenBSD.

My theoretic solution would be for the OpenBSD pxeboot to look
for a boot.conf from the path where it was loaded.

Example:
pxeboot came from tftp:/openbsd/i386_obsd44/pxeboot then look for
tftp:/openbsd/i386_obsd44/etc/boot.conf and if not found try
tftp:/etc/boot.conf.

Then I could use "set image" in boot.conf to select
tftp:/openbsd/i386_obsd44/bsd.bigmem-20080422.rd


An alternative would be that pxeboot took some kind of command
line option to specify a boot.conf.

On Thu, 2008-07-03 at 13:17 -0700, Don Jackson wrote:
> Hello,
>
> I am looking to add a couple of hooks into OpenBSD to enable automated
> install and configuration.
> The existing install.site/siteXY.tgz are great, and I use that hook
> extensively, but I would like to automate
> getting my machines to the point where install.site can be run.
>
> My overall goal is that various attributes of a machine should be
> specified in a configuration management system, and the key identifier
> of
> a machine is its mac address, which should be used to look up
> everything about the machine.
>
> I've given this some thought, and here are the minimum two hooks in
> OpenBSD that I think are required:
>
> 1) pxeboot should request a series of specific "boot.conf" files:
> pxeboot should be modified to first request boot.conf.MAC-ADDRESS
> (the mac address of the machine), if that file is found via tftp, it
> is used.
> If not, pxeboot should then request
> boot.conf.OpenBSD-Major.Minor-machine (eg boot.conf.OpenBSD-4.3-amd64
> ), if found, use that, if not
> pxeboot should finally request boot.conf (as it does today). The
> advantage of this is that one can specify very specific information
> about
> which version of bsd.rd should be used to install the machine,
> and this can be detailed down to the individual machine via the mac
> address.
> If you don't need that much granularity, the
> boot.conf.OpenBSD-Major.Minor-machine allows proper installation for a
> broader class of machines
> and releases than is supported today. BTW, ISC-DHCP already
> provides the ability to specify a unique pxeboot file for a given mac
> address, so
> that part is already complete, the snag is that all OpenBSD
> versions of pxeboot request the same etc/boot.conf file!!!
>

I use pxelinux so when stray hardware follows me home I can just boot it
and pick a random OS from a menu to install. I ran across a slightly
different idea in a blog post below while trying to figure out how to
make multiple version and arch combinations of Open play nice with
pxelinux.

http://www.thegibson.org/blog/archives/10
http://www.thegibson.org/blog/files/...oot_i386.patch
http://www.thegibson.org/blog/files/...ot_amd64.patch

tom

--
| tmclaugh at sdf.lonestar.org tmclaugh at FreeBSD.org |
| FreeBSD http://www.FreeBSD.org |

On Thu, Jul 3, 2008 at 5:13 PM, Henning Brauer
<lists-openbsdtech@bsws.de> wrote:
> * Don Jackson <don.jackson@gmail.com> [2008-07-03 22:22]:
>> 1) pxeboot should request a series of specific "boot.conf" files:
>> pxeboot should be modified to first request boot.conf.MAC-ADDRESS
>> (the mac address of the machine), if that file is found via tftp, it
>> is used.
>
> we discussed that before and I even think there was a diff around
> implementing this. it was held back because of other issues in pxeboot
> which have been resolved since then afaicr... I would really love to
> see that.

yes, i have a diff that does this - i'll have dig it out of my
archives and update it... pxeboot has changed since i last hacked on
this (2 years ago).

CK

--
GDB has a 'break' feature; why doesn't it have 'fix' too?

On Thu, Jul 03, 2008 at 01:17:43PM -0700, Don Jackson wrote:

> My overall goal is that various attributes of a machine should be
> specified in a configuration management system, and the key identifier
> of
> a machine is its mac address, which should be used to look up
> everything about the machine.

Being able to provide answers to the questions during install in
an automated fashion was something, I wanted to have a while ago and
implemented it in the following way:

- installer script checks wether we netboot'ed, if so, tries to download
a config file via ftp
- if the config file is found, we do an auto install and download a script
as well
- if the config file is not found, we do the manual install

The config file is looked up via MAC or IP. It is also possible to
put a 'default.conf' in the ftp directory, which will then be used
for machines not having a configuration there. (but if that is present,
it will not jump to manual mode at all)

The config file being downloaded is a bunch of shell variables that
carry the values needed for the complete install. I tried to keep the
changes to the installer (miniroot/install.{sh|sub}) to the bare minimum,
which is why the script mentioned up above is downloaded via ftp to
bring along some helpers.

I've also wanted to avoid crutches like custom bsd.rd for each and every
machine, while still being able to provide different partitioning, sets-
to-install, etc. etc. via a config.

Currently i've not added anything to install packages, since we do that
via siteXY-hostname.tgz.

felix