Hey hey all..

I'm using a DB2 8.2 in an 64bit environment in SLES9.
Server is a HP Proliant DL385

For storage I'm using a NetApp Fas250 filer.

using a few NFS volumes to store the database.


The one that worries me a bit is a volume meant to store the instance
user(s) on.

I have an nfs mount point /data/instance/

after performing a db2 install, the users db2inst, dasusr and db2fenc
are created on that mount point... No problem.

But for some reason I'm not able to create an instance.
When I try to call db2icrt -w 64 -u db2fenc db2inst I get the following
error message:

DBI1281E The database manager configuration file could not be
initialized.

When I copy the db2inst, dasusr and db2fenc dir to the same mount point,
only without the actual mount in place (meaning that I have those dirs
on the same place, but now locally stored) I can create the instance
without problems.

So obviously there's something wrong with my NFS settings.

I have mounted the share over TCP in rw (remember, I can create the user
directories).

Are there any other parameters required to ensure the shares are
correctly used by db2?

Things I have checked:
- Free space on the share
- Permissions on the share
- ensured the share is mounted over TCP instead of UDP
- default settings are in place

Any thoughts on this matter?

Thanks


-R-

This sounds like the db2inst userid does not have the authority to write
to the mounted file system. Have you verified that db2inst can write to
its directories on the mounted file system?

Phil Sherman



Jurgen Haan wrote:
> Hey hey all..
>
> I'm using a DB2 8.2 in an 64bit environment in SLES9.
> Server is a HP Proliant DL385
>
> For storage I'm using a NetApp Fas250 filer.
>
> using a few NFS volumes to store the database.
>
>
> The one that worries me a bit is a volume meant to store the instance
> user(s) on.
>
> I have an nfs mount point /data/instance/
>
> after performing a db2 install, the users db2inst, dasusr and db2fenc
> are created on that mount point... No problem.
>
> But for some reason I'm not able to create an instance.
> When I try to call db2icrt -w 64 -u db2fenc db2inst I get the following
> error message:
>
> DBI1281E The database manager configuration file could not be
> initialized.
>
> When I copy the db2inst, dasusr and db2fenc dir to the same mount point,
> only without the actual mount in place (meaning that I have those dirs
> on the same place, but now locally stored) I can create the instance
> without problems.
>
> So obviously there's something wrong with my NFS settings.
>
> I have mounted the share over TCP in rw (remember, I can create the user
> directories).
>
> Are there any other parameters required to ensure the shares are
> correctly used by db2?
>
> Things I have checked:
> - Free space on the share
> - Permissions on the share
> - ensured the share is mounted over TCP instead of UDP
> - default settings are in place
>
> Any thoughts on this matter?
>
> Thanks
>
>
> -R-

The server has anonyous rw access enabled, so everyone should be able to
write.

Just to be sure I changed to user db2inst and created a file on the
share. No problem. I can create, alter and delete files on it.

Phil Sherman wrote:
> This sounds like the db2inst userid does not have the authority to write
> to the mounted file system. Have you verified that db2inst can write to
> its directories on the mounted file system?
>
> Phil Sherman
>
>
>
> Jurgen Haan wrote:
>
>> Hey hey all..
>>
>> I'm using a DB2 8.2 in an 64bit environment in SLES9.
>> Server is a HP Proliant DL385
>>
>> For storage I'm using a NetApp Fas250 filer.
>>
>> using a few NFS volumes to store the database.
>>
>>
>> The one that worries me a bit is a volume meant to store the instance
>> user(s) on.
>>
>> I have an nfs mount point /data/instance/
>>
>> after performing a db2 install, the users db2inst, dasusr and db2fenc
>> are created on that mount point... No problem.
>>
>> But for some reason I'm not able to create an instance.
>> When I try to call db2icrt -w 64 -u db2fenc db2inst I get the
>> following error message:
>>
>> DBI1281E The database manager configuration file could not be
>> initialized.
>>
>> When I copy the db2inst, dasusr and db2fenc dir to the same mount
>> point, only without the actual mount in place (meaning that I have
>> those dirs on the same place, but now locally stored) I can create the
>> instance without problems.
>>
>> So obviously there's something wrong with my NFS settings.
>>
>> I have mounted the share over TCP in rw (remember, I can create the
>> user directories).
>>
>> Are there any other parameters required to ensure the shares are
>> correctly used by db2?
>>
>> Things I have checked:
>> - Free space on the share
>> - Permissions on the share
>> - ensured the share is mounted over TCP instead of UDP
>> - default settings are in place
>>
>> Any thoughts on this matter?
>>
>> Thanks
>>
>>
>> -R-

Jurgen Haan wrote:

> The server has anonyous rw access enabled, so everyone should be able to
> write.
>
> Just to be sure I changed to user db2inst and created a file on the
> share. No problem. I can create, alter and delete files on it.

I remember dimly that there were some issues with DB2 databases on NFS. I
don't remember any details, but you should find something via Google.

Also, the instances are created by "root". So root needs to have _all_
privileges on the NFS volume. In particular, root must be able to create
files, change owners and permissions. Some suid-files are created as well,
and those should work as intended if your NFS configuration is properly set
up. You don't use something like root-squash by any chance?

--
Knut Stolze
DB2 Information Integration Development
IBM Germany

Knut Stolze wrote:
> Jurgen Haan wrote:
>
>
>>The server has anonyous rw access enabled, so everyone should be able to
>>write.
>>
>>Just to be sure I changed to user db2inst and created a file on the
>>share. No problem. I can create, alter and delete files on it.
>
>
> I remember dimly that there were some issues with DB2 databases on NFS. I
> don't remember any details, but you should find something via Google.
>
> Also, the instances are created by "root". So root needs to have _all_
> privileges on the NFS volume. In particular, root must be able to create
> files, change owners and permissions. Some suid-files are created as well,
> and those should work as intended if your NFS configuration is properly set
> up. You don't use something like root-squash by any chance?
>

Nope.
NFS is configure quite dumb.
Everyone has full (root) access on the share.. Meaning they are not user
root, but they can do anything they would normally by able to. The
permissions are determined by the authentication on the Linux machine,
not by any mechanism the NFS server. I am able to create files, delete
files, alter files, create directories, delete directories, alter
directories, chown, chmod, create pipes, delete pipes, make symlinks,
delete symlinks. In short, everything.

I'm starting to think this isn't so much a NFS settings problem, but
rather a clash between DB2 and the NFS client from the kernel.
Perhaps some ioctl that is not supported on NFS to gain some direct
access to the volume.

the straightforward answer: no rootsquash.
here's the share:

10.1.4.1:/vol/db2_inst /data/instance nfs
proto=tcp,rsize=8192,wsize=8192,hard,retrans=6,noc to

initially it was:

10.1.4.1:/vol/db2_inst /data/instance nfs proto=tcp

same effect.

perhaps I should try NFSv4.

How does the NFS machine keep track of the users creating the files?
Standard Linux security uses the user's userid number and group number
to identify ownership of files. When I've used NFS, I've had to be
careful to make sure that the userid/group numbers on the NFS system
match the ones on the client accessing the server.

There are mechanismes available to allow a single server to perform all
authentication of users in a network. I don't know if you are using one
of these.

Phil Sherman



Jurgen Haan wrote:
> Knut Stolze wrote:
>
>> Jurgen Haan wrote:
>>
>>
>>> The server has anonyous rw access enabled, so everyone should be able to
>>> write.
>>>
>>> Just to be sure I changed to user db2inst and created a file on the
>>> share. No problem. I can create, alter and delete files on it.
>>
>>
>>
>> I remember dimly that there were some issues with DB2 databases on
>> NFS. I
>> don't remember any details, but you should find something via Google.
>>
>> Also, the instances are created by "root". So root needs to have _all_
>> privileges on the NFS volume. In particular, root must be able to create
>> files, change owners and permissions. Some suid-files are created as
>> well,
>> and those should work as intended if your NFS configuration is
>> properly set
>> up. You don't use something like root-squash by any chance?
>>
>
> Nope.
> NFS is configure quite dumb.
> Everyone has full (root) access on the share.. Meaning they are not user
> root, but they can do anything they would normally by able to. The
> permissions are determined by the authentication on the Linux machine,
> not by any mechanism the NFS server. I am able to create files, delete
> files, alter files, create directories, delete directories, alter
> directories, chown, chmod, create pipes, delete pipes, make symlinks,
> delete symlinks. In short, everything.
>
> I'm starting to think this isn't so much a NFS settings problem, but
> rather a clash between DB2 and the NFS client from the kernel.
> Perhaps some ioctl that is not supported on NFS to gain some direct
> access to the volume.
>
> the straightforward answer: no rootsquash.
> here's the share:
>
> 10.1.4.1:/vol/db2_inst /data/instance nfs
> proto=tcp,rsize=8192,wsize=8192,hard,retrans=6,noc to
>
> initially it was:
>
> 10.1.4.1:/vol/db2_inst /data/instance nfs proto=tcp
>
> same effect.
>
> perhaps I should try NFSv4.

The server is a dumb server... atm it can be seen as a jbod.
It just provides anonymous nfs shares. The server does not keep track of
users. It's a dedicated server btw, a custom server with custom
software. The server does not operate on a user base. hence the
anonymous shares. The user accounts do not authenticate to the server.

The NFS server itself does not access the volumes, it does not need to
care about their uid or gid. db2inst:db2grp is 101:101 on my client
system. Files writen on the nfs have ownership 101:101. So the files
keep their correct ownership.

In the current situation it's only one client that is accessing the
server. That client is the *only* machine that can access those volumes
and alter their contents. (not even the server can do so)

The directories db2inst, db2fenc and dasadm are created with the correct
uid and gid's. And when I su to one of those users, I can create and
alter files in name of that user. So afaik that part is operating
correctly. But for some reason, the db2icrt tool thinks different of
that share.

What tasks does the db2icrt exactly perform?
I dragged db2icrt through a strace to monitor what system calls are
being performed. But the output gives me little valuable information.
(too much information of little significance).
When I know what steps the db2icrt tool performs, I can try those aswell
to see where the error occurs.

-R-

Phil Sherman wrote:
> How does the NFS machine keep track of the users creating the files?
> Standard Linux security uses the user's userid number and group number
> to identify ownership of files. When I've used NFS, I've had to be
> careful to make sure that the userid/group numbers on the NFS system
> match the ones on the client accessing the server.
>
> There are mechanismes available to allow a single server to perform all
> authentication of users in a network. I don't know if you are using one
> of these.
>
> Phil Sherman
>
>
>
> Jurgen Haan wrote:
>
>> Knut Stolze wrote:
>>
>>> Jurgen Haan wrote:
>>>
>>>
>>>> The server has anonyous rw access enabled, so everyone should be
>>>> able to
>>>> write.
>>>>
>>>> Just to be sure I changed to user db2inst and created a file on the
>>>> share. No problem. I can create, alter and delete files on it.
>>>
>>>
>>>
>>>
>>> I remember dimly that there were some issues with DB2 databases on
>>> NFS. I
>>> don't remember any details, but you should find something via Google.
>>>
>>> Also, the instances are created by "root". So root needs to have _all_
>>> privileges on the NFS volume. In particular, root must be able to
>>> create
>>> files, change owners and permissions. Some suid-files are created as
>>> well,
>>> and those should work as intended if your NFS configuration is
>>> properly set
>>> up. You don't use something like root-squash by any chance?
>>>
>>
>> Nope.
>> NFS is configure quite dumb.
>> Everyone has full (root) access on the share.. Meaning they are not
>> user root, but they can do anything they would normally by able to.
>> The permissions are determined by the authentication on the Linux
>> machine, not by any mechanism the NFS server. I am able to create
>> files, delete files, alter files, create directories, delete
>> directories, alter directories, chown, chmod, create pipes, delete
>> pipes, make symlinks, delete symlinks. In short, everything.
>>
>> I'm starting to think this isn't so much a NFS settings problem, but
>> rather a clash between DB2 and the NFS client from the kernel.
>> Perhaps some ioctl that is not supported on NFS to gain some direct
>> access to the volume.
>>
>> the straightforward answer: no rootsquash.
>> here's the share:
>>
>> 10.1.4.1:/vol/db2_inst /data/instance nfs
>> proto=tcp,rsize=8192,wsize=8192,hard,retrans=6,noc to
>>
>> initially it was:
>>
>> 10.1.4.1:/vol/db2_inst /data/instance nfs proto=tcp
>>
>> same effect.
>>
>> perhaps I should try NFSv4.

Jurgen Haan wrote:

> What tasks does the db2icrt exactly perform?
> I dragged db2icrt through a strace to monitor what system calls are
> being performed. But the output gives me little valuable information.
> (too much information of little significance).
> When I know what steps the db2icrt tool performs, I can try those aswell
> to see where the error occurs.

As far as I'm aware, putting anything DB2 on an NFS share is officially
unsupported by IBM. So you're kind of in a pickle. Normally, I'd suggest
opening a PMR, but given that you don't have a problem when creating the
instance locally (the supported scenario), that seems kind of pointless.

The steps db2icrt performs should be actually not that difficult to figure
out - db2icrt itself is a shell script. It then calls other binaries to do
certain work, and the problem you're hitting is inside those binaries. But
as long as you're stracing anyway, you could go modify the shell scripts to
turn on strace at exactly the right time. Hint: the problem is probably
occuring in db2idbm (another shell script) in the very first call to "db2
update dbm cfg using ...". This should reduce the extraneous output from
strace considerably.

Darin McBride wrote:
> Jurgen Haan wrote:
>
>
>>What tasks does the db2icrt exactly perform?
>>I dragged db2icrt through a strace to monitor what system calls are
>>being performed. But the output gives me little valuable information.
>>(too much information of little significance).
>>When I know what steps the db2icrt tool performs, I can try those aswell
>>to see where the error occurs.
>
>
> As far as I'm aware, putting anything DB2 on an NFS share is officially
> unsupported by IBM. So you're kind of in a pickle. Normally, I'd suggest
> opening a PMR, but given that you don't have a problem when creating the
> instance locally (the supported scenario), that seems kind of pointless.
>
> The steps db2icrt performs should be actually not that difficult to figure
> out - db2icrt itself is a shell script. It then calls other binaries to do
> certain work, and the problem you're hitting is inside those binaries. But
> as long as you're stracing anyway, you could go modify the shell scripts to
> turn on strace at exactly the right time. Hint: the problem is probably
> occuring in db2idbm (another shell script) in the very first call to "db2
> update dbm cfg using ...". This should reduce the extraneous output from
> strace considerably.
>

Omg... It never occured to me those little programs could be shellscripts :P

Checking them out atm..

Btw.. We're using a NetApp filer. These are very cool storage devices
with enormous performance. This is due to a large memory buffer in which
write actions are performed. The device arranges the blocks in a raid 4
layout in the buffer and flushes them to the disk in a single write.
As far as the database is concerned, it's operating on a ramdisk.

The netapp also has a snapshot option so it can store old files in their
current state in a separate filesystem. You can use both NFS and iSCSI
on the device. When using iSCSI you have to format the device you get
when you activate the HBA. Downside of iSCSI is that a snapshot will be
taken over the formatted image. When using NFS you can use snapshots per
file. So this is why we're using NFS.

There are a lot of redbooks on both NFS and NetApps btw. So I should be
possible. Whether or not IBM supports it.


Thanks for the feedback anyway. If I find the cause of the errors, I'll
post it in a reply.

-R-

Jurgen Haan wrote:
> Knut Stolze wrote:
>> Jurgen Haan wrote:
>>
>>
>>> The server has anonyous rw access enabled, so everyone should be able to
>>> write.
>>>
>>> Just to be sure I changed to user db2inst and created a file on the
>>> share. No problem. I can create, alter and delete files on it.
>>
>>
>> I remember dimly that there were some issues with DB2 databases on
>> NFS. I
>> don't remember any details, but you should find something via Google.
>>
>> Also, the instances are created by "root". So root needs to have _all_
>> privileges on the NFS volume. In particular, root must be able to create
>> files, change owners and permissions. Some suid-files are created as
>> well,
>> and those should work as intended if your NFS configuration is
>> properly set
>> up. You don't use something like root-squash by any chance?
>>
>
> Nope.
> NFS is configure quite dumb.
>
> Everyone has full (root) access on the share.. Meaning they are not user
> root, but they can do anything they would normally by able to. The
> permissions are determined by the authentication on the Linux machine,
> not by any mechanism the NFS server. I am able to create files, delete
> files, alter files, create directories, delete directories, alter
> directories, chown, chmod, create pipes, delete pipes, make symlinks,
> delete symlinks. In short, everything.
>


Jurgen,

The fact that users can create and remove files has nothing to do with
what Darin was suggesting. root-squash is an option for NFS Servers
that PREVENTS root (UID=0) on an NFS client machine from doing
whatever it likes. Unfortunately I can't recall if NetApp has
implemented this, but it is a standard part of the NFS protocol, and
I don't have access to NetApp docs.

You should test creating a file in the NFS file system when you are
logged in *as root* on the DB2 server. If this doesn't work, then you
need to disable root-squash on the Filer and re-export your volumes.

I have used DB2 with NetApp filers in the past to support VLDB, and they
work fine. IBM and NetApp have a partnership, and IBM *does* support
DB2 running on Filers, although there are some recommendations.

However, we always had the instance owner file systems stored on local
disks, so we didn't run into this issue.


Good luck.