SEO

Well, they're a mystery to me.

Here's the list:

/var/cache/edb/dep/*

"dep" suggests dependencies.

Now, a pair of 35Mb files:

/var/cache/edb/metadata.idx.most_recent
/var/cache/edb/remote_metadata.pickle

Also:

/var/cache/edb/config
/var/cache/edb/mtimedb
/var/cache/edb/counter

Finally:

/var/db/pkg/*

Another 23Mb of files. That may be the record
of packages that have been installed.

Regarding the size of my portage tree: using
a loopback device with a block size of 1024
reduces the size of my portage tree by 265Mb.
There must be a more elegant solution.

<begin rant>

Please convince me that the portage system
is not a sprawling suboptimal mess! A more
thorough knowledge of it may dispel that
impression, but I'm beginning to doubt it.

Surely there is no need for a portage tree
containing more than one hundred thousand
files, most of which contain details of
packages I will never even think about
installing. And then there's the scattering
of other portage files all over the place.

I'm trying very hard to like portage, but
it reeks of ad hoc incrementalism. I can't
help but think that a plane designed by
Gentoo would have several dozen wings,
and enough electronic equipment to fill
half the passenger cabin.

<end rant>

I feel much better now, thanks

R.

On 2004-10-24, Rolleston <rolleston@tiscali.co.uk> wrote:
> Well, they're a mystery to me.
>
> Here's the list:
>
> /var/cache/edb/dep/*
>
> "dep" suggests dependencies.
>
> Now, a pair of 35Mb files:
>
> /var/cache/edb/metadata.idx.most_recent
> /var/cache/edb/remote_metadata.pickle
>

This is your metadata cache generated by Portage. Unless you're using
the dbm backend module for the metadata cache (which I am) you can view
the files in /var/cache/edb/dep/ for some idea of what's stored there.

> Also:
>
> /var/cache/edb/config
> /var/cache/edb/mtimedb
> /var/cache/edb/counter
>
> Finally:
>
> /var/db/pkg/*
>
> Another 23Mb of files. That may be the record
> of packages that have been installed.
>

Yep.

> Regarding the size of my portage tree: using
> a loopback device with a block size of 1024
> reduces the size of my portage tree by 265Mb.
> There must be a more elegant solution.
>

Reiserfs may help with actual space usage with a large number of small
files like the Portage tree.

><begin rant>
>
> Please convince me that the portage system
> is not a sprawling suboptimal mess! A more
> thorough knowledge of it may dispel that
> impression, but I'm beginning to doubt it.
>
> Surely there is no need for a portage tree
> containing more than one hundred thousand
> files, most of which contain details of
> packages I will never even think about
> installing. And then there's the scattering
> of other portage files all over the place.

Do you have an alternate solution to present?

People throw around suggestions like download-on-demand for the Portage
tree, but I haven't seen much for actual code or decent ways to handle
things like dependency & metadata consistency when the tree is not
present on disk. (There was a discussion on the gentoo-dev mailing list
about this recently. Check archives.)

As for other Portage files, they're actually organized rather well as of
Portage 2.0.51. 2.0.50 had nondeletable files in /var/cache/edb and
nastiness like that. With 2.0.51, things you don't want to delete are in
/var/lib/portage, /var/cache can be regenerated, and virtuals are
calculated on the fly rather than relying on the now-deprecated
/var/cache/edb/virtuals. We still continue the BSD tradition of using
/var/db/pkg.

Which files are "scattered all over the place"?

--
Jon Portnoy
avenj/irc.freenode.net, irc.oftc.net

Jon Portnoy wrote:
> On 2004-10-24, Rolleston <rolleston@tiscali.co.uk> wrote:
[...]
>> Now, a pair of 35Mb files:
>>
>> /var/cache/edb/metadata.idx.most_recent
>> /var/cache/edb/remote_metadata.pickle
>>
>
> This is your metadata cache generated by Portage.

Thanks. It doesn't appear to be particularly well documented,
unless I'm missing something. Google says:

Your search - remote_metadata.pickle - did not match any documents.
No pages were found containing "remote_metadata".

Having control over your system, which is surely what Gentoo
is about, is difficult unless one knows _exactly_ what portage
is meant to do. Is there a complete formal specification of the
portage system online, or is the code, erm, self-documenting?

>> Regarding the size of my portage tree: using
>> a loopback device with a block size of 1024
>> reduces the size of my portage tree by 265Mb.
>> There must be a more elegant solution.
>>
>
> Reiserfs may help with actual space usage with a large number of small
> files like the Portage tree.

Yes. I wish I'd known that when I installed Gentoo. Well,
I can live without the space for the moment. I just don't
like the idea that the system is cluttered up with unused
files and directories. It's the ugliness of it that I don't
care for.

>> Surely there is no need for a portage tree
>> containing more than one hundred thousand
>> files, most of which contain details of
>> packages I will never even think about
>> installing. And then there's the scattering
>> of other portage files all over the place.
>
> Do you have an alternate solution to present?
>
> People throw around suggestions like download-on-demand for the Portage
> tree, but I haven't seen much for actual code or decent ways to handle
> things like dependency & metadata consistency when the tree is not
> present on disk. (There was a discussion on the gentoo-dev mailing list
> about this recently. Check archives.)

Well, most of my portage tree just seems to be a copy
of what occurs elsewhere, and is not altered when I
emerge something. Presumably, this part of the tree
could be read from another machine? The immeasurable
intellects of the Gentoo developers will surely not
crumble before this challenge.

I imagine a room of your house stuffed to the ceiling
with a copy of every phone directory in the world.
You know, how could one possibly manage without them all?

Cheers!

R.

Rolleston wrote:
>> Rolleston wrote:
>
> Having control over your system, which is surely what Gentoo
> is about, is difficult unless one knows _exactly_ what portage
> is meant to do. Is there a complete formal specification of the
> portage system online, or is the code, erm, self-documenting?

http://www.gentoo.org/doc/en/index.xml#doc_chap3

Here's how I found it:
http://www.gentoo.org/
Select "Docs"
Select "Gentoo System Documentation"

> I just don't
> like the idea that the system is cluttered up with unused
> files and directories. It's the ugliness of it that I don't
> care for.

Beauty is in the eye of the beholder. Perhaps with time you'll find that
there isn't half as many "unused" files and directories as you thought.

--
Ben M.

Ben Measures wrote:
> http://www.gentoo.org/doc/en/index.xml#doc_chap3
>
> Here's how I found it:
> http://www.gentoo.org/
> Select "Docs"
> Select "Gentoo System Documentation"

I've been there. It's fine, as far as it goes, but it does not
go far enough. It's incomplete, and, if I recall correctly,
"remote_metadata.pickle", and some other files, don't get a
mention. Google cannot find anything, and Google's memory is
far superior to mine.

I don't want to sound too harsh. I like Gentoo. If I didn't,
I wouldn't be using it. However, I'm not going to go around
praising every aspect of it.

>> I just don't
>> like the idea that the system is cluttered up with unused
>> files and directories. It's the ugliness of it that I don't
>> care for.
>
> Beauty is in the eye of the beholder.

I wouldn't know. I'm not an apiarist.

> Perhaps with time you'll find that
> there isn't half as many "unused" files and directories as you thought.

"Aren't", Mr Measures, "aren't". Sheesh

How many of those hundred thousand files do you think I'd need
to maintain a reasonably compact system? If I need them all,
the situation's even worse than I thought...

Cheers,

R.