This is a discussion on Index management within the Oracle Database forums, part of the Database Server Software category; --> "Joel Garry" <joel-garry@home.com> wrote in message news:91884734.0405031221.3a68bcdf@posting.google.c om... > I think this extreme ignores the reality that so many ...
| |||||||
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| ||||
| "Joel Garry" <joel-garry@home.com> wrote in message news:91884734.0405031221.3a68bcdf@posting.google.c om... > I think this extreme ignores the reality that so many "typical > business systems" will still have times of batch processing. Even > with multiple, er, streams, of batches, whatever scheduling algorithm > is used will hopefully allow some chunks of time for each. <brain-storming> Yeah, but is that still a determining factor? I mean, maybe we should ignore fine tuning of batch and go all out for an even I/O performance across the board? Much simpler to keep going, no? > The idea is to account for common possibilities. SAME may be ok for > high-transactional times, but I think the allocation optimisation this > thread is talking about may make some difference during "other" times. Well, I think the thread is saying very clearly that index-level clustering and other "defrag" techniques are not that important as soon as multi-user is involved. Which is the vast majority of the time. Should we really bother with the odd "other" conditions? Does it really gain us much anymore? </brain-storming> > I'd rather have the tools to optimizer multiple layers for different > possible configurations, _and have the tools work together_. This > "trust us, we're experts" stuff from Oracle and MS is condescending > crap inevitably leading to stuff like NT defragmentation and > http://metalink.oracle.com/metalink/...p_id= 3455402 > We've seen over and over again that increasing the number and depth of > abstraction layers with the hope that Moore's law will bail us out > leads to putting stuff in the wrong layers. Amen! The problem is of course how to find a middle ground... > > a decade, when it first appeared in Sybase. I'm not sure it means anything > > in a NTFS environment where physical allocation is all over the place unless > > one takes precautions? Or with any modern SAN? > > I think predictive read-ahead could mean a lot in a modern SAN. Used to think so but not so sure nowadays. The SAN knows remarkably little about the db structure and allocation strategy. With something like ASM (or any SQL Server equivalent in future) it can all go horribly wrong. If "intelligence" is to be added in terms of predictive behaviour, I'd rather it was applied in one place only: either the SAN or the db. Mixing two "intelligences" is a recipe for disaster IME. Not sure yet what Oracle's recommendation on ASM is gonna be, but I suspect they'll eventually ask anyone relying on it to turn off all the "intelligent" bits everywhere else. -- Cheers Nuno Souto wizofoz2k@yahoo.com.au.nospam |
| |||
| "Noons" <wizofoz2k@yahoo.com.au> wrote in message news:40979aeb$0$4547$afc38c87@news.optusnet.com.au ... > Well, I think the thread is saying very clearly that index-level clustering > and other "defrag" techniques are not that important as soon as multi-user > is involved. Which is the vast majority of the time. Should we really bother > with the odd "other" conditions? Does it really gain us much anymore? > </brain-storming> Hi Nuno, Need to be a little careful here. Any techniques that can group or cluster like data together so that we can read this data by accessing fewer blocks is a good thing providing the benefits we gain are not counter-balanced by the costs of maintaining such structures. So for example as I mention in my presentation, if by re-ordering the data in a table to match the order of our most "important" index via business critical index range scan operations, we can dramatically reduce the number of different data blocks we need to visit, which could then noticeably improves important business response times, then that's a good thing. Same potentially goes for some clustered data structures. Like I said, it all depends on the benefits vs. the maintenance costs. Cheers Richard |
| |||
| "Richard Foote" <richard.foote@tbigpond.nospam.com> wrote in message news:naNlc.20002$TT.7962@news-server.bigpond.net.au... > > Well, I think the thread is saying very clearly that index-level > clustering >snippage > Any techniques that can group or cluster like data together so that we can > read this data by accessing fewer blocks is a good thing providing the > benefits we gain are not counter-balanced by the costs of maintaining such > structures. > > So for example as I mention in my presentation, if by re-ordering the data > in a table to match the order of our most "important" index via business > critical index range scan operations, we can dramatically reduce the number > of different data blocks we need to visit, which could then noticeably > improves important business response times, then that's a good thing. Same > potentially goes for some clustered data structures. Like I said, it all > depends on the benefits vs. the maintenance costs. Like I said: index-level clustering. Table-level clustering is a totally different kettle. -- Cheers Nuno Souto wizofoz2k@yahoo.com.au.nospam |
| |||
| "Noons" <wizofoz2k@yahoo.com.au> wrote in message news:<40979aeb$0$4547$afc38c87@news.optusnet.com.a u>... > "Joel Garry" <joel-garry@home.com> wrote in message > news:91884734.0405031221.3a68bcdf@posting.google.c om... > > > I think this extreme ignores the reality that so many "typical > > business systems" will still have times of batch processing. Even > > with multiple, er, streams, of batches, whatever scheduling algorithm > > is used will hopefully allow some chunks of time for each. > > <brain-storming> > Yeah, but is that still a determining factor? I mean, maybe we should > ignore fine tuning of batch and go all out for an even I/O performance > across the board? Much simpler to keep going, no? > > > > The idea is to account for common possibilities. SAME may be ok for > > high-transactional times, but I think the allocation optimisation this > > thread is talking about may make some difference during "other" times. > > Well, I think the thread is saying very clearly that index-level clustering > and other "defrag" techniques are not that important as soon as multi-user > is involved. Which is the vast majority of the time. Should we really bother > with the odd "other" conditions? Does it really gain us much anymore? > </brain-storming> Well, since those conditions are often prerequisite to the ones where the Big Bosses are staring at their lackeys and impatiently tapping their fingers, it gains in the "business requirements" arena. > > > > I'd rather have the tools to optimizer multiple layers for different > > possible configurations, _and have the tools work together_. This > > "trust us, we're experts" stuff from Oracle and MS is condescending > > crap inevitably leading to stuff like NT defragmentation and > > > http://metalink.oracle.com/metalink/...p_id= 3455402 > > We've seen over and over again that increasing the number and depth of > > abstraction layers with the hope that Moore's law will bail us out > > leads to putting stuff in the wrong layers. > > > Amen! > The problem is of course how to find a middle ground... > > > > > a decade, when it first appeared in Sybase. I'm not sure it means anything > > > in a NTFS environment where physical allocation is all over the place unless > > > one takes precautions? Or with any modern SAN? > > > > I think predictive read-ahead could mean a lot in a modern SAN. > > Used to think so but not so sure nowadays. The SAN knows remarkably > little about the db structure and allocation strategy. With something > like ASM (or any SQL Server equivalent in future) it can all go horribly > wrong. If "intelligence" is to be added in terms of predictive behaviour, > I'd rather it was applied in one place only: either the SAN or the db. > Mixing two "intelligences" is a recipe for disaster IME. > > Not sure yet what Oracle's recommendation on ASM is gonna be, but > I suspect they'll eventually ask anyone relying on it to turn off > all the "intelligent" bits everywhere else. I'd hope they'd allow something like hints to keep the benefits of everywhere else. Or even communication between everywhere else and ASM. jg -- @home.com is bogus. http://www.signonsandiego.com/uniont...ws_1b4ipo.html |
| |||
| that works! -- Cheers Nuno Souto wizofoz2k@yahoo.com.au.nospam "Joel Garry" <joel-garry@home.com> wrote in message news:91884734.0405041417.53eee4d@posting.google.co m... > > Well, since those conditions are often prerequisite to the ones where > the Big Bosses are staring at their lackeys and impatiently tapping > their fingers, it gains in the "business requirements" arena. |
| |||
| "Richard Foote" <richard.foote@tbigpond.nospam.com> wrote in message news:<yUIkc.6619$TT.2282@news-server.bigpond.net.au>... > "Howard J. Rogers" <hjr@dizwell.com> wrote in message > news:409334fa$0$25007$afc38c87@news.optusnet.com.a u... > > Richard Foote wrote: > > > > > > Go to the download section at www.actoug.org.au, make what you can of > the > > > PowerPoint presentation, wait for the accompanying whitepaper/book, read > it > > > carefully too and hopefully you'll get the picture. > > > > Is there an emoticon for a big, BIG, BIIIIIIGGGG beaming smiley face. > > Hi Howard, > > I'm not sure. What about > > > > > After having read the first slides containing the delicious quotes from > > Herr Burleson und seiner freunde, I had to go and bake a doughnut. > > The difficult part was selecting which quotes to use, there are so many. I > didn't want to single out any one person and I wanted to highlight how so > many have got it so wrong. So I focused on those "experts" that seem to > publish a lot of "stuff" ... > > > > > After that I had indigestion with your 50-50 leaf block claim. Jonathan > > claims it's 50-50 *on average*, but not by volume, I think. Interesting. > > No, it's definitely by volume. I showed this in a thread here a while back > where after a leaf block split, the number of index entries in each block > varied but the amount of used space sat at the 50% mark in each block. > > Look here: > http://groups.google.com/groups?q=g:...r=&ie=UTF-8&oe > =UTF-8&selm=ant%259.39604%24jM5.100537%40newsfeeds.bigp ond.com. where I > demonstrate my "volume" claim The white paper also has an example. > > It's the same thread whereby Don Burleson showed his total ignorance of how > indexes work and kinda opened my eyes that he wasn't quite what he claimed. > Interestingly, almost the entire history of Don Burleson posts have been > removed from the google archives, there's hardly a one left. As an example, > all the posts in this thread where he made a fool of himself are no longer > there, although thankfully for prosperity sake, his "words of wisdom" can > still be found in among the posts of others. It's all very interesting don't > you think, although it's probably a good move from Don's point of view to > reduce the embarrassment of some of the stuff his written. > > > > > However: I just wanna know why that Sharman guy gets all the invites and > > I don't :-(( > > > > A lot of it has to do with the fact he's a local lad who lives in the > neighbouring suburb from me !! However, our next User Group event is on 23 > June so if your interested in conducting a presentation in front of 60 odd > folks and would like a lovely free lunch to boot, please let me know and > I'll grab you a spot no worries at all !! > > Cheers > > Richard Richard, Roger, My what a mutual appreciation society! Warms my heart to see you two so cuddly. Richard, interesting presentation, of course as usual you ignore all the caveats and expansions that go with the expert quotes, rather reminds me of the fundamentalist bible thumpers we have in Georgia who take quotes and twist them to support all sorts of things...but I digress, I just wish you would give complete examples, of course it is always easier to tear someone else down than come up with original material. I try to never denigreate other folks in public, I may show why general old saws are bad, but I don't take the delite in castigating others they way you seem to! For example, speaking for myself, when I tell people to rebuild indexes I also say they should look at the type of index, how it is used, if it is on concatenated columns and several other factors. I also tell them to compare it to the number of dirty blocks in the table and if it is several times that value the index may benefit from rebuilding. I also tell them sometimes rebuilding will not improve the clustering factor and then, they should consider re-ordering the table data (hmmm...where have we heard that one from?) I have also showed papers and proofs that clustering factor is dramatically affected by column order (which should be intuitive) in a concatenated index and that since it is a major factor in cost calculations, reordering the columns in the indexes and the SQL it supports, can make a less attractive index more palatible. It is odd that in nearly every case where a rebuild has reduced clustering factor, performance is also improved. From all of your postings this should not be the case. It is also odd that when indexes show up very broad, they also usually have large clustering factors and, a high percentage of non-reused space. Yes, an index will reuse space, but not very often in a high IUD environment and a rebuild or coalesce will reduce these indexes in size, thus reducing clustering factor and blocks used, making them once again more palatible for the optimizer and give higher performance. Other than violating relational tenents, reordering table data will dramatically improve access times in many cases. Of course as the table ages it must be rebuilt over and over again, along with the index, to maintain this ordering. I hope those that follow this advice have lots of maintenance window time! Most of my clients barely have time to backup their terabyte size databases, let alone reorder all of their major, multi-million row tables. I heard that first from Don Burleson when I reviewed one of his first Oracle books. It was a carry over from Mainframe tuning that is still valid. Nice of you to give the same advice as Don! Anyway, I wish if you would quote my suggestions, you would also quote the qualifying material that goes with them, but that wouldn't be as much fun now would it? Mike |
| |||
| Mike Ault wrote: > Richard, Roger, Er, that would be "Howard", not "Roger". [snip] >I may show why general old saws are bad, but I don't take the > delite in castigating others they way you seem to! > > For example, speaking for myself, when I tell people to rebuild > indexes ....you make your first mistake, since the advice should be 'not to unless', not 'rebuild unless'. [snip] >I also say they should look at the type of index, how it is > used, if it is on concatenated columns and several other factors. I > also tell them to compare it to the number of dirty blocks in the > table and if it is several times that value the index may benefit from > rebuilding. That sentence doesn't even make sense to me. "I tell them to compare it [what "it" are we talking about? From the context, it sounds like it should be "the number of leaf nodes of the index"] and compare it to the number of dirty blocks in the table [if it's "in" the table, safely on disk, it's not a dirty block is it? Since I know you know that, I have to assume you mean 'blocks from the table currently in the buffer cache that have been dirtied'] and if it is several times that value [again, what value are we talking about? Index leaf node count?] the index may benefit from rebuilding." So if I've got this right, you're saying "if the number of table dirty buffers at any one moment is several times larger than the number of index nodes, a rebuild might be beneficial". But that is clearly a ludicrous proposition, so that can't be what you meant at all. I would like to know what exactly we're supposed to be comparing to what. >I also tell them sometimes rebuilding will not improve the > clustering factor and then, they should consider re-ordering the table > data (hmmm...where have we heard that one from?) > > I have also showed papers and proofs that clustering factor is > dramatically affected by column order (which should be intuitive) in a > concatenated index and that since it is a major factor in cost > calculations, reordering the columns in the indexes and the SQL it > supports, can make a less attractive index more palatible. I would suggest that changing the column order in a concatenated index is not so much an index rebuild as an index redesign. And of course column order is important, and if you get it wrong, you should of course change it. And I'm sure Richard wouldn't disagree with that either. > Anyway, I wish if you would quote my suggestions, you would also quote > the qualifying material that goes with them, but that wouldn't be as > much fun now would it? It would mean trying to squeeze one hell of a lot of material into a presentation (and apparently using mind-reading techniques to work out what some of it means in the first place)! If you don't like being quoted, then don't say it in the first place. If it is so easy to quote you "out of context", then it rather suggests you have been careless with your choice of words. If your published works told the facts of the matter, that an index rebuild hardly ever makes a difference, but that there are always exceptions, then I doubt anyone would be quoting you otherwise. HJR |
| |||
| Comments embedded mikerault@earthlink.net (Mike Ault) wrote in message news:<37fab3ab.0405051352.11555818@posting.google. com>... > > Richard, Roger, Mike, In your typical fashion, you can't even get the names right ... > > My what a mutual appreciation society! Warms my heart to see you two > so cuddly. I know it's lovely. Still, we have a long way to go to match the "closeness" between Don and yourself ... >Richard, interesting presentation, of course as usual you > ignore all the caveats and expansions that go with the expert quotes, > rather reminds me of the fundamentalist bible thumpers we have in > Georgia who take quotes and twist them to support all sorts of > things...but I digress, Not only do you digress , but you're of course utterly wrong (again). I haven't twisted anything, each quote stands up in it's own right (your's especially) and this attempt (again) to suggest otherwise and to try and deflect the issues is both a little tiresome and immature. > I just wish you would give complete examples, > of course it is always easier to tear someone else down than come up > with original material. I haven't torn anyone down, Mike, you're getting emotional again. I've simply highlighted how many of these silly Oracle myths are spread by many of these so-called experts, by giving very clear and concise examples. I purposely listed a range of quotes by a range of different authors to highlight how common this myth spreading is. Each quote is inaccurate no matter the context, as we'll look at later with your specific example. With respect to "original material", if you actually *read* the presentation, you'll find a great deal of it is original and based on subject matter that is not so well documented. I try to never denigreate other folks in > public, I may show why general old saws are bad, but I don't take the > delite in castigating others they way you seem to! Mike, you're getting emotional again. Highlighting common errors and misunderstandings and *correcting" them is not castigating anyone. It's a real shame that you find it so hard to understand that ... Dealing with constructive criticisms is not one of your strengths is it Mike. > > For example, speaking for myself, when I tell people to rebuild > indexes I also say they should look at the type of index, how it is > used, if it is on concatenated columns and several other factors. I > also tell them to compare it to the number of dirty blocks in the > table and if it is several times that value the index may benefit from > rebuilding. I also tell them sometimes rebuilding will not improve the > clustering factor and then, they should consider re-ordering the table > data (hmmm...where have we heard that one from?) Yes, let's speak about you for example. Your quote above about the "number of dirty blocks in the table" is a clear example of you not having a clue what you're talking about. Your other quote about "sometimes rebuilding will not improve the CF" is yet an example. Mike, please do me a favour and try out the following little test. Pick any index you want, any, compute stats so they're 100% accurate and note the CF in dba_indexes. Jot it down on a piece of paper. Then rebuild this index with any pctfree you like, compact it as much as you like. Then re-compute the stats and check out the new CF with the previous value. What do you see ? The values are identical. While you scratch your head and manually close your lower jaw, pick another index, and repeat the test. Keep repeating the test until you finally get the point that by simply rebuilding the index, you do not change the CF. Why is this ? Because the order of the index entries is unchanged, the order of the table remains unchanged, therefore if you only realized what the CF actually represents, you would see that the CF can therefore not change as the relative order of the *rows in the table* do not change with respect to the index. You've just confirmed in your post here that you clearly don't understand this but let's now look at your quote in my presentation: "Deleted space is not reclaimed automatically unless there is an exact match key inserted. This leads to index broadening and increase in the indexes clustering factor. You need to reorganize to reclaim white space. Generally rebuild index when the clustering factor exceeds eight times the number of dirty blocks in the base table, when the levels exceed two or when there are excessive brown nodes in the index." Looking at the quote "Generally rebuild index when the clustering factor exceeds eight times the number of dirty blocks in the base table", we can now see that it is entirely wrong, no matter in what *context* you want to look at it. Rebuilding an index based purely on the value of the CF as you describe is rubbish, the CF remains unchanged. Now, let's look at the other parts of the quote: "Deleted space is not reclaimed automatically unless there is an exact match key inserted." is rubbish, again no matter the context. My presentation clearly demonstrates why this is the case. "This leads to index broadening and increase in the indexes clustering factor" is rubbish, again no matter the context. Having deleted or wasted space does not increase the CF. Inserting (or migrating) rows so that the order becomes less aligned *in the table* in relation to the ordering of the index is how the CF degrades. "You need to reorganize to reclaim white space" is in most cases wrong. My presentation describes those rare scenarios where excessive white space may be an issue but the suggestion that you "need" to reorganise to reclaim white space is wrong. "Generally rebuild index when the clustering factor exceeds eight times the number of dirty blocks in the base table, when the levels exceed two or when there are excessive brown nodes in the index" is wrong, wrong wrong. The CF bit we've discussed, totally wrong. The exceeds 2 levels myth is covered in the presentation. Why you recommend rebuilding *all* your large indexes again and again for no benefit is beyond me. Simple fact. Large indexes require more levels and if by rebuilding these larger indexes you don't reduce the level, why bother. The excessive brown nodes is discussed in the presentation where it shows how this space is reusable. So really Mike, your quote, no matter the *context*, doesn't really stand up at all well does it ... > > I have also showed papers and proofs that clustering factor is > dramatically affected by column order (which should be intuitive) in a > concatenated index and that since it is a major factor in cost > calculations, reordering the columns in the indexes and the SQL it > supports, can make a less attractive index more palatible. Yes Mike but aren't you clouding the waters here. This is dropping and re-creating a totally *different* index, which is hardly the same thing as recommending the rebuilding of an existing index so that you can end up with the same CF ... > > It is odd that in nearly every case where a rebuild has reduced > clustering factor, performance is also improved. Really odd I would say as rebuilding an index doesn't effect the CF !! The old "performance has definitely improved although I can't really tell why, or precisely by how much ..." And you *still* promote this rubbish. How many books have you written .... >From all of your > postings this should not be the case. If you *actually read* my presentation you'll see that an improved CF most definitely can improve performance. Unfortunately, you have no clue how to improve the CF ... >It is also odd that when indexes > show up very broad, they also usually have large clustering factors > and, a high percentage of non-reused space. Mike, suggesting there is a co-relation between the size of an index and the *effective* CF is wrong wrong wrong. Yes, large tables have larger "values" for the CF because they obviously have more rows. However, large indexes can have a perfect CF (equal to the number of blocks in the table) and small indexes can have an awful CF (equal to the number of rows in the table). Mike, I have the very very strong suspicion you have no idea what the CF represents and how it's the order of the *table* in relation to the index that governs the CF, pure and simple. My suspicion is right isn't it ? >Yes, an index will reuse space, Didn't you say only if the new index value is the same ? I'm sure you did, oh, look at your quote ... That's funny you're got to admit !! >but not very often in a high IUD environment Why is that Mike ? Actually read my presentation first, then answer my question, it might just change your mind ... > and a rebuild or > coalesce will reduce these indexes in size, thus reducing clustering Mike, this is rubbish, we've been through this. The fact you get this wrong again and again kinda highlights your lack of understanding here .... Was that 10 books Mike ... > factor and blocks used, making them once again more palatible for the > optimizer and give higher performance. All sounds very good and practical in these performance books you write. Unfortunately, the truth is somewhat different to your sense of reality. *Read* my presentation to learn and find out why. Perhaps I should have my presentation published ... > > Other than violating relational tenents, reordering table data will > dramatically improve access times in many cases. Of course as the > table ages it must be rebuilt over and over again, along with the > index, to maintain this ordering. I hope those that follow this advice > have lots of maintenance window time! Most of my clients barely have > time to backup their terabyte size databases, let alone reorder all of > their major, multi-million row tables. Of course rebuilding a table is not something one can do at the drop of the hat, if at all. But at least by rebuilding and re-ordering the table, you *do* influence the CF. >I heard that first from Don > Burleson when I reviewed one of his first Oracle books. It was a carry > over from Mainframe tuning that is still valid. Nice of you to give > the same advice as Don! When Don is right, I agree with him. When Don is wrong, I don't agree with him. For some reason, you have a problem with that. > > Anyway, I wish if you would quote my suggestions, you would also quote > the qualifying material that goes with them, but that wouldn't be as > much fun now would it? As I've explained, your quote in whatever *context* you like to place it in, is utter tripe. Pure and simple. My presentation explains, as I've briefly attempted to here, why that is the case. Mike, why don't you get off your little pedestal of how you are sooooooo good and how I'm soooooo bad, and simply thank me for pointing out why you are so utterly wrong and for teaching you how indexes and the CF in particular work in Oracle. You might even mention me in your next book for showing you the light (thought I'll use a biblical example too). Be a nice touch don't you think. One last point. Just sit back and think about all the people you've taught over the years and who have read your book and who now wrongly think that deleted index entries are only reused if identical values are inserted, who rebuild indexes in the naive hope that the CF will improve, who rebuild all their massive indexes again and again in the hope they will drop a level, who think that a large value for the CF is bad simply because the table is large, etc. etc. etc. And yet Mike, you think *I'm* the bad guy ... Now that really is funny. Richard |
| |||
| danielroy10junk@hotmail.com (Daniel Roy) wrote in message news:<3722db.0404301139.677f5219@posting.google.co m>... > You sound so much like an Oracle 7 DBA! First take a look at > www.jlcomp.demon.co.uk/faq/table_frag.html, then take a look at the > referenced Steve Adam's paper if you want more details. <snip> Hi Daniel, looks like I have at least one reader then Cheers, Norman. |
| ||||
| "Richard Foote" <richard.foote@tbigpond.nospam.com> wrote in message news:<5Vtkc.5521$TT.227@news-server.bigpond.net.au>... > > Go to the download section at www.actoug.org.au, make what you can of the > PowerPoint presentation, wait for the accompanying whitepaper/book, read it > carefully too and hopefully you'll get the picture. > Hi Richard, in fairness to at l;east one of the quotes experts, Jon Wang, when I saw his article on DBAZine, I wrote up and sent him, and DBAzine, an article proving without doubt that deleted entries in an index are reused as soon as an attempt is made to write any entry that fits into the block(s) in question. John replied and acknowleged his error and promised a following article would correct his information. DBAzine replied too and said that when they had heard back from John, they would probably publish a correction. As yet, nothing Cheers, Norm. (Behind the firewall from hell !) |